Embedding 학습을 위한 공부
P
paper Lee
공부해야할 것들을 적어두기 위한 페이지입니다.
더 봐야하거나 잘못된 부분이 있다면 언제든지 태클걸어주세요!
Model & Train
Sparse
•
BM25
•
BM25F
•
BM42
Cross-Encoder (Rerank)
•
How..?
Bi-Encoder (Embedding)
•
BERT
•
RoBERTa
◦
XLM-RoBERTa (bge3 계열 모델은 위 모델 참조함)
•
DeBERTa
•
ColBERT
◦
What is Late Interaction?
◦
Multi Vector with MUVERA (FDE)
•
ModernBERT
◦
SKT A.X Encoder
•
E5, BGE-m3, mE5
Hybrid?
•
SPLADE
◦
SPLADE는 Tokenizer 영향을 많이 받는 것 같다함
◦
telepix에서도 a.x Encoder를 사용했는데, tokenizer가 좋아서 사용하는 듯
Loss
•
InfoNCE Loss
•
GISTEmbedLoss
•
Cached_contrastiveLoss (Pylate)
Train
•
pylate, Sentence-Transformer, Transformers, etc...
•
DAPT / TAPT
•
RetroMAE
Inference
•
Text Embedding Inference (TEI, Huggingface)
Method
•
Martryoshka Representation Learning
Benchmark
Task : IR, STS, NLI, Clustering
Evaluation Metric
•
Precision & Recall & F1
•
MAP(Mean Average Precision)
•
DCG
•
nDCG(normalized DCG) @K
•
MRR (Mean Reciprocal Rank)
Benchmark
•
MTEB (Embedding Benchmark)
◦
MultiLongDocRetrieval
◦
BelebeleRetrieval
◦
PublicHealthQA
◦
Ko-StrategyQA
◦
MSMARCO
◦
MIRACL
ETC
•
FlagEmbedding for BAAI
•
PAQ
•
COIL
•
XTR
Pa
Suscribirse a 'Paperl'
Si te suscribes al sitio, podrás recibir primero las últimas novedades, como nuevas publicaciones, mediante notificaciones y correo electrónico.
¡Regístrate en Slashpage y suscríbete a 'Paperl'!
¡Regístrate en Slashpage y suscríbete a 'Paperl'!
Suscribirse
