# Embedding 학습을 위한 공부

공부해야할 것들을 적어두기 위한 페이지입니다.

> 더 봐야하거나 잘못된 부분이 있다면 언제든지 태클걸어주세요!

# Model & Train

## Sparse

- BM25

- BM25F

- BM42 

## Cross-Encoder (Rerank)

- How..?

## Bi-Encoder (Embedding)

- BERT

- RoBERTa

    - XLM-RoBERTa (bge3 계열 모델은 위 모델 참조함)

- DeBERTa

- ColBERT

    - What is Late Interaction?

    - Multi Vector with MUVERA (FDE)

- ModernBERT

    - SKT A.X Encoder

- E5, BGE-m3, mE5

### Hybrid?

- SPLADE

    - SPLADE는 Tokenizer 영향을 많이 받는 것 같다함

    - telepix에서도 a.x Encoder를 사용했는데, tokenizer가 좋아서 사용하는 듯

## Loss

- InfoNCE Loss

- GISTEmbedLoss

- Cached_contrastiveLoss (Pylate)

## Train

- pylate, Sentence-Transformer, Transformers, etc...

- DAPT / TAPT

- RetroMAE

## Inference

- Text Embedding Inference (TEI, Huggingface)

### Method

- Martryoshka Representation Learning

# Benchmark

Task : IR, STS, NLI, Clustering

### Evaluation Metric

- Precision & Recall & F1

- MAP(Mean Average Precision)

- DCG

- nDCG(normalized DCG) @K

- MRR (Mean Reciprocal Rank)

### Benchmark

- MTEB (Embedding Benchmark)

- [K-MTEB (Korean Embedding Benchmark)](https://github.com/nlpai-lab/KURE)

    - [Include Rerank (by. BM-K)](https://github.com/BM-K/Korean-MTEB-Retrieval-Evaluators)  

    - MultiLongDocRetrieval

    - BelebeleRetrieval

    - PublicHealthQA

    - Ko-StrategyQA

    - MSMARCO

    - MIRACL

- [AutoRAGEmbeddingBench](https://github.com/Marker-Inc-Korea/AutoRAG-example-korean-embedding-benchmark)

- [BEIR (Zeroshot)](https://github.com/beir-cellar/beir)

## ETC

- FlagEmbedding for BAAI

- PAQ

- COIL

- XTR

For the site tree, see the [root Markdown](https://slashpage.com/paperl.md).