veya2ztn/sci-bert-finetune

Fineturn your embedding model for science paper

0

stars

2

commits

Python

primary language

Sep 24, 2024

updated

README

This is the embedding training code for the scientific embedding project: [2405.11461] DocReLM: Mastering Document Retrieval with Language Model (arxiv.org).

The dataset is from the Synthetic data build via veya2ztn/Synthetic-Science: Those script try to create Synthetic Science QA answer-question pair efficiently and reasoning (github.com) based on veya2ztn/uparxive: llm-friendly dataest for the whole arxiv .tex source. (github.com)

llm_train

This repo integrate embedder training method

ART
SGPT
Finetune
- Pipline training
- Tensor Parallel: 1D, 2D and so on
Gradient Cache
Qlora
Uniem

Contributors

2 commits

Similar repos

adrianlyjak/tinyllama-embed

1

manveertamber/enhancing_domain_adaptation

2

manveertamber/cadet-dense-retrieval

5

rainbow-brain/LLM_Embedding_Finetuning

LLM을 활용하기 위한 Embedding 모델과, LLM 모델의 finetuning 하는 방법을 모은 곳

0

HoweChen0517/finetuning-bert-for-IR

0

LongxingTan/open-retrievals

All-in-One: Text Embedding, Retrieval, Reranking and RAG in Transformers

71

NovaSearch-Team/RAG-Retrieval

Unify Efficient Fine-tuning of RAG Retrieval, including Embedding, ColBERT, ReRanker.

1,129

SeanLee97/AnglE

Train and Infer Powerful Sentence Embeddings with AnglE | 🔥 SOTA on STS and MTEB Leaderboard

573

Languages

Python

59.0%

Jupyter Notebook

36.0%

Shell

2.8%

C++

1.9%

veya2ztn/sci-bert-finetune

Fineturn your embedding model for science paper

0

stars

2

commits

Python

primary language

Sep 24, 2024

updated

README

This is the embedding training code for the scientific embedding project: [2405.11461] DocReLM: Mastering Document Retrieval with Language Model (arxiv.org).

The dataset is from the Synthetic data build via veya2ztn/Synthetic-Science: Those script try to create Synthetic Science QA answer-question pair efficiently and reasoning (github.com) based on veya2ztn/uparxive: llm-friendly dataest for the whole arxiv .tex source. (github.com)

llm_train

This repo integrate embedder training method

ART
SGPT
Finetune
- Pipline training
- Tensor Parallel: 1D, 2D and so on
Gradient Cache
Qlora
Uniem

Similar repos

adrianlyjak/tinyllama-embed

1

manveertamber/enhancing_domain_adaptation

2

manveertamber/cadet-dense-retrieval

5

rainbow-brain/LLM_Embedding_Finetuning

LLM을 활용하기 위한 Embedding 모델과, LLM 모델의 finetuning 하는 방법을 모은 곳

0

HoweChen0517/finetuning-bert-for-IR

0

LongxingTan/open-retrievals

All-in-One: Text Embedding, Retrieval, Reranking and RAG in Transformers

71

NovaSearch-Team/RAG-Retrieval

Unify Efficient Fine-tuning of RAG Retrieval, including Embedding, ColBERT, ReRanker.

1,129

SeanLee97/AnglE

Train and Infer Powerful Sentence Embeddings with AnglE | 🔥 SOTA on STS and MTEB Leaderboard

573

Contributors

2 commits

Languages

Python

59.0%

Jupyter Notebook

36.0%

Shell

2.8%

C++

1.9%