Ettin-SPLADE Fine-tuning

Distills naver/splade-code-06B (600M) into jhu-clsp/ettin-encoder-17m (17M) for StackOverflow code search using MarginMSE knowledge distillation.

Setup

pip install -r requirements.txt

or

pip install torch transformers datasets scikit-learn accelerate sentencepiece

Run

Smoke test — verify it works before committing:

python train_mvp.py --epochs 1 --batch_size 2 --max_triplets 100

Full training — runs 3 epochs, saves checkpoints to ./checkpoints/:

python train_mvp.py --epochs 3 --batch_size 16 --max_triplets 50000

Cloud GPU via Modal — recommended, runs on A10G and auto-evals:

pip install modal && modal setup
modal run run_modal.py

Evaluate

# quick check
python eval/evaluate.py --limit_queries 10 --limit_corpus 1000

# full benchmark (needs GPU)
python eval/evaluate.py --model_path ./checkpoints/epoch_3

Results are printed to console and saved to eval/evaluation_report.md.