Quick intro to the repo:

1. Main files:

.
├─ generate_data/          # offline helpers
│  ├─ generate_embeddings.py
│  └─ concat_embeddings.py
├─ models_pth/             # saved decoders  (auto-created)
├─ embeddings_data/        # per-model vectors (auto-created)
├─ results/                # MTEB scores      (auto-created)
├─ eval.py
├─ train.py
├─ model.py                # decoder defs
└─ config.py               # default hparams

2. Quick workflow:

# 1) Build embeddings ---------------------------------------------
python generate_data/generate_embeddings.py e5-small

# 2) (Optional) concatenate several models ------------------------
python generate_data/concat_embeddings.py      # edit model_keys/output inside

# 3) Train a decoder ----------------------------------------------
python train.py                                # edit tag + hparams inside

# 4) Benchmark on MTEB --------------------------------------------
python eval.py e5-small NFCorpus 0 x 0 0 x demo-run

3. Main files:

• eval.py: evaluate your models (raw concatenated, or with a decoder):

# CLI syntax
python eval.py <model_type> <mteb_task> \
               <use_encoder:0|1> <ckpt_tag_or_x> \
               <truncate_len> \
               <use_quant:0|1> <quantizer_path_or_x> \
               <run_tag> [use_lsh_flag]

• train.py – trains a new decoder

  1. Prepare data
     Put train_embeddings.pth and val_embeddings.pth in
     generate_data/<MY_TAG>/.

  2. Choose architecture
     Open model.py and pick/implement the decoder you want
     (SimpleEncoder, SparseAutoencoder, etc.).

  3. Set hyper-parameters

     • Global defaults live in config.py (batch size, LR, epochs, …).
     • The list of sub-dimensions to try (COMPRESSED_DIMENSIONS) is inside train.py.

  4. Launch training

     python train.py 
     

  Checkpoints will be saved to models_pth/<input_dim>_<output_dim>/ and can be used straight into eval.py.

• generate_data/: to generate and concat new combination of embeddings!

  1. Generate new embeddings for a single model: python generate_embeddings.py <model_key>
  2. Merge (concatenate + L2-normalise) the embeddings of several models with concat_embeddings.py.