tuskanny/fiqa_lateon

Dataset

FiQA-2018, LateOn

3 commits

1 linked in READMEs

updated Sep 23, 2026

See the code

README

FiQA-2018, LateOn

Token-level (late-interaction) embeddings of the BEIR FiQA-2018 corpus and queries, encoded with LateOn, in the TACHIOM multivector format.

Source

BEIR FiQA-2018, test split. Corpus, queries and qrels were read from the official BEIR files via ir_datasets (beir/fiqa/test); PyLate only did the encoding
57,638 documents, 648 queries, 1,706 qrels
Text given to the encoder for each document: the passage text (FiQA documents have no title). The text itself is not included, only its vectors
Row order follows the BEIR corpus and query files; row i of doc_ids.npy / queries_ids.npy identifies row i of doclens.npy / queries.npy

Encoding

Model: lightonai/LateOn @ 62911e105059585d244384c7d17826e35f669c17
Library: PyLate 1.6.0, CPU
Document length cap: 300 tokens (model default)
Query length: 32 tokens (model default)
Query expansion: no (model default)
The model defaults come from config_sentence_transformers.json in the model repository. We did not override any of them
Documents: punctuation tokens are dropped (PyLate skiplist), as are padding tokens
Vectors: 128-d, L2-normalized

Statistics


Token vectors (N)	7,695,260
Avg vectors per document	133.5 (max 300)
Vectors per query	variable, 7 to 32 (no query expansion), zero-padded to 32
Avg vectors per query	16.7

Files

File	dtype	Shape	Content
`documents.npy`	float16 (`<f2`)	`[7695260, 128]`	All document vectors, concatenated document by document
`doclens.npy`	int32	`[57638]`	Vectors per document; `sum == N`
`token_ids.npy`	uint32	`[7695260]`	Input token id of each row of `documents.npy`
`doc_ids.npy`	string	`[57638]`	BEIR doc id of each document
`queries.npy`	float32	`[648, 32, 128]`	Query vectors, zero-padded at the end
`query_lens.npy`	int32	`[648]`	True number of vectors per query
`queries_ids.npy`	string	`[648]`	BEIR query id of each query
`qrels.test.tsv`	TREC	1706 lines	`qid \t 0 \t docid \t relevance`
`groundtruth/gt_top100.tsv`	TSV	64800 lines	Exhaustive top-100: `query_idx \t doc_idx \t rank \t score` (0-based positions)
`groundtruth/gt_ids.npy`	int32	`[648, 100]`	Same, as doc positions
`groundtruth/gt_scores.npy`	float32	`[648, 100]`	Same, MaxSim scores

Zero padding does not change any score: a zero query vector adds 0 to the MaxSim of every document.

Exhaustive-search effectiveness

Exact MaxSim over the full collection (vectorium compute_groundtruth_multivec). These are the reference numbers for approximate search on this data.

nDCG@10	R@100
0.5250	0.8353

beir

colbert

fiqa

late-interaction

multivector

Similar repos

tuskanny/fiqa_colbertv2

Dataset

FiQA-2018, ColBERTv2

tuskanny/nfcorpus_lateon

Dataset

NFCorpus, LateOn

tuskanny/fiqa_modern_colbert

Dataset

FiQA-2018, GTE-ModernColBERT

robro612/fiqa_mlateon

Dataset

fiqa_mlateon

tuskanny/scidocs_lateon

Dataset

SCIDOCS, LateOn

robro612/fiqa_lateon_hpool_regularized

Dataset

fiqa_lateon_hpool_regularized

robro612/scifact_lateon

Dataset

scifact_lateon

robro612/nfcorpus_mlateon

Dataset

nfcorpus_mlateon

Linked in READMEs

TusKANNy/awesome-multivector-retrieval

An extensive and commented list of resources on Late-Interaction Multivector Retrieval.

README badge

tuskanny/fiqa_lateon

Dataset

FiQA-2018, LateOn

3 commits

1 linked in READMEs

updated Sep 23, 2026

See the code

README

FiQA-2018, LateOn

Token-level (late-interaction) embeddings of the BEIR FiQA-2018 corpus and queries, encoded with LateOn, in the TACHIOM multivector format.

Source

BEIR FiQA-2018, test split. Corpus, queries and qrels were read from the official BEIR files via ir_datasets (beir/fiqa/test); PyLate only did the encoding
57,638 documents, 648 queries, 1,706 qrels
Text given to the encoder for each document: the passage text (FiQA documents have no title). The text itself is not included, only its vectors
Row order follows the BEIR corpus and query files; row i of doc_ids.npy / queries_ids.npy identifies row i of doclens.npy / queries.npy

Encoding

Model: lightonai/LateOn @ 62911e105059585d244384c7d17826e35f669c17
Library: PyLate 1.6.0, CPU
Document length cap: 300 tokens (model default)
Query length: 32 tokens (model default)
Query expansion: no (model default)
The model defaults come from config_sentence_transformers.json in the model repository. We did not override any of them
Documents: punctuation tokens are dropped (PyLate skiplist), as are padding tokens
Vectors: 128-d, L2-normalized

Statistics


Token vectors (N)	7,695,260
Avg vectors per document	133.5 (max 300)
Vectors per query	variable, 7 to 32 (no query expansion), zero-padded to 32
Avg vectors per query	16.7

Files

File	dtype	Shape	Content
`documents.npy`	float16 (`<f2`)	`[7695260, 128]`	All document vectors, concatenated document by document
`doclens.npy`	int32	`[57638]`	Vectors per document; `sum == N`
`token_ids.npy`	uint32	`[7695260]`	Input token id of each row of `documents.npy`
`doc_ids.npy`	string	`[57638]`	BEIR doc id of each document
`queries.npy`	float32	`[648, 32, 128]`	Query vectors, zero-padded at the end
`query_lens.npy`	int32	`[648]`	True number of vectors per query
`queries_ids.npy`	string	`[648]`	BEIR query id of each query
`qrels.test.tsv`	TREC	1706 lines	`qid \t 0 \t docid \t relevance`
`groundtruth/gt_top100.tsv`	TSV	64800 lines	Exhaustive top-100: `query_idx \t doc_idx \t rank \t score` (0-based positions)
`groundtruth/gt_ids.npy`	int32	`[648, 100]`	Same, as doc positions
`groundtruth/gt_scores.npy`	float32	`[648, 100]`	Same, MaxSim scores

Zero padding does not change any score: a zero query vector adds 0 to the MaxSim of every document.

Exhaustive-search effectiveness

Exact MaxSim over the full collection (vectorium compute_groundtruth_multivec). These are the reference numbers for approximate search on this data.