README
rank1-run-files: Pre-computed Run Files for Reranking Evaluation
📄 Paper | 🚀 GitHub Repository
This dataset contains pre-computed run files used by the rank1 family of models on various retrieval benchmarks. These files are what were used for top-k rereranking and also include the re-annotated DL19 qrels. These files are needed to download to reproduce our results.
Benchmarks Included
The dataset includes run files for the following benchmarks:
- BEIR (multiple datasets including NFCorpus, SciFact, etc.)
- NevIR
- TREC-DL 2019
- BRIGHT
Associated Models and Resources
| Resource | Description |
|---|---|
| rank1-7b | Base rank1 model (7B parameters) |
| rank1-14b | Larger rank1 variant (14B parameters) |
| rank1-32b | Largest rank1 variant (32B parameters) |
| rank1-mistral-2501-24b | Mistral-based rank1 variant (24B parameters) |
| rank1-llama3-8b | Llama 3.1-based rank1 variant (8B parameters) |
| rank1-r1-msmarco | All R1 output examples from MS MARCO |
| rank1-training-data | Training data used for rank1 models |
Citation
If you use these run files in your research, please cite:
@misc{weller2025rank1testtimecomputereranking,
title={Rank1: Test-Time Compute for Reranking in Information Retrieval},
author={Orion Weller and Kathryn Ricci and Eugene Yang and Andrew Yates and Dawn Lawrie and Benjamin Van Durme},
year={2025},
eprint={2502.18418},
archivePrefix={arXiv},
primaryClass={cs.IR},
url={https://arxiv.org/abs/2502.18418},
}
License
Contributors
2 commits
1 commits
Similar repos
Linked in READMEs
README
rank1-run-files: Pre-computed Run Files for Reranking Evaluation
📄 Paper | 🚀 GitHub Repository
This dataset contains pre-computed run files used by the rank1 family of models on various retrieval benchmarks. These files are what were used for top-k rereranking and also include the re-annotated DL19 qrels. These files are needed to download to reproduce our results.
Benchmarks Included
The dataset includes run files for the following benchmarks:
- BEIR (multiple datasets including NFCorpus, SciFact, etc.)
- NevIR
- TREC-DL 2019
- BRIGHT
Associated Models and Resources
| Resource | Description |
|---|---|
| rank1-7b | Base rank1 model (7B parameters) |
| rank1-14b | Larger rank1 variant (14B parameters) |
| rank1-32b | Largest rank1 variant (32B parameters) |
| rank1-mistral-2501-24b | Mistral-based rank1 variant (24B parameters) |
| rank1-llama3-8b | Llama 3.1-based rank1 variant (8B parameters) |
| rank1-r1-msmarco | All R1 output examples from MS MARCO |
| rank1-training-data | Training data used for rank1 models |
Citation
If you use these run files in your research, please cite:
@misc{weller2025rank1testtimecomputereranking,
title={Rank1: Test-Time Compute for Reranking in Information Retrieval},
author={Orion Weller and Kathryn Ricci and Eugene Yang and Andrew Yates and Dawn Lawrie and Benjamin Van Durme},
year={2025},
eprint={2502.18418},
archivePrefix={arXiv},
primaryClass={cs.IR},
url={https://arxiv.org/abs/2502.18418},
}
License
Similar repos
Linked in READMEs
Contributors
2 commits
1 commits