Dataset Summary

This dataset contains the data for personalized passkey retrieval task in the paper Improving Text Embeddings with Large Language Models.

Data Fields

• query: a string feature.
• candidates: List of string feature, 100 candidates for each query.
• label: a int32 feature, the index of the correct candidate in the candidates list, always 0.
• context_length: a int32 feature, the approximate length for the candidate documents.

How to use this dataset

You can load the dataset in your python code as follows:

from datasets import load_dataset

dataset = load_dataset("intfloat/personalized_passkey_retrieval")

The data in this repo is generated by the script generate_passkey_data.py. You can also tweak the script to generate your own data.

Citation Information

If you use this dataset in your research, please cite this paper:

@inproceedings{Wang2023ImprovingTE,
  title={Improving Text Embeddings with Large Language Models},
  author={Liang Wang and Nan Yang and Xiaolong Huang and Linjun Yang and Rangan Majumder and Furu Wei},
  year={2023},
}