Dataset Card for PersonalLLM

The PersonalLLM dataset is a collection of prompts, responses, and rewards designed for personalized language model methodology development and evaluation. This dataset is presented in the paper PersonalLLM: Tailoring LLMs to Individual Preferences.

Dataset Details

Dataset Description

• Curated by: Andrew Siah*, Tom Zollo*, Naimeng Ye, Ang Li, Namkoong Hongseok
• Funded by: Digital Future Initiative at Columbia Business School
• Language(s) (NLP): English
• License: CC BY 4.0 License

Dataset Sources

• Repository: https://github.com/namkoong-lab/PersonalLLM

Uses

The PersonalLLM dataset is a comprehensive collection of prompts, responses, and associated metadata designed for developing and evaluating personalized language models. Key features include:

1. Structure:

   • Prompts from diverse sources (RewardBench, Anthropic HH-RLHF, HelpSteer)
   • 8 responses per prompt from various models,
     • cohere/command-r-plus
     • openai/gpt-4-turbo
     • openai/gpt-4o
     • anthropic/claude-3-opus
     • anthropic/claude-3-sonnet
     • meta-llama/llama-3-70b-instruct
     • google/gemini-pro-1.5
     • mistralai/mixtral-8x22b-instruct
   • 10 reward models rewarding all prompts and responses
     • hendrydong/Mistral-RM-for-RAFT-GSHF-v0
     • OpenAssistant/oasst-rm-2-pythia-6.9b-epoch-1
     • OpenAssistant/oasst-rm-2.1-pythia-1.4b-epoch-2.5
     • OpenAssistant/reward-model-deberta-v3-large-v2
     • PKU-Alignment/beaver-7b-v1.0-cost
     • Ray2333/reward-model-Mistral-7B-instruct-Unified-Feedback
     • sfairXC/FsfairX-LLaMA3-RM-v0.1
     • weqweasdas/RM-Gemma-2B
     • weqweasdas/RM-Gemma-7B
     • weqweasdas/RM-Mistral-7B

2. Size and Split:

   • Training set: 9,402 entries
   • Test set: 1,000 entries

3. Content:

   • Prompts: Covering a range of topics and intents
   • Responses: Generated by 8 different models
   • Metadata: Includes model names, subset information, and unique IDs

4. Data Quality:

   • Filtered for uniqueness (no duplicate prompts)
   • Length-restricted (prompts > 2400 characters removed)
   • Keeping the first 9,402 rows for training, and 1,000 rows for testing.

5. Use Cases:

   • Developing personalized language models
   • Comparing performance across different personalization algorithms, e.g. Recommendation Systems, Retrieval Methods.

6. Ethical Considerations:

   • Designed with potential for bias analysis
   • Allows for studying model alignment with personalized human preferences

7. Extensibility:

   • Structure allows for easy addition of new prompts and responses.

Citation

BibTeX:

@article{2024personalllm,
  title={PersonalLLM: Tailoring LLMs to Individual Preferences},
  author={Andrew Siah and Thomas P. Zollo and Naimeng Ye and Ang Li and Hongseok Namkoong},
  journal={Columbia University},
  address={New York, NY 10027},
  year={2024},
  note={\texttt{\{andrew.siah, tpz2105, ny2336, al4263, hn2369\}@columbia.edu}}
}

APA:

Siah, A., Zollo, T. P., Ye, N., Li, A., & Namkoong, H. (2024). PersonalLLM: Tailoring LLMs to individual preferences.

Contact

Andrew at andrew.siah@columbia.edu