Osprey-724K Dataset Card

Osprey-724K is an instruction dataset with mask-text pairs, containing around 724K GPT-generated multimodal dialogues to encourage MLLMs for fine-grained pixel-level image understanding. It contains object-level, part-level and additional instruction samples for robustness and flexibility.

Dataset type:

• Object-level: osprey_conversation.json, osprey_detail_description.json
• Part-level: osprey_part_level.json
• Robustness&Flexibility: osprey_lvis_positive_negative.json, osprey_short_form.json

Paper and Code

Paper: https://arxiv.org/abs/2312.10032
Code: https://github.com/CircleRadon/Osprey

License

Attribution-NonCommercial 4.0 International
It should abide by the policy of OpenAI: https://openai.com/policies/terms-of-use.

Citations

@misc{Osprey,
  title={Osprey: Pixel Understanding with Visual Instruction Tuning},
  author={Yuqian Yuan, Wentong Li, Jian Liu, Dongqi Tang, Xinjie Luo, Chi Qin, Lei Zhang and Jianke Zhu},
  year={2023},
  eprint={2312.10032},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}