PixMo-AskModelAnything

PixMo-AskModelAnything is an instruction-tuning dataset for vision-language models. It contains human-authored question-answer pairs about diverse images with long-form answers.

PixMo-AskModelAnything is a part of the PixMo dataset collection and was used to train the Molmo family of models

Quick links:

• 📃 Paper
• 🎥 Blog with Videos

Loading

data = datasets.load_dataset("allenai/pixmo-ask-model-anything", split="train")

Data Format

Each row contains an image URL and a Q/A pair. Note the image URLs can be repeated since many images have multiple Q/A pairs.

Image Checking

Image hashes are included to support double-checking that the downloaded image matches the annotated image. It can be checked like this:

from hashlib import sha256
import requests

example = data[0]
image_bytes = requests.get(example["image_url"]).content
byte_hash = sha256(image_bytes).hexdigest()
assert byte_hash == example["image_sha256"]

License

This dataset is licensed under ODC-BY-1.0. It is intended for research and educational use in accordance with Ai2's Responsible Use Guidelines. This dataset includes data generated from Claude which are subject to Anthropic terms of service and usage policy.