HeterQA

HeterQA is an 857-question benchmark for record-level retrieval over multiple heterogeneous sources. In this Yelp instantiation, target records are businesses and queries combine record fields with review text, photos, spatial constraints, and graph-derived evidence.

This hosted release contains HeterQA annotations and benchmark metadata only. It does not redistribute original Yelp reviews, photos, full business records, or other Yelp source content. Users who need to reconstruct full retrieval indexes must obtain the Yelp Open Dataset separately and comply with its terms.

Files

• data/queries.jsonl: released questions, subset labels, and optional structured geographic reference locations.
• data/answers.jsonl: certified answer sets using Yelp business_id. Query IDs are stable natural identifiers from 1 to 857; internal generated IDs, timestamps, and run suffixes are not released.
• data/qrels/test.tsv: BEIR-style relevance judgments.
• data/evidence.jsonl: structured answer-level evidence summaries. Review text evidence uses Yelp review_id locators when recoverable, not raw review content.
• data/source_manifest.json: provenance and release scope.
• metadata/croissant.json: Croissant metadata.
• schemas/*.schema.json: JSON Schemas for the JSONL files.
• METHOD.md: extraction, evidence, and redaction protocol.

Query Location Anchors

Coordinate-anchored queries include an optional reference_location object:

{
  "type": "user",
  "latitude": 39.876110,
  "longitude": -75.111982,
  "name": null
}

The type field is either user or poi. A user reference has a null name; a poi reference stores the landmark name in name. The query text refers to the corresponding user position or named landmark without embedding the numeric coordinate pair.

The field is present in 352 questions: 208 user references and 144 point-of- interest references. The other 505 questions do not contain reference_location.

Distances, directions, proximity operators, categories, attributes, Boolean structure, and certified answer sets are unchanged from version 1.0.0.

Relationship to Yelp

HeterQA is an annotation layer over the Yelp Open Dataset. This repository keeps Yelp business_id values as answer identifiers so users can reconstruct the retrieval corpus after obtaining the Yelp source data separately.

To use HeterQA with Yelp records:

1. Download the Yelp Open Dataset from Yelp and follow Yelp's terms of use.
2. Load yelp_academic_dataset_business.json and join by business_id.
3. Load yelp_academic_dataset_review.json if your experiment needs review text. Review-text evidence locators join through Yelp review_id and business_id.
4. Load photos.json and photo files named by photo_id if your experiment needs image evidence.

This repository does not include Yelp reviews, photos, full business profiles, addresses, user data, or local source paths.

Intended Use

The dataset is intended for evaluating record-level retrieval systems that must return certified target records from heterogeneous evidence. It is not intended for production recommendation, business-quality claims, user profiling, or redistribution of Yelp source content.

License Scope

CC-BY-4.0 applies only to HeterQA annotations and metadata created by the authors. It does not apply to the original Yelp Open Dataset or any Yelp source content.