Benchmark Radar Dataset

Overview

Benchmark Radar is a living registry, search engine, and discovery pipeline for AI evaluation benchmarks. This dataset mirrors the full-corpus findings of the Benchmark Radar technical report ("Benchmark Radar: Daily Discovery and Full-Corpus Search Across the AI Evaluation Landscape", arXiv:2609.11115).

As described in the paper's Two Input Paths framework, Benchmark Radar combines two complementary systems:

1. Catalog & Scores: A curated, normalized cross-registry archive of 3,228 benchmark suites and 13,084 reported model score observations across LLM Stats, OpenCompass Hub, Artificial Analysis, and premier model technical reports.
2. Radar Discoveries: A 24/7 automated intelligence pipeline tracking emerging papers, code repositories, datasets, and community attention across 37+ sources, capturing 19,002 research artifacts and 33,707 daily observations.

---

Dataset Structure & Usage

The dataset is partitioned into 4 configs (subsets), easily loaded via Hugging Face datasets:

from datasets import load_dataset

# 1. Load benchmark catalog (default)
catalog = load_dataset("ktwu01/benchmark-radar", "catalog")

# 2. Load model evaluation scores
scores = load_dataset("ktwu01/benchmark-radar", "scores")

# 3. Load emerging radar artifacts (papers, repositories, datasets)
artifacts = load_dataset("ktwu01/benchmark-radar", "radar_artifacts")

# 4. Load daily discovery observations (attention, downloads, events)
observations = load_dataset("ktwu01/benchmark-radar", "radar_observations")

1. catalog (Default Config)

Contains 3,228 normalized benchmark records.

• benchmark_id: Canonical unique identifier (e.g. llm-stats:mmlu-pro, opencompass:1580)
• slug: URL-safe unique identifier
• name: Benchmark display name
• source: Source provider (llm_stats, opencompass_hub, artificial_analysis, model_reports)
• source_url: URL to original source entry
• description: Plaintext description and task summary
• categories: Assigned category tags (e.g. Reasoning, Code, Multimodal)
• languages: Languages evaluated (e.g. en, zh)
• modality: Evaluated modality (text, multimodal, code, etc.)
• publisher: Creator organization or research lab
• released_at: Official release date (YYYY-MM-DD proxy)
• openness: Data and code openness tier
• paper_url: Associated paper URL (arXiv or DOI)
• repo_url: Associated code repository URL (GitHub)
• dataset_url: Direct dataset download or Hub URL
• document_count: Number of verified source documents citing this benchmark
• model_count: Number of distinct models evaluated
• score_count: Number of numeric scores recorded
• highest_score: Maximum observed score across all evaluated models
• score_unit: Score metric / unit (e.g. %, accuracy, Elo)

2. scores

Contains 13,084 reported evaluation score observations across 870+ frontier models.

• obs_id: Unique observation identifier
• key: Benchmark key
• model_id: Source-specific stable model identifier, or null when the source does not provide one
• model_name: Model display name
• organization: Model creator/lab (e.g. DeepSeek, OpenAI, Anthropic, Google, Meta)
• value: Numeric reported score
• raw_value: Original score string from source
• value_kind: Value data type (number, percentage, etc.)
• reported_date: Model announcement or report publication date
• date_precision: Precision level of reported date
• reported_by: Source reporting modality (self_reported, third_party)
• source: Score ingestion source
• source_url: URL of source leaderboard or technical report
• document_id: Source document citation ID

3. radar_artifacts

Contains 19,002 academic artifacts surfaced by daily radar.

• id: Artifact identifier (e.g. artifact:arxiv:2203.17257)
• type: Entity type (artifact)
• label: Title or repository name
• url: Primary external URL
• categories: Extracted capability categories
• sources: Discovery sources that observed this artifact
• first_seen_at: First discovery date (YYYY-MM-DD)
• last_seen_at: Latest discovery date (YYYY-MM-DD)
• observation_count: Total appearances across snapshots
• latest_score: Attention and composite radar score
• metrics: Dictionary of observed signals (e.g. stars, citations, downloads)

4. radar_observations

Contains 33,707 discrete daily discovery events.

• id: Observation identifier
• entity_id: Referenced artifact ID
• snapshot_date: Radar snapshot date (YYYY-MM-DD)
• source: Discovery source platform
• source_id: Upstream source identifier
• url: Surfaced event URL
• discovered_at: Timestamp of discovery
• published_at: Original creation / publication timestamp
• total_score: Radar composite relevance score
• event_kind: Event category (discovered, released, updated)
• organizations: Identified affiliated organizations
• metrics: Point-in-time metrics (stars, likes, downloads)

---

Data Provenance and Principles

• Full-Corpus Coverage: All 3,228 benchmarks across all sources are preserved. Unscored benchmarks are retained with verified paper, code, and dataset links.
• Strict Evidence Citation: Every score links directly to its source document or technical report.
• Daily Automated Sync: Radar discoveries are refreshed daily at 05:00 UTC via GitHub Actions.

Licensing

The technical report and Benchmark Radar's original editorial content are available under CC BY-NC-SA 4.0. Commercial dataset packaging or product integration requires prior written permission. Third-party benchmark metadata and source material retain their original terms. Review the repository licensing notice before reuse, especially for commercial dataset packaging.

Citation

@article{benchmark_radar_2026,
  title={Benchmark Radar: Daily Discovery and Full-Corpus Search
          Across the AI Evaluation Landscape},
  author={Wu, Koutian and Contributors},
  journal={arXiv preprint arXiv:2609.11115},
  year={2026}
}