This dataset contains the public model-by-benchmark score matrix used by BenchPress. The release includes the lossless audited JSON, benchmark cost evidence, flat model and benchmark metadata, one row per observed score, and the paper-canonical dense subset used in the BenchPress experiments.
The source repository is
microsoft/benchpress.
data/llm_benchmark_data.json is the authoritative rich score-matrix artifact.
It preserves nested alternative candidates, audit provenance, source URLs, and
normalized benchmark cost fields that cannot be represented losslessly in CSV.
data/benchmark_cost_evidence.json is the authoritative raw public
cost-evidence artifact. metadata.json records the SHA-256 digest and byte size
of both canonical JSON files. The CSV and Parquet files are deterministic flat
exports from the score-matrix JSON.
| File | Contents |
|---|---|
data/llm_benchmark_data.json | Lossless audited score matrix: models, benchmarks, scores, candidates, and audit provenance. |
data/benchmark_cost_evidence.json | Raw public token, dollar, and run-budget evidence used by benchmark cost metadata. |
data/scores_all.csv / .parquet | Flat numeric score rows in the audit pool. |
data/scores_paper.csv / .parquet | Long-form rows for the paper-canonical matrix. |
data/models.csv / .parquet | Model metadata and canonical evaluation settings. |
data/benchmarks.csv / .parquet | Benchmark metadata and canonical benchmark settings. |
data/score_matrix_paper_wide.csv | Wide model x benchmark matrix for the paper-canonical subset. |
data/README.md, data/SCHEMA.md | Dataset conventions and the canonical JSON schema. |
data/LICENSE-CDLA-2.0.md | Dataset license text. |
metadata.json | Export counts, matrix construction metadata, file inventory, and canonical JSON hashes. |
from datasets import load_dataset
scores = load_dataset("microsoft/benchpress-score-matrix", "scores_paper")["train"].to_pandas()
models = load_dataset("microsoft/benchpress-score-matrix", "models")["train"].to_pandas()
benchmarks = load_dataset("microsoft/benchpress-score-matrix", "benchmarks")["train"].to_pandas()
For the lossless audit artifact:
import json
from urllib.request import urlopen
url = (
"https://huggingface.co/datasets/microsoft/benchpress-score-matrix/"
"resolve/main/data/llm_benchmark_data.json"
)
with urlopen(url) as response:
matrix = json.load(response)
The flat score tables include:
model_id, benchmark_id, scorereference_url, source_type, audit_status, matches_canonicalreported_setting_json, notesThe lossless JSON additionally preserves candidates, audit-rule identifiers,
audit notes, timestamps, and benchmark-level cost evidence.
models and benchmarks include an in_paper_matrix flag that identifies
rows retained by the paper-canonical threshold filter.
The paper-canonical matrix applies the BenchPress construction pipeline: audit-status filtering, canonical representative selection, and the iterative threshold filter. Current export counts:
The dataset files are released under CDLA-Permissive-2.0. The source repository's MIT license applies to code and documentation, not to this dataset license grant.
Scores come from heterogeneous public sources: model cards, official blogs, technical reports, benchmark leaderboards, and third-party aggregators. Each score retains source and audit metadata so downstream users can choose their own filtering policy.
This dataset contains the public model-by-benchmark score matrix used by BenchPress. The release includes the lossless audited JSON, benchmark cost evidence, flat model and benchmark metadata, one row per observed score, and the paper-canonical dense subset used in the BenchPress experiments.
The source repository is
microsoft/benchpress.
data/llm_benchmark_data.json is the authoritative rich score-matrix artifact.
It preserves nested alternative candidates, audit provenance, source URLs, and
normalized benchmark cost fields that cannot be represented losslessly in CSV.
data/benchmark_cost_evidence.json is the authoritative raw public
cost-evidence artifact. metadata.json records the SHA-256 digest and byte size
of both canonical JSON files. The CSV and Parquet files are deterministic flat
exports from the score-matrix JSON.
| File | Contents |
|---|---|
data/llm_benchmark_data.json | Lossless audited score matrix: models, benchmarks, scores, candidates, and audit provenance. |
data/benchmark_cost_evidence.json | Raw public token, dollar, and run-budget evidence used by benchmark cost metadata. |
data/scores_all.csv / .parquet | Flat numeric score rows in the audit pool. |
data/scores_paper.csv / .parquet | Long-form rows for the paper-canonical matrix. |
data/models.csv / .parquet | Model metadata and canonical evaluation settings. |
data/benchmarks.csv / .parquet | Benchmark metadata and canonical benchmark settings. |
data/score_matrix_paper_wide.csv | Wide model x benchmark matrix for the paper-canonical subset. |
data/README.md, data/SCHEMA.md | Dataset conventions and the canonical JSON schema. |
data/LICENSE-CDLA-2.0.md | Dataset license text. |
metadata.json | Export counts, matrix construction metadata, file inventory, and canonical JSON hashes. |
from datasets import load_dataset
scores = load_dataset("microsoft/benchpress-score-matrix", "scores_paper")["train"].to_pandas()
models = load_dataset("microsoft/benchpress-score-matrix", "models")["train"].to_pandas()
benchmarks = load_dataset("microsoft/benchpress-score-matrix", "benchmarks")["train"].to_pandas()
For the lossless audit artifact:
import json
from urllib.request import urlopen
url = (
"https://huggingface.co/datasets/microsoft/benchpress-score-matrix/"
"resolve/main/data/llm_benchmark_data.json"
)
with urlopen(url) as response:
matrix = json.load(response)
The flat score tables include:
model_id, benchmark_id, scorereference_url, source_type, audit_status, matches_canonicalreported_setting_json, notesThe lossless JSON additionally preserves candidates, audit-rule identifiers,
audit notes, timestamps, and benchmark-level cost evidence.
models and benchmarks include an in_paper_matrix flag that identifies
rows retained by the paper-canonical threshold filter.
The paper-canonical matrix applies the BenchPress construction pipeline: audit-status filtering, canonical representative selection, and the iterative threshold filter. Current export counts:
The dataset files are released under CDLA-Permissive-2.0. The source repository's MIT license applies to code and documentation, not to this dataset license grant.
Scores come from heterogeneous public sources: model cards, official blogs, technical reports, benchmark leaderboards, and third-party aggregators. Each score retains source and audit metadata so downstream users can choose their own filtering policy.