Historical snapshots of the Arena leaderboard.
from datasets import load_dataset
# Load all historical text style control data
ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="full")
# Load the current text style control leaderboard
ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="latest")
# Filter to overall category
ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="latest",
filters=[("category", "==", "overall")]
)
# Track a specific model's rating over time
ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="full",
filters=[("category", "==", "overall"), ("model_name", "==", "gpt-4o-2024-05-13")],
columns=["model_name", "rating", "rank", "leaderboard_publish_date"]
)
# Load raw (non-style-controlled) text ratings
ds = load_dataset("lmarena-ai/leaderboard-dataset", "text", split="full")
# Load the current Agent Arena leaderboard
ds = load_dataset("lmarena-ai/leaderboard-dataset", "agent", split="latest")
Each arena is a separate subset, arenas with style control have an additional subset with a _style_control suffix .
text, text_style_controlvision, vision_style_controlsearch, search_style_controldocument, document_style_controlwebdev (Code Arena)text_to_imageimage_edittext_to_videoimage_to_videovideo_editagent (Agent Arena aggregate), plus per-signal subsets: agent_bash_recovery_steps, agent_praise_complaint, agent_steerability, agent_task_outcome_explicit, agent_tool_hallucinationfull: All historically published leaderboardslatest: Only the most recently published leaderboardsFor a complete record of all leaderboard methodology changes, see the Leaderboard Changelog.
| Column | Type | Description |
|---|---|---|
| model_name | string | Model identifier |
| organization | string | Model creator/organization |
| license | string | Model license |
| rating | float | Arena Score |
| rating_lower | float | Lower confidence bound |
| rating_upper | float | Upper confidence bound |
| variance | float | Rating variance |
| vote_count | int | Number of battles for this model |
| rank | int | Rank within this leaderboard |
| category | string | Leaderboard category (e.g., overall, coding, math) |
| leaderboard_publish_date | string | Date that this score was published (YYYY-MM-DD) |
Agent Arena subsets use IPS scores instead of Bradley-Terry:
| Column | Type | Description |
|---|---|---|
| model_name | string | Model identifier |
| organization | string | Model creator/organization |
| license | string | Model license |
| score | float | IPS score (τ̂) |
| score_ci_lower | float | Lower 95% confidence bound |
| score_ci_upper | float | Upper 95% confidence bound |
| observation_count | int | Signal observations for this model |
| session_count | int | Distinct sessions (aggregate subset only) |
| rank | int | Rank within this leaderboard |
| category | string | Leaderboard category |
| leaderboard_publish_date | string | Date that this score was published (YYYY-MM-DD) |
500 commits
Historical snapshots of the Arena leaderboard.
from datasets import load_dataset
# Load all historical text style control data
ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="full")
# Load the current text style control leaderboard
ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="latest")
# Filter to overall category
ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="latest",
filters=[("category", "==", "overall")]
)
# Track a specific model's rating over time
ds = load_dataset("lmarena-ai/leaderboard-dataset", "text_style_control", split="full",
filters=[("category", "==", "overall"), ("model_name", "==", "gpt-4o-2024-05-13")],
columns=["model_name", "rating", "rank", "leaderboard_publish_date"]
)
# Load raw (non-style-controlled) text ratings
ds = load_dataset("lmarena-ai/leaderboard-dataset", "text", split="full")
# Load the current Agent Arena leaderboard
ds = load_dataset("lmarena-ai/leaderboard-dataset", "agent", split="latest")
Each arena is a separate subset, arenas with style control have an additional subset with a _style_control suffix .
text, text_style_controlvision, vision_style_controlsearch, search_style_controldocument, document_style_controlwebdev (Code Arena)text_to_imageimage_edittext_to_videoimage_to_videovideo_editagent (Agent Arena aggregate), plus per-signal subsets: agent_bash_recovery_steps, agent_praise_complaint, agent_steerability, agent_task_outcome_explicit, agent_tool_hallucinationfull: All historically published leaderboardslatest: Only the most recently published leaderboardsFor a complete record of all leaderboard methodology changes, see the Leaderboard Changelog.
| Column | Type | Description |
|---|---|---|
| model_name | string | Model identifier |
| organization | string | Model creator/organization |
| license | string | Model license |
| rating | float | Arena Score |
| rating_lower | float | Lower confidence bound |
| rating_upper | float | Upper confidence bound |
| variance | float | Rating variance |
| vote_count | int | Number of battles for this model |
| rank | int | Rank within this leaderboard |
| category | string | Leaderboard category (e.g., overall, coding, math) |
| leaderboard_publish_date | string | Date that this score was published (YYYY-MM-DD) |
Agent Arena subsets use IPS scores instead of Bradley-Terry:
| Column | Type | Description |
|---|---|---|
| model_name | string | Model identifier |
| organization | string | Model creator/organization |
| license | string | Model license |
| score | float | IPS score (τ̂) |
| score_ci_lower | float | Lower 95% confidence bound |
| score_ci_upper | float | Upper 95% confidence bound |
| observation_count | int | Signal observations for this model |
| session_count | int | Distinct sessions (aggregate subset only) |
| rank | int | Rank within this leaderboard |
| category | string | Leaderboard category |
| leaderboard_publish_date | string | Date that this score was published (YYYY-MM-DD) |
500 commits