Dataset: GPT-5 Kaggle Agent Traces (Gome)
1
5 commits
1 linked in READMEs
updated Mar 13, 2026
This folder contains the raw parallel-trace execution logs from the Gome (GPT-5, 12 h, 1*V100) experiments reported in:
Reasoning as Gradient: Scaling MLE Agents Beyond Tree Search [Paper]
The three files here correspond to three of those traces running across 40 Kaggle competitions. Each trace records the full hypothesis β code β execution β feedback loop.
Note: These are raw per-trace logs and do not include the final multi-seed selection step (Β§3.6 of the paper), where top-k candidates are re-run with multiple seeds before submission. The leaderboard scores reported in the paper reflect that additional step.
Each file shares the same nested structure:
{
"<competition-name>": {
"scenario": { ... }, β competition description & environment
"loop_0": { ... }, β first evolution loop
"loop_1": { ... }, β second evolution loop
...
},
...
}
scenario β Competition descriptionMetadata describing the competition environment seen by the agent before it starts.
| Field | Type | Description |
|---|---|---|
background | str | Structured competition description fed to the agent (task type, data description, metric) |
submission_specifications | str | Required output format for submission |
evaluation | str | Detailed metric definition |
metric_name | str | Primary evaluation metric name (e.g., "Multi-class Logarithmic Loss") |
metric_direction | bool | false = lower is better, true = higher is better |
raw_description | str | Original Kaggle competition page (Markdown); not used when use_raw_description=false |
runtime_environment | str | Python version, GPU info, CUDA version of the execution sandbox |
rendered | str | Pre-formatted prompt combining background + specs + evaluation + environment |
rewrite_scen | str | One-sentence competition summary used internally |
loop_N β One evolution iterationEach loop corresponds to one hypothesis β code β execution β feedback cycle. Fields marked core are present in every loop; others are present in ~82β93% of loops (absent when a loop timed out or was rejected before execution).
| Field | Type | Present | Description |
|---|---|---|---|
hypotheses_rewrite | dict | core | 2β3 candidate hypotheses proposed in this loop, each with component tag (FeatureEng / Model / Workflow), rationale, and multi-dimensional scores (alignment, impact, novelty, feasibility, risk/reward) |
final_hypothesis | dict | core | The hypothesis selected for implementation; includes component, problem description, and problem label (SCENARIO_PROBLEM / EXECUTION_ERROR / etc.) |
task | dict | core | Detailed step-by-step implementation specification generated from the final hypothesis; includes description and package_info |
base_code | str | core | Starting code inherited from the previous best iteration (empty string on loop_0) |
sota_hypothesis | dict | ~90β93% | Best-performing hypothesis seen so far up to this loop |
code | str | ~82β90% | Full Python main.py generated and executed by the agent |
running_time | float | ~82β90% | Execution wall time in seconds |
valid_score | dict | ~82β90% | Validation score(s) reported by the code: {metric_name: {model_name: score}} |
test_report | dict | ~82β90% | Kaggle leaderboard result: public score, medal thresholds, above/below-median flag |
feedback | dict | ~82β90% | Evaluator's structured feedback: decision (accept/reject), reason, observations, hypothesis_evaluation, new_hypothesis, code_change_summary |
5 commits
Dataset: GPT-5 Kaggle Agent Traces (Gome)
1
5 commits
1 linked in READMEs
updated Mar 13, 2026
This folder contains the raw parallel-trace execution logs from the Gome (GPT-5, 12 h, 1*V100) experiments reported in:
Reasoning as Gradient: Scaling MLE Agents Beyond Tree Search [Paper]
The three files here correspond to three of those traces running across 40 Kaggle competitions. Each trace records the full hypothesis β code β execution β feedback loop.
Note: These are raw per-trace logs and do not include the final multi-seed selection step (Β§3.6 of the paper), where top-k candidates are re-run with multiple seeds before submission. The leaderboard scores reported in the paper reflect that additional step.
Each file shares the same nested structure:
{
"<competition-name>": {
"scenario": { ... }, β competition description & environment
"loop_0": { ... }, β first evolution loop
"loop_1": { ... }, β second evolution loop
...
},
...
}
scenario β Competition descriptionMetadata describing the competition environment seen by the agent before it starts.
| Field | Type | Description |
|---|---|---|
background | str | Structured competition description fed to the agent (task type, data description, metric) |
submission_specifications | str | Required output format for submission |
evaluation | str | Detailed metric definition |
metric_name | str | Primary evaluation metric name (e.g., "Multi-class Logarithmic Loss") |
metric_direction | bool | false = lower is better, true = higher is better |
raw_description | str | Original Kaggle competition page (Markdown); not used when use_raw_description=false |
runtime_environment | str | Python version, GPU info, CUDA version of the execution sandbox |
rendered | str | Pre-formatted prompt combining background + specs + evaluation + environment |
rewrite_scen | str | One-sentence competition summary used internally |
loop_N β One evolution iterationEach loop corresponds to one hypothesis β code β execution β feedback cycle. Fields marked core are present in every loop; others are present in ~82β93% of loops (absent when a loop timed out or was rejected before execution).
| Field | Type | Present | Description |
|---|---|---|---|
hypotheses_rewrite | dict | core | 2β3 candidate hypotheses proposed in this loop, each with component tag (FeatureEng / Model / Workflow), rationale, and multi-dimensional scores (alignment, impact, novelty, feasibility, risk/reward) |
final_hypothesis | dict | core | The hypothesis selected for implementation; includes component, problem description, and problem label (SCENARIO_PROBLEM / EXECUTION_ERROR / etc.) |
task | dict | core | Detailed step-by-step implementation specification generated from the final hypothesis; includes description and package_info |
base_code | str | core | Starting code inherited from the previous best iteration (empty string on loop_0) |
sota_hypothesis | dict | ~90β93% | Best-performing hypothesis seen so far up to this loop |
code | str | ~82β90% | Full Python main.py generated and executed by the agent |
running_time | float | ~82β90% | Execution wall time in seconds |
valid_score | dict | ~82β90% | Validation score(s) reported by the code: {metric_name: {model_name: score}} |
test_report | dict | ~82β90% | Kaggle leaderboard result: public score, medal thresholds, above/below-median flag |
feedback | dict | ~82β90% | Evaluator's structured feedback: decision (accept/reject), reason, observations, hypothesis_evaluation, new_hypothesis, code_change_summary |
5 commits