Agent Planning Benchmark Dataset

This repository contains the complete APB data prepared for Hugging Face. It uses the paper-aligned task names: Holistic Planning, Step-wise Planning, Tool-Extraneous, Tool-Broken, and Unsolvable.

Real-traffic subtypes such as doc, excel, normal, ppt, and train are stored in source_subset when applicable.

Contents

• data/holistic_planning.jsonl: Holistic Planning records.
• data/holistic_tool_extraneous.jsonl: Holistic Tool-Extraneous records.
• data/holistic_unsolvable.jsonl: Holistic Unsolvable records.
• data/step_wise_planning.jsonl: Step-wise Planning records.
• data/step_wise_tool_extraneous.jsonl: Step-wise Tool-Extraneous records.
• data/step_wise_tool_broken.jsonl: Step-wise Tool-Broken records.
• data/all.jsonl: all records concatenated.
• data/sources/<split>/*.jsonl: split-by-source views.
• assets_archives/Holistic_assets.tar and assets_archives/Step-wise_assets.tar: referenced files only, archived to keep the Hugging Face upload stable.
• metadata/summary.json: counts and validation summary.

Splits

Total records: 4209

Regimes

Sources

Holistic Planning

Holistic Tool-Extraneous

Holistic Unsolvable

Step-wise Planning

Step-wise Tool-Extraneous

Step-wise Tool-Broken

Prediction Horizons

Step-wise Planning

Step-wise Tool-Extraneous

Step-wise Tool-Broken

Main Schema

• id: stable public identifier.
• split: one of the six split file stems above.
• split_display_name: paper-aligned display name.
• planning_regime: Holistic or Step-wise.
• task_family: Planning, Tool-Extraneous, Tool-Broken, or Unsolvable.
• task_category: display name combining regime and task family.
• source: public source name.
• source_subset: subtype for real-traffic examples when applicable.
• query: user task.
• expected_behavior: target behavior such as generate_plan, predict_next_step, recover_from_broken_tool, or reject.
• prediction_horizon: next1, next2, or next3 for Step-wise records.
• plan and tool_chain_json: Holistic reference fields.
• trajectory_prefix_json, ground_truth_steps_json, and reference_remaining_steps_json: Step-wise reference fields.
• tools_json, base_tools_json, extraneous_tools_json: available tool definitions.
• tool_names_json, base_tool_names_json, extraneous_tool_names_json: normalized tool name lists.
• extraneous_tool_count: N for Tool-Extraneous cases.
• unsolvable_type: subtype for Unsolvable cases.
• broken_tool_name, replacement_tool_json: Tool-Broken metadata.
• files: referenced files, with paths relative to this folder.
• valid_reference: false when a required reference or referenced asset is missing.
• validation_errors: JSON string containing validation reasons.

Loading

from datasets import load_dataset

holistic = load_dataset("json", data_files="data/holistic_planning.jsonl", split="train")
step_wise = load_dataset("json", data_files="data/step_wise_planning.jsonl", split="train")
all_records = load_dataset("json", data_files="data/all.jsonl", split="train")

Restoring Assets

The files field stores paths relative to the repository root, such as assets/Holistic/... or assets/Step-wise/.... After downloading the repository, restore those paths with:

tar -xf assets_archives/Holistic_assets.tar
tar -xf assets_archives/Step-wise_assets.tar

This creates assets/Holistic/ and assets/Step-wise/ in the repository root.

Evaluation and judge code is intentionally excluded from this Hugging Face data package. Use the separate GitHub code package generated by scripts/prepare_apb_full_github_code.py.