Causal Plan

Causal Plan is a unified multimodal dataset release for training and evaluating causal reasoning over visually grounded plans. The repository is organized as one entry point with three clearly separated resources:

Causal_Plan/
  CausalPlan-1M-QA/
  CausalPlan-1M-FourStage-Metadata/
  Causal-Plan-Bench/
  DATASET_MANIFEST.json
  verify_alignment.py
  README.md

The QA examples, item-level four-stage metadata, and benchmark package are stored in the same repository so that every QA row can be resolved to the exact anonymous item and multimodal evidence it uses.

Resource Summary

Expected release counts:

Croissant Metadata

The release includes a root Croissant file for the unified collection plus one sub-resource Croissant file for each major resource:

The files include Croissant core metadata and minimal Responsible AI fields. Validation details are stored in validation/CROISSANT_VALIDATION_RECORD.md, with a machine-readable summary in validation/croissant_validation_summary.json.

For a fast review workflow, see REVIEWER_QUICK_CHECK.md.

Storage Notes

The full repository is large because CausalPlan-1M-FourStage-Metadata/item_shards/ preserves the full item-level multimodal packages. The four-stage item shards dominate storage, at roughly 5.3 TiB. The QA files and benchmark package are much smaller by comparison.

For most inspection and training workflows, do not blindly download every LFS object. Download the QA files and metadata first, then fetch only the item shards needed for the examples you use.

Installation

python -m pip install -U huggingface_hub pandas pyarrow
git lfs install

pandas and pyarrow are only required for parquet-based alignment checks. Basic layout verification works with the Python standard library.

Recommended Download Patterns

Metadata-only inspection

This downloads the root documentation and lightweight metadata indexes, without the large item tar shards.

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="anonymous-causal-plan/Causal_Plan",
    repo_type="dataset",
    local_dir="Causal_Plan",
    allow_patterns=[
        "README.md",
        "DATASET_MANIFEST.json",
        "verify_alignment.py",
        "CausalPlan-1M-QA/README.md",
        "CausalPlan-1M-QA/metadata/**",
        "CausalPlan-1M-FourStage-Metadata/README.md",
        "CausalPlan-1M-FourStage-Metadata/metadata/**",
        "Causal-Plan-Bench/README.md",
        "Causal-Plan-Bench/benchmark_data/README.md",
        "Causal-Plan-Bench/evaluation/**",
    ],
)

QA-only training data

This downloads the 1M QA rows and QA metadata. It does not download the four-stage item tar shards.

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="anonymous-causal-plan/Causal_Plan",
    repo_type="dataset",
    local_dir="Causal_Plan",
    allow_patterns=[
        "README.md",
        "DATASET_MANIFEST.json",
        "verify_alignment.py",
        "CausalPlan-1M-QA/**",
        "CausalPlan-1M-FourStage-Metadata/metadata/**",
    ],
)

Git clone without auto-downloading all LFS objects

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/datasets/anonymous-causal-plan/Causal_Plan
cd Causal_Plan

# Pull only the SFT QA files.
git lfs pull --include "CausalPlan-1M-QA/**"

# Pull only the benchmark package.
git lfs pull --include "Causal-Plan-Bench/**"

# Pull one specific item shard on demand.
git lfs pull --include "CausalPlan-1M-FourStage-Metadata/item_shards/item_shard_blob_p04_000027.tar"

QA File Format

The SFT files are JSONL:

CausalPlan-1M-QA/
  SFT_stage_one/<Task_XX_Name>/data.jsonl
  SFT_stage_two/<Task_XX_Name>/data.jsonl

Each row contains:

The canonical join fields are meta.item_key and meta.media_keys. Redundant repository-id hint fields from earlier internal packaging are not part of the released QA metadata.

Resolving QA Rows to Multimodal Evidence

Use meta.item_key to find the item shard, then use meta.media_keys as tar member paths inside that shard.

QA row
  -> meta.item_key
  -> CausalPlan-1M-FourStage-Metadata/metadata/item_to_shard_manifest.jsonl:item_key
  -> shard_path + member_prefix
  -> CausalPlan-1M-FourStage-Metadata/<shard_path>
  -> tar members listed by meta.media_keys

Minimal example:

import json
import tarfile
from huggingface_hub import hf_hub_download

repo_id = "anonymous-causal-plan/Causal_Plan"

qa_rel = "CausalPlan-1M-QA/SFT_stage_two/Task_08_Goal_Recognition/data.jsonl"
qa_path = hf_hub_download(repo_id=repo_id, repo_type="dataset", filename=qa_rel)
with open(qa_path, "r", encoding="utf-8") as f:
    qa = json.loads(next(f))

lookup_rel = "CausalPlan-1M-FourStage-Metadata/metadata/item_to_shard_manifest.jsonl"
lookup_path = hf_hub_download(repo_id=repo_id, repo_type="dataset", filename=lookup_rel)
item_to_shard = {}
with open(lookup_path, "r", encoding="utf-8") as f:
    for line in f:
        row = json.loads(line)
        item_to_shard[row["item_key"]] = row

item = item_to_shard[qa["meta"]["item_key"]]
shard_rel = "CausalPlan-1M-FourStage-Metadata/" + item["shard_path"]
shard_path = hf_hub_download(repo_id=repo_id, repo_type="dataset", filename=shard_rel)

with tarfile.open(shard_path) as tar:
    for media_key in qa["meta"]["media_keys"]:
        assert media_key.startswith(item["member_prefix"])
        member = tar.getmember(media_key)
        print(member.name, member.size)

Four-Stage Item Metadata

Each item tar member is stored under an anonymous item key:

SFT_stage_one/items/item_xxx/
SFT_stage_two/items/item_xxx/

An item package may include:

source_media/
stage1/
stage2/
stage3/
stage4/
stage_final/
support_media/
manifest/item_files.jsonl
manifest/item_summary.json
manifest/stage_layout.json

Use these metadata files to locate and audit item packages:

CausalPlan-1M-FourStage-Metadata/metadata/item_manifest.jsonl
CausalPlan-1M-FourStage-Metadata/metadata/item_manifest.parquet
CausalPlan-1M-FourStage-Metadata/metadata/item_to_shard_manifest.jsonl
CausalPlan-1M-FourStage-Metadata/metadata/item_to_shard_manifest.parquet

Benchmark Usage

The benchmark package is self-contained:

Causal-Plan-Bench/
  benchmark_data/
    mcq/
    qa/
    multimodal_data/
  evaluation/

Install benchmark dependencies and run validation:

cd Causal-Plan-Bench
python -m pip install -r requirements.txt
cd evaluation
bash validate_benchmark_prompts_and_data.sh

Run the packaged benchmark driver:

cd Causal-Plan-Bench/evaluation
bash run_full_benchmark_evaluation.sh

Model-backed open-QA judging requires the API credentials described in Causal-Plan-Bench/evaluation/README.md.

Verification

Basic release-layout verification:

python verify_alignment.py --root .

Full parquet alignment verification after downloading the required LFS metadata objects:

git lfs pull --include "CausalPlan-1M-QA/metadata/*.parquet,CausalPlan-1M-FourStage-Metadata/metadata/*.parquet"
python verify_alignment.py --root . --parquet

The verifier checks:

• expected top-level folders and metadata files;
• 1,000,000 QA rows;
• 22,201 item ids;
• 1,035,236 QA-to-media edges;
• zero missing QA media edges in the public alignment summary;
• QA item ids resolvable through the item manifests.

Anonymization and Release Scope

The public release uses anonymous item keys and package-relative media paths. Local source paths, user names, generation timestamps, raw generation prompts, raw model responses, account metadata, and runtime caches are excluded from the four-stage item packages.

The root repository is the canonical public entry point:

anonymous-causal-plan/Causal_Plan