BlenderBench Dataset

Dataset Description

BlenderBench is a comprehensive benchmark dataset for evaluating models on 3D scene editing tasks in Blender. The dataset challenges agents to understand visual differences between initial and target scenes, then generate appropriate Blender Python code to transform the initial scene to match the target.

Key Features

• 27 instances across 3 difficulty levels
• Multi-modal: Combines visual understanding with code generation
• Realistic 3D scenes: Using Blender Studio assets
• Progressive difficulty: From simple camera adjustments to complex scene manipulations
• Rich annotations: Each instance includes task descriptions, start/goal code, and rendered images

Task Categories

The dataset covers three main difficulty levels:

1. Level 1 (9 instances): Camera adjustments
2. Level 2 (9 instances): Multi-step editing
3. Level 3 (9 instances): Compositional editing (level 1 + level 2)

Dataset Structure

Each instance in the dataset contains:

instance/
├── blender_file.blend       # Blender scene file
├── start.py                  # Initial scene configuration code
├── goal.py                   # Target scene configuration code
├── task.txt                  # Natural language task description
└── renders/
    ├── start/
    │   └── render1.png      # Rendered image of initial scene (512x512)
    └── goal/
        └── render1.png      # Rendered image of target scene (512x512)

Data Fields

When loaded using datasets.load_dataset(), each example contains:

• instance_id (string): Unique identifier (e.g., "level1/camera1")
• level (string): Difficulty level ("level1", "level2", or "level3")
• instance_name (string): Instance name within the level
• task_description (string): Natural language description of the task
• start_code (string): Python code for the initial scene setup
• goal_code (string): Python code for the target scene configuration
• start_render (image): Rendered image of the initial scene (512x512 PNG)
• goal_render (image): Rendered image of the target scene (512x512 PNG)
• blend_file_path (string): Relative path to the .blend file
• blend_file_size_mb (float): Size of the .blend file in MB

Data Splits

The dataset provides a single training split containing all instances. You can filter by difficulty level using the dataset configurations:

• all: All instances (default)
• level1: Only level 1 instances
• level2: Only level 2 instances
• level3: Only level 3 instances

Usage

Installation

pip install datasets huggingface_hub

Loading the Dataset

from datasets import load_dataset

# Load all instances
dataset = load_dataset("DietCoke4671/BlenderBench")

# Load only level 1 instances
dataset = load_dataset("DietCoke4671/BlenderBench", "level1")

# Access an example
example = dataset["train"][0]
print(f"Task: {example['task_description']}")
print(f"Level: {example['level']}")

# Display images
example['start_render'].show()  # Initial scene
example['goal_render'].show()   # Target scene

# Access code
print(f"Start code:\n{example['start_code']}")
print(f"Goal code:\n{example['goal_code']}")

Using with Blender

To actually render and evaluate generated code, you'll need Blender installed:

import subprocess
from pathlib import Path

def render_with_blender(blend_file, code, output_dir):
    """
    Execute Blender code and render the result.

    Args:
        blend_file: Path to .blend file
        code: Python code to execute in Blender
        output_dir: Directory to save rendered output
    """
    # Save code to temporary file
    code_file = Path(output_dir) / "temp_code.py"
    with open(code_file, 'w') as f:
        f.write(code)

    # Run Blender
    cmd = [
        "blender",
        "--background",
        str(blend_file),
        "--python", str(code_file),
        "--render-output", str(output_dir / "render.png"),
        "--render-frame", "1"
    ]

    subprocess.run(cmd)

# Example usage
example = dataset["train"][0]
render_with_blender(
    blend_file=f"path/to/{example['blend_file_path']}",
    code=example['start_code'],
    output_dir="output/"
)

Example: Building an AI Agent

from datasets import load_dataset
import openai

# Load dataset
dataset = load_dataset("DietCoke4671/BlenderBench", "level1")

def solve_blender_task(example):
    """
    Use an AI agent to solve a BlenderBench task.
    """
    # Prepare prompt with task description and images
    prompt = f"""
You are an expert Blender Python programmer. Your task is to:

Task: {example['task_description']}

Given the initial scene code and rendered image, generate Blender Python code
to transform the scene to match the target image.

Initial code:
{example['start_code']}

Generate the modified code that will produce the target scene.
"""

    # Call your AI model (e.g., GPT-4 with vision)
    response = openai.ChatCompletion.create(
        model="gpt-4-vision-preview",
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": prompt},
                    {
                        "type": "image_url",
                        "image_url": {"url": encode_image(example['start_render'])}
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": encode_image(example['goal_render'])}
                    }
                ]
            }
        ]
    )

    return response.choices[0].message.content

# Solve all level 1 tasks
for example in dataset["train"]:
    solution = solve_blender_task(example)
    print(f"Solution for {example['instance_id']}:")
    print(solution)