SVG Benchmark Dataset

Dataset Description

This dataset contains benchmark data for SVG generation tasks.

Splits

• image2svg: Image to SVG conversion task (300 samples)
• text2svg: Text to SVG generation task (300 samples)

Features

Benchmark Construction

MMSVGBench is a purely synthetic benchmark designed to rigorously evaluate the generalization capabilities of SVG generation models. All prompts and images are generated from scratch using GPT models, ensuring the benchmark data is non-reproducible and unseen during model training.

Text-to-SVG Benchmark

The text prompts are generated using carefully designed meta-prompts with GPT, covering two complexity levels:

Icon-Level Prompts (150 samples)

You are a world-class prompt engineer and visual concept designer specializing in 
evaluating and benchmarking text-to-SVG generation models. Your goal is to produce 
diverse, creative, and challenging prompts that can test a model's SVG generation capabilities.

Task: Generate prompts for SVG generation at the icon level — each SVG should be simple 
enough to be recognizable as a single icon, but rich enough to test compositional 
reasoning, abstraction, and vector expressiveness.

Requirements:
1. Diversity of content: Cover a broad range of semantic categories: Objects, tools, 
   animals, nature, symbols, UI icons, abstract shapes, emotions, cultural motifs, etc. 
   Include both literal and conceptual ideas (e.g., "clock" vs. "time passing").

2. Diversity of style: Vary stylistic cues such as: minimalist, flat, outline, glyph, 
   doodle, isometric, retro, pixel-like, ornamental, neon, abstract, corporate, 
   hand-drawn, futuristic.

3. Controlled complexity: Maintain an SVG-appropriate complexity — no photorealistic 
   or scene-level descriptions. Each prompt should correspond to a single icon or 
   small composition of 2–4 elements.

4. Output format: Return prompts as a numbered list (150 in total). Mix the above 
   types and styles naturally.

Example Icon Prompts:

• A simple circle icon
• A magnifying glass icon
• A heart outline
• A solid star shape

Illustration-Level Prompts (150 samples)

You are a world-class prompt engineer and visual concept designer specializing in 
evaluating and benchmarking text-to-SVG generation models. Your goal is to produce 
diverse, creative, and challenging prompts that can test a model's ability to generate 
illustration-level SVGs.

Task: Generate prompts for SVG illustrations — each illustration should be more complex 
than an icon, containing 1-3 interacting elements, showing composition, perspective, 
and layered structure, while still maintaining clear readability and a coherent visual theme.

Requirements:
1. Diversity of content: Cover a broad range of semantic categories: scenes, objects, 
   animals, humans, nature, symbols, abstract concepts, emotions, cultural motifs, 
   or narrative vignettes, while keeping compositions minimal.

2. Diversity of style: Vary stylistic cues such as: minimalist illustration, flat vector, 
   outline, glyph, doodle, isometric, retro, pixel-like, ornamental, neon, abstract, 
   corporate, hand-drawn, futuristic, semi-realistic, watercolor, comic, storybook, 
   fantasy, or surreal.

3. Controlled complexity: Illustration-level complexity: 1-3 elements interacting; 
   Focus on composition, perspective, layering, and visual balance; Avoid photorealism 
   or scene-level chaos; maintain readability as a vector illustration.

4. Output format: Return prompts as a numbered list (150 in total), mixing keyword, 
   descriptive, and hybrid prompts naturally.

Example Illustration Prompts:

• A stylized illustration featuring a pie chart, line graph on a blue background
• A man is sitting on a stool using a laptop computer
• A magnifying glass and a map with a location pin symbolizing search and navigation
• A compact makeup palette with nine different shades of eyeshadow
• A minimalist illustration of an open locket with a portrait inside

Image-to-SVG Benchmark

The image-to-SVG benchmark follows a similar two-stage generation pipeline:

1. Prompt Generation: Using the same GPT-based meta-prompting strategy as text-to-SVG to generate diverse image descriptions
2. Image Generation: Using GPT-4o to generate corresponding images from the prompts

This ensures the reference images are:

• Synthetically generated and unique
• Free from copyright concerns
• Diverse in style and content
• Suitable for vector representation

Why Purely Synthetic?

The synthetic nature of MMSVGBench provides several key advantages:

Quantitative Evaluations

Quantitative results between OmniSVG and current state-of-the-art text-to-SVG and image-to-SVG baseline methods. Bold numbers and underlined numbers represent the best and second best performance respectively. Our OmniSVG model demonstrates superior performance compared to SOTA SVG generation baselines.

MMSVGBench(Icon)

Text-to-SVG

Image-to-SVG

MMSVGBench(Illustration)

Text-to-SVG

Image-to-SVG

Usage

from datasets import load_dataset

# Load the dataset
dataset = load_dataset("OmniSVG/MMSVGBench")

# Access image2svg split
image2svg_data = dataset["image2svg"]

# Access text2svg split
text2svg_data = dataset["text2svg"]

# Example: iterate over image2svg data
for sample in image2svg_data:
    print(f"ID: {sample['id']}")
    print(f"Type: {sample['type']}")
    print(f"URL: {sample['url']}")
    # sample['image'] is a PIL Image