🎬 VideoBiasEval: Uncovering How AI Video Models Learn Human Prejudice

---

🚨 Key Discovery: Alignment Makes Video AI More Biased, Not Less

"When we teach AI to follow human preferences, it learns our prejudices too."

📊 Shocking Results at a Glance:

First systematic analysis revealing how alignment tuning amplifies social biases in video generation models through the entire pipeline: reward datasets → reward models → video generation.

🔍 What We Discovered

🎭 The Bias Pipeline: From Human Preferences to AI Prejudice

1. 📚 Human Preference Datasets Are Biased

   • HPDv2: 62% of actions show male preference
   • Pick-a-Pic: 58% show female preference
   • Both datasets: 40-43% White representation (6× overrepresentation)

2. 🤖 Reward Models Amplify These Biases

   • CLIP (base): Slight female preference (-0.073)
   • HPSv2.0 (after training): Strong male preference (+0.604)
   • That's a +676.5% bias shift!

3. 🎬 Video Models Inherit and Intensify Biases

   • ModelScope → InstructVideo: +4.8% male bias
   • VideoCrafter → T2V-Turbo-V1: +7.25% male bias
   • Diversity drops by 50% after alignment

🎯 Most Bias-Sensitive Actions

Actions where gender representation changes most dramatically:

• 😴 "sleep": +18.2% male shift
• 🏃 "stretch": +15.7% male shift
• 📖 "read": +14.3% male shift
• 🍳 "cook": -12.1% female shift
• 🥖 "bake": -10.8% female shift

📁 Repository Structure

VideoBiasEval/
├── VideoGen/                        # 🎬 Video generation pipeline
│   ├── generate.py                  # Main video generation entry point
│   ├── models/                      # Model implementations (9+ models)
│   │   ├── __init__.py
│   │   ├── allegro.py               # Allegro implementation
│   │   ├── cog_videox.py            # CogVideoX implementation
│   │   ├── hunyuan_video.py         # Hunyuan Video implementation
│   │   ├── modelscope.py            # ModelScope implementation
│   │   ├── open_sora.py             # Open-Sora implementation
│   │   ├── pyramid_flow.py          # Pyramid Flow implementation
│   │   ├── t2v_turbo.py             # T2V-Turbo V1/V2 implementation
│   │   ├── vader_videocrafter.py    # VADER-VideoCrafter implementation
│   │   └── video_crafter.py         # VideoCrafter implementation
│   ├── utils/                       # Utility functions
│   │   ├── gpt4o_api.py             # GPT-4o integration for evaluation
│   │   ├── prompts_generate.py      # Event-based prompt generation
│   │   └── video_utils.py           # Video processing utilities
│   ├── config/                      # Configuration files
│   │   ├── models/                  # Model-specific configs (JSON)
│   │   ├── attribute/               # Gender & ethnicity definitions
│   │   ├── prompts/                 # Prompt templates
│   │   │   ├── gender_only_prompts_long_env.json
│   │   │   └── gender_ethn_prompts_long_env.json
│   │   ├── videos/                  # Video metadata (auto-generated by inference)
│   │   │   └── {model}/{run_id}/{setting}.json
│   │   └── openai/                  # OpenAI API configuration
│   ├── scripts/                     # Inference scripts for each model
│   │   ├── inference_pyramid_flow.sh
│   │   ├── inference_video_crafter.sh
│   │   └── ... (one per model)
│   ├── scripts_alignment/           # Alignment tuning evaluation
│   │   ├── evaluate_hpsv2.sh
│   │   └── evaluate_pick.sh
│   ├── scripts_prepare/             # Model preparation scripts
│   │   ├── prepare_pyramid_flow.py
│   │   └── ... (model downloads/setup)
│   ├── requirements/                # Dependencies per model
│   │   ├── requirements_all.txt
│   │   ├── requirements_video_crafter.txt
│   │   └── ... (model-specific)
│   ├── outputs_{model}/             # Generated videos (created by inference)
│   │   └── r{run_id}/{setting}/
│   │       └── *.mp4
│   ├── CKPT/                        # Model checkpoints directory
│   └── README.md                    # Detailed VideoGen documentation
│
├── PhotoGen/                        # 📸 Photo generation for reward model analysis
│   ├── preprocess-prompts/          # Convert video prompts to photo prompts
│   │   ├── preprocess_gender_prompts.py
│   │   ├── preprocess_ethnicity_prompts.py
│   │   └── README.md
│   ├── encode-prompts/              # Encode prompts for reward scoring
│   │   ├── extract_reward_gender_only_prompt.py
│   │   ├── extract_reward_ethn_only_prompt.py
│   │   └── README.md
│   └── generate-images/             # FLUX image generation
│       ├── generate_images.py       # FLUX image generation script
│       ├── generate.sh              # Image generation orchestrator
│       ├── process_tar.py           # Process tar archives
│       ├── reformat_labels.py       # Reformat preference labels
│       ├── upload.py                # Upload to HuggingFace
│       ├── RM-preference-labels.json # Reward model preference labels
│       └── requirements.txt         # Image generation dependencies
│
├── RewardModelAnalysis/             # 🏆 Reward model bias evaluation
│   ├── reward_one.sh                # ⭐ Reward model inference script
│   ├── src/                         # Source code for reward model evaluation
│   │   ├── driver.py                # Main driver script
│   │   ├── clip_code.py             # CLIP reward model
│   │   ├── hps_code.py              # HPSv2.0/v2.1 reward model
│   │   ├── image_reward_code.py     # ImageReward model
│   │   ├── pick_score_code.py       # PickScore model
│   │   ├── intern_video_code.py     # InternVideo model
│   │   ├── download_ckpt.py         # Download model checkpoints
│   │   ├── download_dataset.py      # Download evaluation datasets
│   │   └── utils.py                 # Utility functions
│   ├── prompts-gpu0/                # Prompt datasets for evaluation
│   ├── archive/                     # Archived experiments and notebooks
│   │   ├── hps-v2-inference/        # HPSv2 inference experiments
│   │   └── reward_inference/        # Legacy reward inference code
│   └── output/                      # Reward model evaluation results (auto-generated)
│       └── score_{model}_{setting}_{people_type}_{env_type}.json
│
├── AlignmentTuning/                 # 🔧 Alignment tuning training code
│   └── t2v-turbo/                   # T2V-Turbo alignment tuning implementation
│       ├── README.md                # T2V-Turbo training documentation
│       ├── train_t2v_turbo_v1.sh    # Training script for T2V-Turbo-v1
│       ├── train_t2v_turbo_v2.sh    # Training script for T2V-Turbo-v2
│       ├── app.py                   # Inference demo application
│       ├── app_ms.py                # ModelScope inference demo
│       ├── predict.py               # Prediction script
│       ├── inverse_ddim.py          # Inverse DDIM implementation
│       ├── motion_prior_sample.py   # Motion prior sampling
│       ├── cog.yaml                 # Conda environment configuration
│       ├── configs/                 # Model and training configurations
│       ├── reward_fn/               # Reward function implementations
│       ├── pipeline/                # Training pipeline
│       ├── lvdm/                    # Latent video diffusion model
│       ├── model_scope/             # ModelScope integration
│       ├── intern_vid2/             # InternVideo2 integration
│       ├── ode_solver/              # ODE solver utilities
│       ├── preprocess_scripts/      # Data preprocessing scripts
│       └── bpe_simple_vocab_16e6.txt.gz # BPE vocabulary for CLIP
│
├── VideoEval/                       # 🔍 Bias evaluation pipeline
│   ├── run_evaluation.sh            # ⭐ Master automation script
│   ├── extract_frames.sh            # Frame extraction orchestrator
│   ├── extract_frames.py            # Frame extraction implementation
│   ├── inference_vllm.sh            # VLM inference orchestrator
│   ├── inference_vllm.py            # VLM inference implementation
│   ├── metrics.ipynb                # 📊 Bias metrics analysis (TAS, PBS_G, RDS_e, SDI)
│   ├── config/                      # Evaluation configurations
│   │   ├── attribute/               # Attribute definitions
│   │   └── prompts/                 # Evaluation prompts
│   │       ├── gender_only_prompts_long_env.json
│   │       └── gender_ethn_prompts_long_env.json
│   ├── outputs_{model}/             # Evaluation outputs (per model)
│   │   └── r{run_id}/
│   │       ├── {setting}_frames/    # Extracted frames
│   │       │   └── {video_name}/
│   │       │       └── frame_*.jpg
│   │       ├── {setting}_frames.json # Frame metadata
│   │       ├── qwen2_vl/            # VLM results (Qwen2-VL)
│   │       │   ├── Frames/{setting}/
│   │       │   │   └── qwen2_vl_results.json
│   │       │   └── Log/
│   │       │       └── {setting}_results_log.txt
│   │       ├── qwen2_5_vl/          # VLM results (Qwen2.5-VL)
│   │       └── internvl_chat/       # VLM results (InternVL2.5)
│   └── README.md                    # Detailed VideoEval documentation
│
├── Figures/                         # 📈 Figure generation and analysis
│   ├── FinalFigures/                # Publication-ready figures
│   │   ├── Main/                    # Main paper figures
│   │   ├── AlignmentTuning/         # Alignment tuning results
│   │   ├── AlignmentTuning_ethnicity/
│   │   └── main_ethnicity/
│   ├── MainFigureCSV/               # Video generation model metrics (CSV)
│   ├── RewardModelCSV/              # Reward model evaluation data (CSV)
│   ├── AlignmentTuningCSV/          # Alignment tuning metrics (CSV)
│   ├── MetricRegression/            # Statistical analysis and regression
│   │   ├── regression.ipynb         # Main regression analysis
│   │   ├── gender/                  # Gender bias regression
│   │   │   └── regression.ipynb
│   │   ├── ethn/                    # Ethnicity representation regression
│   │   │   └── regression.ipynb
│   │   └── pbs_csv/                 # Controllable preference data
│   │       └── X/                   # Organized by attribute
│   ├── RewardModelFigure/           # Reward model analysis figures
│   └── README.md                    # Figure generation documentation
│
├── RewardDatasetAnalysis/           # 🗂️ Human preference dataset bias analysis
│   ├── reward_datasets_analysis.ipynb  # Main analysis notebook
│   ├── HPDv2/                       # HPDv2 dataset analysis
│   │   ├── data/                    # Filtered data by gender
│   │   │   ├── hpdv2_train_filter_man_action_mapped.json
│   │   │   ├── hpdv2_train_filter_woman_action_mapped.json
│   │   │   └── hpdv2_train_filter_person_action_mapped.json
│   │   ├── qwen2_vl_7b_results.json    # VLM inference results
│   │   ├── qwen2_5_vl_7b_results.json
│   │   └── internvl2.5_8b_results.json
│   ├── PickaPic/                    # Pick-a-Pic dataset analysis
│   │   ├── data/                    # Filtered data by gender
│   │   │   ├── pick_a_pic_train_filter_man.json
│   │   │   ├── pick_a_pic_train_filter_woman.json
│   │   │   └── pick_a_pic_train_filter_person.json
│   │   ├── qwen2_vl_7b_results.json
│   │   ├── qwen2_5_vl_7b_results.json
│   │   └── internvl2.5_8b_results.json
│   └── figure/                      # Analysis visualizations
│       ├── reward_dataset_preference_gender.pdf.png
│       └── reward_dataset_preference_bias_dist_ethn.pdf.png
│
├── Results/                         # Aggregated VLM inference results
│   ├── qwen2_vl/
│   ├── qwen2_5_vl/
│   └── internvl_chat/
│
├── figures_png/                     # PNG exports of publication figures
│   └── videobias_overview.png
│
├── convert_pdf_to_png.py            # Utility for PDF → PNG conversion
├── .gitignore                       # Git ignore rules
├── LICENSE                          # MIT License
└── README.md                        # ⭐ This file (main documentation)

Key Directories:

• VideoGen: Generate videos with 9+ models, produces .mp4 files and metadata .json
• PhotoGen: Generate synthetic images with FLUX for reward model bias evaluation
• RewardModelAnalysis: Evaluate bias in reward models (CLIP, HPSv2.0, ImageReward, PickScore, InternVideo)
• AlignmentTuning: Training code for alignment tuning experiments (T2V-Turbo-v1/v2)
• VideoEval: Extract frames and run VLM ensemble for bias evaluation
• Figures: Statistical analysis and publication figure generation
• RewardDatasetAnalysis: Analysis of human preference datasets (HPDv2, Pick-a-Pic) for gender/ethnicity bias
• Results: Centralized VLM inference results across all experiments

🔄 Complete Research Pipeline

The repository supports the complete bias analysis pipeline across three stages:

Stage 1: Human Preference Dataset Analysis

RewardDatasetAnalysis/ → Analyze HPDv2 & Pick-a-Pic datasets for inherent biases

Stage 2: Reward Model Bias Evaluation

PhotoGen/ → Generate synthetic images with FLUX
    ↓
RewardModelAnalysis/ → Evaluate 6 reward models (CLIP, HPSv2, ImageReward, etc.)
    ↓
Figures/RewardModelCSV/ → Quantify reward model bias metrics

Stage 3: Video Generation Model Evaluation

VideoGen/ → Generate videos with 9+ models
    ↓
VideoEval/ → Extract frames & run VLM ensemble
    ↓
Figures/MainFigureCSV/ → Compute bias metrics (TAS, PBS_G, RDS_e, SDI)

Stage 4: Alignment Tuning Analysis

AlignmentTuning/ → Train models with biased reward functions
    ↓
VideoGen/ → Generate videos from aligned models
    ↓
VideoEval/ → Evaluate alignment impact
    ↓
Figures/AlignmentTuningCSV/ → Measure bias amplification

Stage 5: Statistical Analysis

Figures/MetricRegression/ → Regression analysis & correlation studies
    ↓
Figures/FinalFigures/ → Publication-ready visualizations

💡 Why This Matters

"As video AI becomes mainstream, these biases will shape how millions see the world."

• 🎥 Video generation is the future of content creation
• ⚖️ Undetected biases propagate harmful stereotypes at scale
• 🔬 Our framework enables bias detection before deployment
• 🛠️ Controllable solutions show bias can be mitigated

🚀 Quick Start: Evaluate Your Own Video Model

Prerequisites

• Python 3.8+
• CUDA-capable GPU(s) with sufficient VRAM (16-24GB recommended)
• CUDA 11.8+ and cuDNN installed

1. Installation

# Clone the repository
git clone https://github.com/yourusername/VideoBiasEval.git
cd VideoBiasEval

# Create conda environment for VideoGen
conda create -n videobias python=3.8
conda activate videobias

# Install core dependencies
cd VideoGen
pip install -r requirements/requirements_all.txt

# Install Apex (required for some models)
git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --no-build-isolation \
    --config-settings "--build-option=--cpp_ext" \
    --config-settings "--build-option=--cuda_ext" ./
cd ..

2. Model-Specific Setup

Each model requires its own setup. Here are examples for popular models:

Example: Pyramid Flow

# Prepare model
python scripts_prepare/prepare_pyramid_flow.py --model_path /path/to/models

# Configure model settings in config/models/pyramid_flow.json
# - model_dtype: bf16 or fp16
# - model_variant: diffusion_transformer_768p or diffusion_transformer_384p
# - sp_group_size: number of GPUs to use

Example: VideoCrafter

# Create environment
conda create -n videocrafter python=3.8.5 -y 
conda activate videocrafter
pip install -r requirements/requirements_video_crafter.txt

# Download model checkpoint
mkdir -p CKPT/base_512_v2
huggingface-cli download VideoCrafter/VideoCrafter2 --local-dir CKPT/base_512_v2/

# Set checkpoint path in config/models/video_crafter.json

See VideoGen/README.md for detailed setup instructions for all 9+ supported models.

3. Generate Videos

cd VideoGen

# Example: Generate videos with Pyramid Flow
bash scripts/inference_pyramid_flow.sh

# Example: Generate videos with VideoCrafter
bash scripts/inference_video_crafter.sh 1 0 1 gender_only_prompts_long_env

# Videos will be saved to: outputs_{model_name}/r{run_id}/{setting}/*.mp4
# Metadata will be saved to: config/videos/{model_name}/r{run_id}/{setting}.json

4. Extract Frames and Run VLM Evaluation

cd ../VideoEval

# Automated pipeline (recommended)
bash run_evaluation.sh

# Or run manually step-by-step:
# Step 1: Extract 16 frames per video
bash extract_frames.sh

# Step 2: Run VLM inference for attribute extraction
export CUDA_VISIBLE_DEVICES=0,1,2,3
bash inference_vllm.sh

The evaluation pipeline:

1. ✅ Extracts 16 evenly-spaced frames from each video
2. 🤖 Runs VLM ensemble (Qwen2-VL, Qwen2.5-VL, InternVL2.5) for attribute detection
3. 📊 Applies majority voting across frames and ensemble fusion across models

5. Compute Bias Metrics

cd VideoEval

# Install notebook dependencies
pip install jupyter spacy matplotlib pandas
python -m spacy download en_core_web_sm

# Launch metrics notebook
jupyter notebook metrics.ipynb

The notebook computes four key metrics:

• TAS: Temporal Attribute Stability (consistency across frames)
• PBS_G: Proportion Bias Score for Gender (gender bias per action)
• RDS_e: Representation Deviation Score for Ethnicity (over/underrepresentation)
• SDI: Simpson's Diversity Index (overall diversity)

🧬 The VideoBiasEval Framework

🎯 Three-Stage Evaluation Pipeline

graph LR
    A[Event Prompts] --> B[Video Generation]
    B --> C[VLM Ensemble]
    C --> D[Bias Metrics]

1️⃣ Event-based Prompting: 42 Actions × 7 Ethnicities × 4 Genders = 1,176 Test Cases

We scientifically selected actions based on U.S. labor statistics:

• Male-associated: exercise, drive, lift (+15-20% male in workforce)
• Female-associated: bake, clean, kneel (+15-20% female in workforce)
• Neutral actions: eat, stand, walk (balanced representation)

2️⃣ State-of-the-Art Attribute Detection

We use a robust VLM ensemble with temporal aggregation for accurate attribute classification:

Detection Pipeline:

1. Frame Extraction: Extract 16 evenly-spaced frames from each video
2. VLM Inference: Each of 3 VLMs classifies gender/ethnicity per frame
3. Temporal Majority Voting: Determine video-level attribute from 16 frame predictions per VLM
4. Ensemble Fusion: Combine predictions from 3 VLMs using majority voting
5. Confidence Scoring: Track prediction agreement across frames and models

Why This Works:

• Temporal aggregation reduces noise from single-frame misclassifications
• Ensemble fusion leverages complementary strengths of different VLMs
• High correlation with humans validates automated evaluation approach
• Scalable to thousands of videos without manual annotation

3️⃣ Novel Multi-Granular Metrics

We introduce four complementary metrics to capture different aspects of bias:

Metric Details:

• TAS (Temporal Attribute Stability): Measures consistency of social attributes across video frames. Low TAS indicates "flickering" or unstable identity depiction, which can be problematic for coherent narrative representation.

• PBS_G (Proportion Bias Score for Gender): Quantifies gender bias within ethnicity groups for each action. Reveals stereotypical associations (e.g., "cooking" with female, "lifting" with male).

• RDS_e (Representation Deviation Score for Ethnicity): Compares actual ethnicity distribution against uniform baseline. Exposes systemic over/underrepresentation across model outputs.

• SDI (Simpson's Diversity Index): Captures overall demographic diversity. Low SDI indicates homogeneous outputs lacking representational variety.

🚀 9+ Video Models Evaluated

Model Pairing for Alignment Analysis:

• Base → Aligned: ModelScope → InstructVideo
• Base → Aligned: VideoCrafter → T2V-Turbo-V1
• Base → Aligned: VideoCrafter → VADER-VideoCrafter

Each model requires specific environment setup. See VideoGen/README.md for detailed installation instructions.

📈 Reproducing Results

Complete Workflow

Step 1: Generate Videos with Multiple Models

cd VideoGen

# Example models to evaluate
models=("pyramid_flow" "video_crafter" "t2v_turbo" "open_sora")

# Generate videos for each model (10 runs per setting)
for model in "${models[@]}"; do
    for run_id in {1..10}; do
        bash scripts/inference_${model}.sh ${run_id} 0 1 gender_ethn_prompts_long_env
    done
done

# Videos saved to: outputs_{model}/r{run_id}/gender_ethn_prompts_long_env/*.mp4

Step 2: Extract Attributes with VLM Ensemble

cd ../VideoEval

# Configure evaluation settings in run_evaluation.sh:
# - SETTINGS: prompt configurations to evaluate
# - MODEL_FOLDERS: model output directories to process
# - RUN_IDS: which runs to include (r1, r2, ..., r10)
# - VLM_MODELS: which VLMs to use (qwen2_vl, internvl_chat, qwen2_5_vl)

# Run automated evaluation pipeline
bash run_evaluation.sh

# This performs:
# 1. Frame extraction (16 frames per video)
# 2. VLM inference with ensemble voting
# 3. Result aggregation and logging

Step 3: Compute Bias Metrics

cd VideoEval

# Launch metrics notebook
jupyter notebook metrics.ipynb

# In the notebook, run:
# - evaluate_model_comprehensive() for each model
# - Generates TAS, PBS_G, RDS_e, SDI metrics
# - Outputs CSV files and visualizations

Step 4: Generate Paper Figures

cd ../Figures/MetricRegression

# Main regression analysis
jupyter notebook regression.ipynb

# Gender-specific analysis
jupyter notebook gender/regression.ipynb

# Ethnicity-specific analysis
jupyter notebook ethn/regression.ipynb

# Figures saved to: ../FinalFigures/

Specific Experiments

0. Alignment Tuning with Reward Model

# setup
cd AlignmentTuning/t2v-turbo/
conda env create -f cog.yaml
conda activate t2v-turbo

# download checkpoints：biased reward model + VideoCrafter2
huggingface-cli download --repo-type dataset ZefanCai/VideoBias HPSv2_pt/rm_image_pairs_man_pref_100.pt --local-dir .
huggingface-cli download VideoCrafter/VideoCrafter2 model.ckpt --local-dir .

# copy vocab
ENV_SITE_PACKAGES=$(python -c "import site; print(site.getsitepackages()[0])")
cp bpe_simple_vocab_16e6.txt.gz "$ENV_SITE_PACKAGES/hpsv2/src/open_clip/bpe_simple_vocab_16e6.txt.gz"


# prepare data ([WebVid-10M](https://github.com/m-bain/webvid) data. Save in the `webdataset` format.)
# edit train_t2v_turbo_v1.sh
# train_shards_path_or_url="/path/to/webvid10m/webvid-train-{000000..000999}.tar"

# train
bash train_t2v_turbo_v1.sh

1. Reward Model Bias Evaluation

Evaluate bias in reward models using synthetic images:

cd PhotoGen/generate-images

# Step 1: Generate synthetic images with FLUX
# Configure settings in generate.sh (model, prompts, etc.)
bash generate.sh

# Images are uploaded to HuggingFace dataset
# Dataset URL: https://huggingface.co/datasets/videobias/eval

cd ../../RewardModelAnalysis

# Step 2: Evaluate reward models on generated images
# Supported models: clip, hpsv2.0, hpsv2.1, image_reward, pick_score, intern_video

# Example: Evaluate HPSv2.0 on gender setting with long environment
MODEL_NAME="hpsv2.0"
GPU_ID=0
SETTING_NUM=2
ENV_TYPE="long"
PEOPLE_TYPE="gender"
OUTPUT_PATH="score_${MODEL_NAME}_setting_${SETTING_NUM}_${PEOPLE_TYPE}_${ENV_TYPE}.json"

bash reward_one.sh

# Results saved to: output/score_{model}_{setting}_{people_type}_{env_type}.json

# Step 3: Analyze results in Figures/RewardModelFigure/
cd ../Figures/RewardModelFigure
# Use notebooks to visualize reward model bias patterns

2. Alignment Tuning Analysis

Evaluate how alignment tuning affects bias:

cd VideoGen

# Evaluate base models vs aligned variants
# ModelScope → InstructVideo
bash scripts/inference_modelscope.sh
bash scripts_alignment/inference_instructvideo.sh

# VideoCrafter → T2V-Turbo-V1
bash scripts/inference_video_crafter.sh
bash scripts/inference_t2v_turbo.sh

# Run evaluation pipeline on both
cd ../VideoEval
bash run_evaluation.sh

# Compare metrics in metrics.ipynb

3. Reward Model Controllability

Test controlled bias manipulation:

cd VideoGen/scripts_alignment

# Evaluate with HPSv2.0 (male-biased)
bash evaluate_hpsv2.sh

# Evaluate with Pick-a-Pic (female-biased)
bash evaluate_pick.sh

# Analyze controllability in Figures/MetricRegression/pbs_csv/

4. Reward Dataset Bias Analysis

Analyze biases in human preference datasets (HPDv2 and Pick-a-Pic):

cd RewardDatasetAnalysis

# Launch the analysis notebook
jupyter notebook reward_datasets_analysis.ipynb

# The notebook provides:
# - Gender/ethnicity distribution analysis across preference pairs
# - VLM-based attribute extraction from preferred images
# - Bias quantification (preference bias scores, representation metrics)
# - Visualization of bias patterns by action categories

Dataset Structure:

• HPDv2: Human Preference Dataset v2
• Pick-a-Pic: Community-driven preference dataset with gender-specific analysis

Analysis Output:

• Gender preference distributions per action (e.g., cooking, driving)
• Ethnicity representation across preferred vs. non-preferred images
• Statistical significance tests for bias patterns
• Publication figures saved to figure/

Key Findings:

• HPDv2 shows 62% male preference across actions
• Pick-a-Pic shows 58% female preference across actions
• Both datasets exhibit 40-43% White overrepresentation
• Action-specific biases align with societal stereotypes

🔧 Troubleshooting

Common Issues and Solutions

CUDA Out of Memory

• Symptoms: RuntimeError: CUDA out of memory during VLM inference
• Solutions:
  • Reduce batch size in VideoEval/inference_vllm.py (default: 5000 → 2000)
  • Use fewer GPUs or reduce tensor parallel size
  • Process models sequentially instead of in parallel
  • Use GPUs with larger VRAM (24GB recommended)

Video Files Not Found

• Symptoms: WARNING: Video file not found during frame extraction
• Solutions:
  • Verify video generation completed successfully
  • Check paths in VideoGen/config/videos/{model}/{run_id}/{setting}.json
  • Ensure VideoGen and VideoEval are in the same parent directory
  • Check file permissions

Model Download Failures

• Symptoms: HuggingFace download errors
• Solutions:
  • Check internet connection
  • Set HuggingFace token: export HF_TOKEN="your_token"
  • Some models require access approval on HuggingFace

Slow Frame Extraction

• Solutions:
  • Use hardware-accelerated video decoding if available
  • Process videos in parallel: parallel -j 4 python extract_frames.py ...
  • Expected speed: ~0.5-1 second per video

Inference Process Stuck

• Solutions:
  • Check GPU availability: nvidia-smi
  • Monitor log files: tail -f VideoEval/outputs_*/*/Log/*.txt
  • Verify model is not still downloading
  • Check for dead processes: ps aux | grep python

Debug Mode

Enable verbose logging:

# In VideoEval
bash run_evaluation.sh --verbose

# Check detailed logs
tail -f VideoEval/outputs_*/*/qwen2_vl/Log/*_log.txt

🏆 Key Contributions

1. 🔬 First Systematic Analysis: We're the first to trace bias propagation through the entire alignment pipeline
2. 📏 Novel Evaluation Framework: VideoBiasEval provides standardized bias measurement for video AI
3. 💾 Large-Scale Dataset: 2.94M curated preference pairs for controlled experiments
4. 🎮 Controllable Solutions: Demonstrated that bias can be systematically controlled
5. 🤖 VLM Ensemble Method: Robust attribute extraction with 89% human agreement (gender)

📝 Citation

@article{cai2025videobias,
  title={From Preferences to Prejudice: The Role of Alignment Tuning in 
         Shaping Social Bias in Video Diffusion Models},
  author={Cai*, Zefan and Qiu*, Haoyi and Zhao*, Haozhe and 
          Wan, Ke and Li, Jiachen and Gu, Jiuxiang and 
          Xiao, Wen and Peng, Nanyun and Hu, Junjie},
  journal={arXiv preprint},
  year={2025}
}

🤝 Contribute to Fair AI

We believe in community-driven solutions to AI bias. Here's how you can help:

🐛 Found a Bias?

Report it! Open an issue with:

• Model name and version
• Prompt used
• Observed bias pattern
• Demographic group affected

💡 Have Ideas?

• Propose new evaluation metrics
• Suggest bias mitigation strategies
• Contribute evaluation code
• Improve VLM ensemble methods

🔧 Want to Add Your Model?

We welcome contributions of new video generation models! Follow these steps:

1. Implement the Model Interface

Create a new file in VideoGen/models/your_model.py:

class YourModelGenerator:
    def __init__(self, config):
        """Initialize your model with config parameters"""
        self.config = config
        # Load model checkpoint, set up pipeline, etc.
        
    def generate(self, prompt, output_path, **kwargs):
        """
        Generate video from text prompt
        
        Args:
            prompt (str): Text prompt
            output_path (str): Path to save video
            **kwargs: Additional model-specific parameters
            
        Returns:
            str: Path to generated video
        """
        # Your generation logic here
        pass

2. Add Model Configuration

Create VideoGen/config/models/your_model.json:

{
    "model_name": "your_model",
    "model_path": "/path/to/checkpoint",
    "model_dtype": "bf16",
    "num_inference_steps": 50,
    "guidance_scale": 7.5,
    "video_length": 16,
    "resolution": [512, 512]
}

3. Create Inference Script

Create VideoGen/scripts/inference_your_model.sh:

#!/bin/bash
RUN_ID=${1:-1}
START_IDX=${2:-0}
END_IDX=${3:-1}
SETTING=${4:-"gender_only_prompts_long_env"}

export CUDA_VISIBLE_DEVICES=0

python generate.py \
    --config config/models/your_model.json \
    --prompts config/prompts/${SETTING}.json \
    --output_video_path outputs_your_model/r${RUN_ID}/${SETTING}/ \
    --run_id ${RUN_ID} \
    --start_idx ${START_IDX} \
    --end_idx ${END_IDX}

4. Add Requirements

Create VideoGen/requirements/requirements_your_model.txt with dependencies.

5. Test Your Implementation

# Generate test videos
cd VideoGen
bash scripts/inference_your_model.sh 1 0 10 gender_only_prompts_long_env

# Run evaluation
cd ../VideoEval
bash run_evaluation.sh

# Compute metrics
jupyter notebook metrics.ipynb

6. Submit Pull Request

Include in your PR:

• Model implementation and tests
• Configuration files
• Installation instructions in VideoGen/README.md
• Sample outputs (if possible)
• Bias evaluation results

We especially welcome:

• New open-source video models
• Models with novel architectures
• Models trained with different alignment strategies
• Models targeting specific domains (medical, educational, etc.)

🌟 Star History

📄 License

MIT License - see LICENSE

⚠️ Ethical Considerations

This research aims to expose and mitigate biases, not perpetuate them. We encourage:

• 🔍 Transparent bias reporting in all video AI systems
• 🌈 Diverse representation in training data
• 🤝 Inclusive development teams
• 📊 Regular bias audits using frameworks like ours

📖 Quick Reference

Essential Commands

# Setup
conda create -n videobias python=3.8 && conda activate videobias
cd VideoGen && pip install -r requirements/requirements_all.txt

# Generate videos (example: VideoCrafter, run 1, prompts 0-100)
cd VideoGen
bash scripts/inference_video_crafter.sh 1 0 100 gender_ethn_prompts_long_env

# Run evaluation pipeline
cd ../VideoEval
bash run_evaluation.sh  # Automated: frame extraction + VLM inference

# Compute metrics
jupyter notebook metrics.ipynb

Key Files

Configuration Tips

For VideoGen:

• Edit model configs: VideoGen/config/models/{model}.json
• Adjust GPU allocation: Set CUDA_VISIBLE_DEVICES in inference scripts
• Change prompt settings: Modify VideoGen/config/prompts/*.json

For VideoEval:

• Configure models to evaluate: Edit MODEL_FOLDERS in run_evaluation.sh
• Select VLMs: Edit VLM_MODELS array (qwen2_vl, internvl_chat, qwen2_5_vl)
• Skip completed steps: Use --skip-frame-extraction or --skip-inference flags

For PhotoGen:

• Configure FLUX model: Edit generate.sh to set model path and API keys
• Customize prompts: Modify preprocess-prompts/*.py for new prompt templates
• Adjust image settings: Edit generate_images.py for resolution, guidance scale, etc.

For RewardModelAnalysis:

• Select reward model: Set MODEL_NAME in reward_one.sh (clip, hpsv2.0, image_reward, etc.)
• Configure evaluation: Set SETTING_NUM, ENV_TYPE, PEOPLE_TYPE in reward_one.sh
• Download models: Use src/download_model.py and src/download_ckpt.py

For AlignmentTuning:

• Configure training data: Edit train_shards_path_or_url in train_t2v_turbo_v1.sh
• Set reward model: Modify reward_fn/ to use different reward models
• Adjust hyperparameters: Edit configs in configs/ directory

Useful Debugging Commands

# Check GPU status
nvidia-smi
watch -n 1 nvidia-smi  # Monitor continuously

# View evaluation logs
tail -f VideoEval/outputs_*/*/qwen2_vl/Log/*_log.txt

# Verify generated videos
ls -lh VideoGen/outputs_video_crafter/r1/gender_ethn_prompts_long_env/*.mp4

# Check frame extraction status
ls -lh VideoEval/outputs_video_crafter/r1/gender_ethn_prompts_long_env_frames/

# Check reward model evaluation results
ls -lh RewardModelAnalysis/output/

# Monitor alignment tuning progress
tail -f AlignmentTuning/t2v-turbo/logs/training.log

# Verify PhotoGen image generation
ls -lh PhotoGen/generate-images/outputs/

Documentation Links

• VideoGen Details: VideoGen/README.md - Model setup, inference scripts
• VideoEval Details: VideoEval/README.md - Evaluation pipeline, VLM inference
• AlignmentTuning Details: AlignmentTuning/t2v-turbo/README.md - T2V-Turbo training and alignment tuning
• PhotoGen Details:
  • PhotoGen/preprocess-prompts/README.md - Prompt preprocessing
  • PhotoGen/encode-prompts/README.md - Prompt encoding for reward models
• Figures README: Figures/README.md - Statistical analysis and visualization
• RewardDatasetAnalysis: RewardDatasetAnalysis/reward_datasets_analysis.ipynb - Human preference dataset bias analysis

---