AttackWM

AttackWM studies adversarial attacks on diffusion-based driving world models, with a primary focus on practical input perturbation attacks and a stronger white-box latent manipulation reference. The project is built on top of ReSim, and uses target videos to evaluate whether a driving world model can be pushed toward unsafe or physically implausible future rollouts.

The current codebase includes:

• Black-box-style input attacks that perturb only the three-frame prefix latent.
• A white-box latent steering attack that intervenes directly in the denoising process.
• A PhysCond-WMA baseline adapted to the ReSim pipeline.
• Evaluation utilities for target alignment, FID, FVD, and qualitative inspection.

Overview

We study two attack surfaces on the same driving world model:

• Input attack: perturb the input prefix latent under a small L_\infty budget while keeping commands, trajectories, and model weights fixed.
• Latent steering: directly steer intermediate diffusion latents toward a target event during denoising.

In our experiments, input attacks remain visually close to clean generations and produce only limited target shifts, while latent steering is substantially stronger but also assumes a much more privileged threat model.

Visual Results

Qualitative comparison between ground truth, clean generation, black-box input attack, and white-box latent steering:

Optimization dynamics of the input-space attacks:

• input_attack_dynamics.pdf

Latent-steering perturbation statistics:

• rms.pdf

Repository Layout

• ReSim/: underlying driving world model code and attack implementations.
• ReSim/sat/input_attack.py: input-attack optimization logic.
• ReSim/sat/latent_steering.py: white-box latent steering attack.
• ReSim/sat/physcond_wma.py: PhysCond-WMA-style baseline utilities.
• ReSim/sat/configs/: runnable config overlays for different attacks.
• progress/: experiment notes, ablations, and implementation logs.

Setup

The project follows the ReSim environment. A minimal setup is:

conda create -n resim python=3.10 -y
conda activate resim

pip install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124
pip install -r ReSim/requirements.txt

cd ReSim/SwissArmyTransformer
pip install -e . --no-build-isolation
cd ../..

You will also need:

• ReSim-compatible checkpoints
• dataset annotations for inference
• per-clip target videos for targeted attacks

Detailed checkpoint and data preparation instructions are in ReSim/README.md.

Running Attacks

All sampling commands are launched from ReSim/sat through inference_custom.sh.

1. Clean ReSim inference

cd ReSim/sat
bash inference_custom.sh configs/infer_nus.yaml

2. Input attack

Scheme 1 with a 5-step surrogate:

cd ReSim/sat
bash inference_custom.sh configs/infer_nus.yaml configs/input_attack_scheme0_surrogate.yaml

Scheme 1 with a 10-step surrogate:

cd ReSim/sat
bash inference_custom.sh configs/infer_nus.yaml configs/input_attack_scheme0_surrogate_10steps.yaml

Scheme 2:

cd ReSim/sat
bash inference_custom.sh configs/infer_nus.yaml configs/input_attack_scheme2_denoised_steps.yaml

Scheme 3:

cd ReSim/sat
bash inference_custom.sh configs/infer_nus.yaml configs/input_attack_scheme4_momentum.yaml

3. White-box references

Latent steering:

cd ReSim/sat
bash inference_custom.sh configs/infer_nus.yaml configs/latent_steering_override.yaml configs/smoke_latent_steering.yaml

PhysCond-WMA baseline:

cd ReSim/sat
bash inference_custom.sh configs/infer_nus.yaml configs/physcond_wma_override.yaml

Configuration Notes

Common attack-related fields live in the config overlays under ReSim/sat/configs:

• attack_method: chooses input_attack, latent_steering, or physcond_wma.
• attack_targeted: enables target-conditioned attack mode.
• attack_target_path: path to the target video; {token} can be used for per-clip matching.
• attack_input_eps and attack_input_eps_mode: perturbation budget for input attacks.
• attack_input_steps, attack_input_lr, and related surrogate settings: PGD hyperparameters.
• steer_eps: latent-steering budget.

The example configs in this repo are a good starting point, but most path fields need to be changed for a new machine.

Outputs

A typical attack run writes:

• generated videos such as SampleAttack, SampleClean, GT, and TargetInput
• attack metadata JSON with per-step statistics
• saved perturbations such as attack_delta.pt

For input attacks, the metadata includes:

• final target_loss: target-alignment loss on the generated future
• clean_loss: distance from attacked output to clean output
• per-iteration surrogate loss, target term, gradient norm, and perturbation norm

Evaluation

The repository includes comparison utilities for batch evaluation. In particular:

• ReSim/sat/compare_eval.py: aggregates target loss, FID, and FVD across runs

For the exact experiment configurations, interpretation of each scheme, and the logged statistics, see:

• progress/2026-06-18_input_attack_schemes.md

Further Reading

If you want more detail without expanding this README too much, these are the best entry points:

• ReSim/README.md: model installation, checkpoints, and base inference
• docs/ReSim_inference.md: practical inference notes for this repository
• docs/development.md: implementation details and development notes
• docs/token204_runbook.md: a concrete single-clip debugging and smoke-test workflow
• progress/2026-06-03_physcond_wma_first_version.md: PhysCond-WMA integration and early baseline notes
• progress/2026-06-17_latent_steering_first_version.md: latent-steering implementation and smoke-test notes
• progress/2026-06-18_input_attack_schemes.md: input-attack variants, formulas, and results