MAI on MT-FashionIQ

This repository reproduces MAI on MT-FashionIQ for training and evaluation:

• Paper: MAI: A Multi-turn Aggregation-Iteration Model for Composed Image Retrieval
• Official MAI repo: PKU-ICST-MIPL/MAI_ICLR2025
• MT-FashionIQ source repo: yfyuan01/MultiturnFashionRetrieval

1) Project Scope

This codebase is used to:

• train MAI on converted MT-FashionIQ data;
• test MAI on MT-FashionIQ validation sets;
• support both:
  • legacy fixed-turn compose format (e.g., pad2 files), and
  • paper-style variable-turn format (Fiq_train_all.json / Fiq_val_all.json).

2) Repository Structure

Top-level layout:

• src/: model, dataset loading, training, validation, testing
• scripts/: data conversion scripts
• dataset/MT-FashionIQ/: converted JSON files, index files, images
• outputs/: checkpoints and logs

Common entry files:

• Training: src/train_ddp.py
• Testing: src/test.py
• Core model: src/lavis/models/blip2_models/blip2_qformer_cir_align_prompt.py

3) MT-FashionIQ Images

MT-FashionIQ images come from:

• SIGIR_images.zip (Google Drive)

Place extracted images under:

• dataset/MT-FashionIQ/images/

4) Required Data Files

For paper-style variable-turn training/testing, make sure these files exist:

• dataset/MT-FashionIQ/Fiq_train_all.json
• dataset/MT-FashionIQ/Fiq_val_all.json
• dataset/MT-FashionIQ/Fiq_index_names.txt

5) Train MAI (Paper Mode)

Example (single node, 4 GPUs):

CUDA_VISIBLE_DEVICES=2,3,4,5 \
python -m torch.distributed.launch \
  --nproc_per_node=4 \
  --master_port 29505 \
  src/train_ddp.py \
  --dataset fiq \
  --fiq-turn-fusion-mode paper \
  --fiq-max-turns 4 \
  --blip-model-name blip2_cir_align_prompt \
  --num-epochs 50 \
  --batch-size 16 \
  --learning-rate 1e-5 \
  --num-workers 8 \
  --output-dir outputs/ckpt

Checkpoint output format:

• outputs/ckpt/blip_fiq/<MM-DD-HH>/epoch<E>.pth

6) Test MAI on Fiq_val_all.json (Paper Mode)

After training, evaluate with:

CUDA_VISIBLE_DEVICES=2 \
python src/test.py \
  --dataset fiq \
  --fiq-turn-fusion-mode paper \
  --model-path outputs/ckpt/blip_fiq/03-30-10/epoch49.pth

If needed, use your own checkpoint path by replacing --model-path.