Introduction

• Paper: Empowering Reliable Visual-Centric Instruction Following in MLLMs
• Github: KerenWLHe/VC-IFEval

😊 This is the official repo of VC-IFEngine datasets.

📊 Image sources: LLaVA-Instruct, Visual Genome, ALLaVA-4V

Format

SFT — messages format (role: user / assistant):

{
  "messages": [
    {"role": "user", "content": "<image>\n{instruction}"},
    {"role": "assistant", "content": "{response}"}
  ],
  "images": ["Data/CC3M/images/example.jpg"]
}

DPO — conversations + chosen / rejected format:

{
  "conversations": [
    {"from": "human", "value": "<image>\n{instruction}"}
  ],
  "chosen": {"from": "gpt", "value": "{full-constraint response}"},
  "rejected": {"from": "gpt", "value": "{constraint-degraded response}"},
  "images": ["Data/CC3M/images/example.jpg"]
}

Citation

@misc{he2026empoweringreliablevisualcentricinstruction,
      title={Empowering Reliable Visual-Centric Instruction Following in MLLMs}, 
      author={Weilei He and Feng Ju and Zhiyuan Fan and Rui Min and Minhao Cheng and Yi R. Fung},
      year={2026},
      eprint={2601.03198},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2601.03198}, 
}