VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understanding

Ahmad Mahmood*, Ashmal Vayani*, Muzammal Naseer, Salman Khan, and Fahad Khan

* Equally contributing first authors

---

📢 Latest Updates

• Mar-26-24: VURF paper uploaded on arXiv. Code will be released soon!
• Dec-13-24: Interpreters for video functions added.

Environment

To set up the environment for VURF, follow these steps:

1. SeViLA Setup

   • Visit the SeViLA repository and follow the instructions to set up the environment.

2. Tracking Modules

   • For the Tracking module, install the dependencies by following the steps in the Tracking-Anything-with-DEVA repository.

3. Video-ChatGPT Installation

   • Install Video-ChatGPT from the Video-ChatGPT repository.

Make sure to follow the installation guidelines provided in each repository to ensure compatibility with VURF.

📜 Citation

  @misc{mahmood2024vurf,
      title={VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understanding}, 
      author={Ahmad Mahmood and Ashmal Vayani and Muzammal Naseer and Salman Khan and Fahad Shahbaz Khan},
      year={2024},
      eprint={2403.14743},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Disclaimers

This repository is based on and modified from Visprog