README
CRAFT-MAGMaR
MAGMaR-2026 plus the CRAFT-specific artefacts our pipeline needs end-to-end. Sources: the MAGMaR-2026 test release from akhilvssg/magmar-2026-test-asr-embeddings
- content produced by the CRAFT pipeline (chunking, multilingual ASR, adaptive keyframe selection).
Contents
| Path | What it is |
|---|---|
en/ | All MAGMaR videos in one flat directory: 90 original mp4s plus 72 CRAFT-produced <video_id>__chunkNNN.mp4 splits. Point VIDEO_ROOT here. |
asr/ | Per-video transcripts (<video_id>.json) from Qwen3-ASR-1.7B with omniASR-LLM-7B fallback for Burmese / Nepali. Schema: {video_id, asr_model, language, text, text_en, needs_fallback, no_audio, asr_loop_detected}. |
aks/outscores/ | DKS / AKS keyframe scores per (query, video) pair (CLIP-based). |
aks/selected_frames/ | Selected keyframe indices per (query, video) pair, top-64. |
MAGMaR2026_queries.jsonl | Official MAGMaR-2026 test queries (19 personas). |
MAGMaR2026_queries_dev.jsonl | Official MAGMaR-2026 dev queries (8 personas). |
topic_video_mapping_v2.json | Test split: topic / event -> list of (chunked) video IDs. |
topic_video_mapping_dev_v2.json | Dev split: topic / event -> list of (chunked) video IDs. |
video_chunk_map.json | chunk_id -> {video_id, start, end}, used to remap chunked citations back to parent video IDs at output-formatting time. |
This bundle does not redistribute the MAGMaR-2026 ground-truth labels. Score your runs with the MIRAGE judge against the official ground truth obtained directly from the MAGMaR workshop.
Usage with CRAFT
# Download
hf download mbhosale/CRAFT-MAGMaR --repo-type dataset \
--local-dir /path/to/MAGMaR2026_test
# Run CRAFT against it (no chunking needed, ships pre-chunked)
git clone https://github.com/bhosalems/CRAFT.git && cd CRAFT
conda create -n craft python=3.13 -y && conda activate craft
pip install -r requirements.txt
SKIP_CHUNK=1 \
VIDEO_ROOT=/path/to/MAGMaR2026_test \
ASR_DIR=/path/to/MAGMaR2026_test/asr \
PARALLEL_QUERIES=8 PARALLEL_STEP15=8 PARALLEL_STEP5=8 \
bash run_query.sh outputs/craft_magmar_main
See the CRAFT repository for the full pipeline (Stage 1b VLM extraction, hybrid critic loop, UNLI calibration, higher-level inference, report assembly) and the MIRAGE evaluator setup.
Licensing
The base videos and queries come from the MAGMaR-2026 test release at akhilvssg/magmar-2026-test-asr-embeddings. Please honour that dataset's license for any downstream redistribution. CRAFT-produced artefacts (chunks, ASR cache, AKS outputs) are released under the same terms.
Citation
@article{bhosale2026craft,
title={CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answering},
author={Bhosale, Mahesh and Wasi, Abdul and Trivedi, Vishvesh and Yan, Pengyu and Gorugantu, Akhil and Doermann, David},
journal={arXiv preprint arXiv:2605.19075},
year={2026}
}
Contributors
5 commits
Similar repos
README
CRAFT-MAGMaR
MAGMaR-2026 plus the CRAFT-specific artefacts our pipeline needs end-to-end. Sources: the MAGMaR-2026 test release from akhilvssg/magmar-2026-test-asr-embeddings
- content produced by the CRAFT pipeline (chunking, multilingual ASR, adaptive keyframe selection).
Contents
| Path | What it is |
|---|---|
en/ | All MAGMaR videos in one flat directory: 90 original mp4s plus 72 CRAFT-produced <video_id>__chunkNNN.mp4 splits. Point VIDEO_ROOT here. |
asr/ | Per-video transcripts (<video_id>.json) from Qwen3-ASR-1.7B with omniASR-LLM-7B fallback for Burmese / Nepali. Schema: {video_id, asr_model, language, text, text_en, needs_fallback, no_audio, asr_loop_detected}. |
aks/outscores/ | DKS / AKS keyframe scores per (query, video) pair (CLIP-based). |
aks/selected_frames/ | Selected keyframe indices per (query, video) pair, top-64. |
MAGMaR2026_queries.jsonl | Official MAGMaR-2026 test queries (19 personas). |
MAGMaR2026_queries_dev.jsonl | Official MAGMaR-2026 dev queries (8 personas). |
topic_video_mapping_v2.json | Test split: topic / event -> list of (chunked) video IDs. |
topic_video_mapping_dev_v2.json | Dev split: topic / event -> list of (chunked) video IDs. |
video_chunk_map.json | chunk_id -> {video_id, start, end}, used to remap chunked citations back to parent video IDs at output-formatting time. |
This bundle does not redistribute the MAGMaR-2026 ground-truth labels. Score your runs with the MIRAGE judge against the official ground truth obtained directly from the MAGMaR workshop.
Usage with CRAFT
# Download
hf download mbhosale/CRAFT-MAGMaR --repo-type dataset \
--local-dir /path/to/MAGMaR2026_test
# Run CRAFT against it (no chunking needed, ships pre-chunked)
git clone https://github.com/bhosalems/CRAFT.git && cd CRAFT
conda create -n craft python=3.13 -y && conda activate craft
pip install -r requirements.txt
SKIP_CHUNK=1 \
VIDEO_ROOT=/path/to/MAGMaR2026_test \
ASR_DIR=/path/to/MAGMaR2026_test/asr \
PARALLEL_QUERIES=8 PARALLEL_STEP15=8 PARALLEL_STEP5=8 \
bash run_query.sh outputs/craft_magmar_main
See the CRAFT repository for the full pipeline (Stage 1b VLM extraction, hybrid critic loop, UNLI calibration, higher-level inference, report assembly) and the MIRAGE evaluator setup.
Licensing
The base videos and queries come from the MAGMaR-2026 test release at akhilvssg/magmar-2026-test-asr-embeddings. Please honour that dataset's license for any downstream redistribution. CRAFT-produced artefacts (chunks, ASR cache, AKS outputs) are released under the same terms.
Citation
@article{bhosale2026craft,
title={CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answering},
author={Bhosale, Mahesh and Wasi, Abdul and Trivedi, Vishvesh and Yan, Pengyu and Gorugantu, Akhil and Doermann, David},
journal={arXiv preprint arXiv:2605.19075},
year={2026}
}
Similar repos
Linked in READMEs
Contributors
5 commits