README
VideoHallucer
Paper: https://huggingface.co/papers/2406.16338
Dataset Description
- Repository: VideoHallucer
- Paper: 2406.16338
- Point of Contact: mailto:Yuxuan Wang
This work introduces VideoHallucer, the first comprehensive benchmark for hallucination detection in large video-language models (LVLMs). VideoHallucer categorizes hallucinations into two main types: intrinsic and extrinsic, offering further subcategories for detailed analysis, including object-relation, temporal, semantic detail, extrinsic factual, and extrinsic non-factual hallucinations. We adopt an adversarial binary VideoQA method for comprehensive evaluation, where pairs of basic and hallucinated questions are crafted strategically.
Data Statistics
| Object-Relation Hallucination | Temporal Hallucination | Semantic Detail Hallucination | External Factual Hallucination | External Nonfactual Hallucination | |
|---|---|---|---|---|---|
| Questions | 400 | 400 | 400 | 400 | 400 |
| Videos | 183 | 165 | 400 | 200 | 200 |
Evaluation
We provide VideoHallucerKit for evaluation
Leaderboard
See our page
Similar repos
README
VideoHallucer
Paper: https://huggingface.co/papers/2406.16338
Dataset Description
- Repository: VideoHallucer
- Paper: 2406.16338
- Point of Contact: mailto:Yuxuan Wang
This work introduces VideoHallucer, the first comprehensive benchmark for hallucination detection in large video-language models (LVLMs). VideoHallucer categorizes hallucinations into two main types: intrinsic and extrinsic, offering further subcategories for detailed analysis, including object-relation, temporal, semantic detail, extrinsic factual, and extrinsic non-factual hallucinations. We adopt an adversarial binary VideoQA method for comprehensive evaluation, where pairs of basic and hallucinated questions are crafted strategically.
Data Statistics
| Object-Relation Hallucination | Temporal Hallucination | Semantic Detail Hallucination | External Factual Hallucination | External Nonfactual Hallucination | |
|---|---|---|---|---|---|
| Questions | 400 | 400 | 400 | 400 | 400 |
| Videos | 183 | 165 | 400 | 200 | 200 |
Evaluation
We provide VideoHallucerKit for evaluation
Leaderboard
See our page