Official repository for the IR-TD dataset from the ICCV 2025 paper:
"IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark"
IR-TD (InfraRed-Text Dataset) is the first large-scale, real-world infrared image-text benchmark, curated for advancing multi-modal vision-language research in the infrared domain.
It is released together with our ICCV 2025 paper:
IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark.
IR-TD is organized into three main subsets:
The IR-TD benchmark supports the following 9 representative tasks:
| Task | Description |
|---|---|
| Recognition | Multi-choice object/category recognition |
| Grounding | Object localization with bounding boxes |
| Location | Return coordinates of all instances of a category |
| Relationship | Spatial relationship reasoning (left/right/front/back/etc) |
| Re-ID | Person re-identification (cross-scene/camera) |
| Security | Identify categories absent from the image |
| Aerial Counting | Vehicle counting from UAV imagery |
| Pedestrian Counting | People counting in crowd scenes |
| Scene | Scene classification (urban/field/forest, etc.) |
Annotations are generated via a combination of LLM-based natural language description, rule-based QA pair construction, and standardized templates for each task.
IR-TD merges and re-annotates images from diverse open datasets, covering:
Unaligned visible-infrared pairs are processed with semantic cropping to ensure cross-modal alignment.
To enable researchers to quickly explore and experiment with the dataset, we have released an early version containing some of the most highly requested samples.
This release is intended for academic research use only. Feedback is welcome as we continue to improve and expand the dataset.
The text description of the data has been uploaded to ICCV2025-IRGPT/json_sft/json_sft.zip Due to the large scale of the data, we are unable to upload it. Since most of them are public datasets, we suggest that you download them by yourself. Due to copyright and other reasons, we can not upload some small-scale datasets now (Such as 27, 32-50). You can now block this part of the content when using the dataset.
Due to unknown reasons, Google Netdisk is temporarily unavailable. We have updated the link to Baidu Netdisk
If you find this dataset helpful in your research, please consider citing our paper:
@article{cao2025irgpt,
title={IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark},
author={Cao, Zhe and Zhang, Jin and Zhang, Ruiheng},
journal={arXiv preprint arXiv:2507.14449},
year={2025}
}
Official repository for the IR-TD dataset from the ICCV 2025 paper:
"IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark"
IR-TD (InfraRed-Text Dataset) is the first large-scale, real-world infrared image-text benchmark, curated for advancing multi-modal vision-language research in the infrared domain.
It is released together with our ICCV 2025 paper:
IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark.
IR-TD is organized into three main subsets:
The IR-TD benchmark supports the following 9 representative tasks:
| Task | Description |
|---|---|
| Recognition | Multi-choice object/category recognition |
| Grounding | Object localization with bounding boxes |
| Location | Return coordinates of all instances of a category |
| Relationship | Spatial relationship reasoning (left/right/front/back/etc) |
| Re-ID | Person re-identification (cross-scene/camera) |
| Security | Identify categories absent from the image |
| Aerial Counting | Vehicle counting from UAV imagery |
| Pedestrian Counting | People counting in crowd scenes |
| Scene | Scene classification (urban/field/forest, etc.) |
Annotations are generated via a combination of LLM-based natural language description, rule-based QA pair construction, and standardized templates for each task.
IR-TD merges and re-annotates images from diverse open datasets, covering:
Unaligned visible-infrared pairs are processed with semantic cropping to ensure cross-modal alignment.
To enable researchers to quickly explore and experiment with the dataset, we have released an early version containing some of the most highly requested samples.
This release is intended for academic research use only. Feedback is welcome as we continue to improve and expand the dataset.
The text description of the data has been uploaded to ICCV2025-IRGPT/json_sft/json_sft.zip Due to the large scale of the data, we are unable to upload it. Since most of them are public datasets, we suggest that you download them by yourself. Due to copyright and other reasons, we can not upload some small-scale datasets now (Such as 27, 32-50). You can now block this part of the content when using the dataset.
Due to unknown reasons, Google Netdisk is temporarily unavailable. We have updated the link to Baidu Netdisk
If you find this dataset helpful in your research, please consider citing our paper:
@article{cao2025irgpt,
title={IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark},
author={Cao, Zhe and Zhang, Jin and Zhang, Ruiheng},
journal={arXiv preprint arXiv:2507.14449},
year={2025}
}