TextME: Training Datasets

This dataset contains the training data for TextME, a text-only modality expansion framework.

Note: Pretrained projection checkpoints and offset vectors are available in the model repository.

Dataset Description

TextME enables zero-shot cross-modal transfer by leveraging the consistent modality gap property of pretrained contrastive encoders. This dataset provides caption datasets (text descriptions) for training projection networks — no paired cross-modal data is required.

Caption Datasets

* Remote sensing is supported by the framework but not reported in the paper.

Usage

from datasets import load_dataset

# Load a specific caption dataset
coco = load_dataset("SoyeonHH/textme-data", data_files="captions/coco.parquet")
audiocaps = load_dataset("SoyeonHH/textme-data", data_files="captions/audiocaps.parquet")
internvid = load_dataset("SoyeonHH/textme-data", data_files="captions/internvid.parquet")

# Load all caption datasets at once
all_captions = load_dataset("SoyeonHH/textme-data", data_dir="captions")

Related Resources

• Model checkpoints & offset vectors: SoyeonHH/TextME
• Code: GitHub
• Paper: arXiv:2602.03098

References

1. T.-Y. Lin et al., "Microsoft COCO: Common Objects in Context," ECCV, 2014. ↗
2. C. D. Kim et al., "AudioCaps: Generating Captions for Audios in the Wild," NAACL-HLT, 2019. ↗
3. M. Deitke et al., "Objaverse: A Universe of Annotated 3D Objects," CVPR, 2023. ↗
4. X. Wang et al., "ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks," CVPR, 2017. ↗
5. S. Kim et al., "PubChem 2025 Update," Nucleic Acids Research, 2025. ↗
6. F. Liu et al., "RemoteCLIP: A Vision Language Foundation Model for Remote Sensing," IEEE TGRS, 2024. ↗
7. Y. Wang et al., "InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation," ICLR, 2024. ↗

Citation

@article{hong2025textme,
  title={TextME: Bridging Unseen Modalities Through Text Descriptions},
  author={Hong, Soyeon and Kim, Jinchan and You, Jaegook and Choi, Seungtaek and Kwak, Suha and Cho, Hyunsouk},
  journal={arXiv preprint arXiv:2602.03098},
  year={2025}
}

License

This dataset is released under the MIT License.