README
Magma: A Foundation Model for Multimodal AI Agents
Jianwei Yang*1† Reuben Tan1† Qianhui Wu1† Ruijie Zheng2‡ Baolin Peng1‡ Yongyuan Liang2‡
Yu Gu1 Mu Cai3 Seonghyeon Ye4 Joel Jang5 Yuquan Deng5 Lars Liden1 Jianfeng Gao1▽
1 Microsoft Research; 2 University of Maryland; 3 University of Wisconsin-Madison
4 KAIST; 5 University of Washington
* Project lead † First authors ‡ Second authors ▽ Leadership
[arXiv Paper] [Project Page] [Hugging Face Paper] [Github Repo]
Data Card
Breakdown
Magma-820K contains 820k image instruction tuning data with the following breakdown:
| Dataset Name | Count |
|---|---|
| coco | 360,173 |
| LLaVA-Pretrain | 30,000 |
| sam | 9,000 |
| wikiart | 500 |
| share_textvqa | 500 |
| web-celebrity | 500 |
| web-landmark | 500 |
| vg | 86,417 |
| gqa | 72,140 |
| ocr_vqa | 80,000 |
| laion4v_images | 11,741 |
| ChartQA_Dataset | 28,299 |
| synthdog_split20k | 20,000 |
| DocVQA | 10,194 |
| ai2d | 2,482 |
| DVQA | 16,000 |
| Infographics-VQA | 23,946 |
| figureqa | 20,000 |
| tqa | 1,493 |
| scienceqa | 4,976 |
Image Datasets
Please refer to the respective dataset papers for more details on the images used in Magma-820K.
Contributors
6 commits
Similar repos
README
Magma: A Foundation Model for Multimodal AI Agents
Jianwei Yang*1† Reuben Tan1† Qianhui Wu1† Ruijie Zheng2‡ Baolin Peng1‡ Yongyuan Liang2‡
Yu Gu1 Mu Cai3 Seonghyeon Ye4 Joel Jang5 Yuquan Deng5 Lars Liden1 Jianfeng Gao1▽
1 Microsoft Research; 2 University of Maryland; 3 University of Wisconsin-Madison
4 KAIST; 5 University of Washington
* Project lead † First authors ‡ Second authors ▽ Leadership
[arXiv Paper] [Project Page] [Hugging Face Paper] [Github Repo]
Data Card
Breakdown
Magma-820K contains 820k image instruction tuning data with the following breakdown:
| Dataset Name | Count |
|---|---|
| coco | 360,173 |
| LLaVA-Pretrain | 30,000 |
| sam | 9,000 |
| wikiart | 500 |
| share_textvqa | 500 |
| web-celebrity | 500 |
| web-landmark | 500 |
| vg | 86,417 |
| gqa | 72,140 |
| ocr_vqa | 80,000 |
| laion4v_images | 11,741 |
| ChartQA_Dataset | 28,299 |
| synthdog_split20k | 20,000 |
| DocVQA | 10,194 |
| ai2d | 2,482 |
| DVQA | 16,000 |
| Infographics-VQA | 23,946 |
| figureqa | 20,000 |
| tqa | 1,493 |
| scienceqa | 4,976 |
Image Datasets
Please refer to the respective dataset papers for more details on the images used in Magma-820K.
Similar repos
Linked in READMEs
Contributors
6 commits