10
stars
1
commits
2
linked in READMEs
Mar 21, 2024
updated
JA-VLM-Bench-In-the-Wild is Japanese version of LLaVA-Bench-In-the-Wild. We carefully collected a diverse set of 42 images with 50 questions in total. (For LLaVA-Bench-In-the-Wild, 24 images with 60 questions)
The images contain Japanese culture and objects in Japan. The Japanese questions and answers were generated with assistance from GPT-4V (gpt-4-vision-preview), OpenAI’s large-scale language-generation model and removed nonsense data by humans. Compared to JA-VG-VQA-500, it contains more challenging questions and requires richer responses.
To evaluate Japanese VLMs, please go to our Github repository.
from datasets import load_dataset
dataset = load_dataset("SakanaAI/JA-VLM-Bench-In-the-Wild", split="test")
The images in this dataset are sourced from Unsplash and are free to use under the Unsplash License. They cannot be sold without significant modification and cannot be used to replicate similar or competing services.
@misc{akiba2024evomodelmerge,
title = {Evolutionary Optimization of Model Merging Recipes},
author. = {Takuya Akiba and Makoto Shing and Yujin Tang and Qi Sun and David Ha},
year = {2024},
eprint = {2403.13187},
archivePrefix = {arXiv},
primaryClass = {cs.NE}
}
1 commits
10
stars
1
commits
2
linked in READMEs
Mar 21, 2024
updated
JA-VLM-Bench-In-the-Wild is Japanese version of LLaVA-Bench-In-the-Wild. We carefully collected a diverse set of 42 images with 50 questions in total. (For LLaVA-Bench-In-the-Wild, 24 images with 60 questions)
The images contain Japanese culture and objects in Japan. The Japanese questions and answers were generated with assistance from GPT-4V (gpt-4-vision-preview), OpenAI’s large-scale language-generation model and removed nonsense data by humans. Compared to JA-VG-VQA-500, it contains more challenging questions and requires richer responses.
To evaluate Japanese VLMs, please go to our Github repository.
from datasets import load_dataset
dataset = load_dataset("SakanaAI/JA-VLM-Bench-In-the-Wild", split="test")
The images in this dataset are sourced from Unsplash and are free to use under the Unsplash License. They cannot be sold without significant modification and cannot be used to replicate similar or competing services.
@misc{akiba2024evomodelmerge,
title = {Evolutionary Optimization of Model Merging Recipes},
author. = {Takuya Akiba and Makoto Shing and Yujin Tang and Qi Sun and David Ha},
year = {2024},
eprint = {2403.13187},
archivePrefix = {arXiv},
primaryClass = {cs.NE}
}
1 commits