Shadowgraph
About
Sign in

HiromasaYamanishi/LLaVATour

3

stars

435

commits

Python

primary language

Oct 10, 2024

updated

README

LLaVATour

Overview

LLaVATour is a large-scale multimodal model specialized for Japanese tourism. It is trained on a dataset collected from Jalan.net, consisting of over 1.3 million entries covering various tasks such as landmark recognition, description generation, and review generation.

Dataset Distribution Model Architecture

Results

Landmark Recognition

ModelAcc ↑ (total)Acc ↑ (pop)Acc ↑ (mid)Acc ↑ (low)
ChatGPT-4V0.0260.07710.02140.0172
ChatGPT-4o0.1550.2750.1600.119
Gemini-Pro-1.50.0920.1740.0930.071
ResNet18-FT0.1430.2070.1520.119
LLaVA-1.50.01690.03780.01610.0123
LLaVA-Tour0.1660.2430.1920.122

Review Generation

ModelBLUE ↑R-1 ↑R-L ↑DIV ↑PROPN ↑TFIDF15 ↑
ChatGPT-4V0.6220.2500.1650.9550.2780.100
LLaVA-Tour1.4890.2490.1790.9190.4490.127
LLaVA-Tour + context1.5740.2580.1800.9240.4810.139

Case Examples

Landmark Recognition

Landmark Recognition Example

Review Generation

Review Generation Example

Contributors

(top 30 of 42)

haotian-liu

348 commits

ChunyuanLI

28 commits

yorickvP

5 commits

eggry

5 commits

Similar repos

haotian-liu/LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and…

25,017

shared contributors

HiromasaYamanishi/TourMix1M

2

same org

HiromasaYamanishi/TourMLLM

4

same org

LLaVA-VL/LLaVA-Plus-Codebase

LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

769

shared contributors

qnguyen3/hermes-llava

54

shared contributors

gligen/GLIGEN

Open-Set Grounded Text-to-Image Generation

2,223

shared contributors

tosiyuki/LLaVA-JP

LLaVA-JP is a Japanese VLM trained by LLaVA method

64

The-AI-Alliance/GEO-Bench-VLM

GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks

117

Languages

Python

59.3%

HTML

38.1%

Shell

2.1%

HiromasaYamanishi/LLaVATour

3

stars

435

commits

Python

primary language

Oct 10, 2024

updated

README

LLaVATour

Overview

LLaVATour is a large-scale multimodal model specialized for Japanese tourism. It is trained on a dataset collected from Jalan.net, consisting of over 1.3 million entries covering various tasks such as landmark recognition, description generation, and review generation.

Dataset Distribution Model Architecture

Results

Landmark Recognition

ModelAcc ↑ (total)Acc ↑ (pop)Acc ↑ (mid)Acc ↑ (low)
ChatGPT-4V0.0260.07710.02140.0172
ChatGPT-4o0.1550.2750.1600.119
Gemini-Pro-1.50.0920.1740.0930.071
ResNet18-FT0.1430.2070.1520.119
LLaVA-1.50.01690.03780.01610.0123
LLaVA-Tour0.1660.2430.1920.122

Review Generation

ModelBLUE ↑R-1 ↑R-L ↑DIV ↑PROPN ↑TFIDF15 ↑
ChatGPT-4V0.6220.2500.1650.9550.2780.100
LLaVA-Tour1.4890.2490.1790.9190.4490.127
LLaVA-Tour + context1.5740.2580.1800.9240.4810.139

Case Examples

Landmark Recognition

Landmark Recognition Example

Review Generation

Review Generation Example

Similar repos

haotian-liu/LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and…

25,017

shared contributors

HiromasaYamanishi/TourMix1M

2

same org

HiromasaYamanishi/TourMLLM

4

same org

LLaVA-VL/LLaVA-Plus-Codebase

LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

769

shared contributors

qnguyen3/hermes-llava

54

shared contributors

gligen/GLIGEN

Open-Set Grounded Text-to-Image Generation

2,223

shared contributors

tosiyuki/LLaVA-JP

LLaVA-JP is a Japanese VLM trained by LLaVA method

64

The-AI-Alliance/GEO-Bench-VLM

GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks

117

Contributors

(top 30 of 42)

haotian-liu

348 commits

ChunyuanLI

28 commits

yorickvP

5 commits

eggry

5 commits

Languages

Python

59.3%

HTML

38.1%

Shell

2.1%