Shadowgraph
About
Sign in

LongXinKou/embodied-eval

1

stars

175

commits

Python

primary language

Jan 5, 2026

updated

README

embodied-eval

Why embodied-eval

🚀 TODO

  • Open-EQA
    • EM-EQA
    • A-EQA
  • GOAT-Bench

🔨 Setup

  1. Clone this repository & Install packages
git clone https://github.com/LongXinKou/embodied-eval.git
cd embodied-eval
conda create -n embodied-eval python==3.10
conda activate embodied-eval
pip install -r requirements.txt

🎁 Benchmark

CategoryItemsPaper
VQAEgoToMEgoToM: Benchmarking Theory of Mind Reasoning from Egocentric Videos
VQAVSI-BenchThinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
VQA (planning)EgoPlan-Bench2EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios
VQA (planning)EgoPlan-BenchEgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning
VQAVidEgoThinkVidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AI
VQAEgoThinkEgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models
A-EQAEXPRESS-BenchBeyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answering
A-EQA + VQAOpenEQAOpenEQA: Embodied Question Answering in the Era of Foundation Models
A-EQAExplore-EQAExplore until Confident: Efficient Exploration for Embodied Question Answering
EQA (plannig + action)Embodied-BenchEMBODIEDBENCH: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents

Contributors

LongXinKou

175 commits

Similar repos

Iron-Wph/behaviour_subtask

0

EmbodiedArena/embodied-arena

3

AdaCheng/EgoThink

[CVPR'24 Highlight] The official code and data for paper "EgoThink: Evaluating First-Person…

67

facebookresearch/open-eqa

OpenEQA Embodied Question Answering in the Era of Foundation Models

364

Jaewoo97/trajgaze

0

vision-x-nyu/thinking-in-space

Official repo and evaluation implementation of VSI-Bench

740

yulin-luo/RoboBench

This is the official evaluation code for Robobench

25

XiaomiMiMo/MiMo-Embodied

MiMo-Embodied

405

Languages

Python

97.1%

Shell

2.9%

LongXinKou/embodied-eval

1

stars

175

commits

Python

primary language

Jan 5, 2026

updated

README

embodied-eval

Why embodied-eval

🚀 TODO

  • Open-EQA
    • EM-EQA
    • A-EQA
  • GOAT-Bench

🔨 Setup

  1. Clone this repository & Install packages
git clone https://github.com/LongXinKou/embodied-eval.git
cd embodied-eval
conda create -n embodied-eval python==3.10
conda activate embodied-eval
pip install -r requirements.txt

🎁 Benchmark

CategoryItemsPaper
VQAEgoToMEgoToM: Benchmarking Theory of Mind Reasoning from Egocentric Videos
VQAVSI-BenchThinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
VQA (planning)EgoPlan-Bench2EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios
VQA (planning)EgoPlan-BenchEgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning
VQAVidEgoThinkVidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AI
VQAEgoThinkEgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models
A-EQAEXPRESS-BenchBeyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answering
A-EQA + VQAOpenEQAOpenEQA: Embodied Question Answering in the Era of Foundation Models
A-EQAExplore-EQAExplore until Confident: Efficient Exploration for Embodied Question Answering
EQA (plannig + action)Embodied-BenchEMBODIEDBENCH: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents

Similar repos

Iron-Wph/behaviour_subtask

0

EmbodiedArena/embodied-arena

3

AdaCheng/EgoThink

[CVPR'24 Highlight] The official code and data for paper "EgoThink: Evaluating First-Person…

67

facebookresearch/open-eqa

OpenEQA Embodied Question Answering in the Era of Foundation Models

364

Jaewoo97/trajgaze

0

vision-x-nyu/thinking-in-space

Official repo and evaluation implementation of VSI-Bench

740

yulin-luo/RoboBench

This is the official evaluation code for Robobench

25

XiaomiMiMo/MiMo-Embodied

MiMo-Embodied

405

Contributors

LongXinKou

175 commits

Languages

Python

97.1%

Shell

2.9%