CapRL Evaluation Files

This dataset contains the files used by the CapRL Prism evaluation scripts.

Files

• json_file/: 12 Prism evaluation JSON files.
• bench_image_folder.zip: images used by the JSON files. After unzipping, it creates bench_image_folder/.

Each JSON stores image paths relative to the dataset root, for example:

bench_image_folder/lmm_eval_chartqa/41699051005347.png

Usage

huggingface-cli download internlm/CapRL-Evaluation-Files --repo-type dataset --local-dir CapRL-Evaluation-Files
cd CapRL-Evaluation-Files
unzip bench_image_folder.zip

Use files under json_file/ as --data-path for Prism_Evaluation/Eval_CapRL.py, and pass the dataset root as --image-root.

Example:

python -m Eval_CapRL \
  --data-path /path/to/CapRL-Evaluation-Files/json_file/lmm_eval_chartqa.json \
  --image-root /path/to/CapRL-Evaluation-Files \
  --tag chartqa \
  ...

--image-root should point to the directory that contains bench_image_folder/.