Dataset Card for Dataset Name

CriticBench is a comprehensive benchmark designed to assess LLMs' abilities to generate, critique/discriminate and correct reasoning across a variety of tasks. CriticBench encompasses five reasoning domains: mathematical, commonsense, symbolic, coding, and algorithmic. It compiles 15 datasets and incorporates responses from three LLM families.

Dataset Details

Dataset Description

• Curated by: THU
• Funded by [optional]: [More Information Needed]
• Shared by [optional]: [More Information Needed]
• Language(s) (NLP): EN
• License: MIT

Dataset Sources [optional]

• Repository: https://github.com/CriticBench/CriticBench
• Paper [optional]: https://arxiv.org/pdf/2402.14809.pdf
• Demo [optional]: https://criticbench.github.io/

Uses

Direct Use

[More Information Needed]

Dataset Structure

[More Information Needed]

Dataset Creation

Curation Rationale

[More Information Needed]

Source Data

Data Collection and Processing

[More Information Needed]

Who are the source data producers?

[More Information Needed]

Annotations [optional]

Annotation process

[More Information Needed]

Who are the annotators?

[More Information Needed]

Personal and Sensitive Information

[More Information Needed]

Bias, Risks, and Limitations

[More Information Needed]

Recommendations

Users should be made aware of the risks, biases and limitations of the dataset. More information needed for further recommendations.

Citation [optional]

BibTeX:

@misc{lin2024criticbench,
    title={CriticBench: Benchmarking LLMs for Critique-Correct Reasoning}, 
    author={Zicheng Lin and Zhibin Gou and Tian Liang and Ruilin Luo and Haowei Liu and Yujiu Yang},
    year={2024},
    eprint={2402.14809},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

APA:

Lin, Z., Gou, Z., Liang, T., Luo, R., Liu, H., & Yang, Y. (2024). CriticBench: Benchmarking LLMs for Critique-Correct Reasoning.

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Dataset Card Authors [optional]

[More Information Needed]

Dataset Card Contact

[More Information Needed]