A Comparative Study of Open-Source Large Language Models

Dataset Overview

Welcome to the dataset repository for our paper, "A Comparative Study of Open-Source Large Language Models, GPT-4 and Claude 2: Multiple-Choice Test Taking in Nephrology." The preprint of the paper can be accessed here.

Files

This repository contains two key files:

1. NEJM_All_Questions_And_Answers.csv: This file includes all the questions and corresponding answers used in the study.

2. Ground_Truth_Answers.csv: This file provides ground truth explanations associated with the questions in the main dataset.

Usage

To utilize this dataset for your research or experimentation:

1. Download: Obtain the dataset files from this repository.
2. Load: Import the dataset into your preferred data analysis or machine learning environment.
3. Explore: Investigate the questions, answers, and ground truth explanations for your specific use case.

Paper

Our paper is accepted to NEJM-AI. For now please read the pre-print at the link: https://arxiv.org/abs/2308.04709