README
RWQ battle records dataset
The dataset stores the battle records of 24 popular LLMs conduct Elo pairwise battles on RWQ questions and use GPT-4 as judger to determine the winner on each round of QA.
Columns
| Column Name | Data Type | Description |
|---|---|---|
| question | string | The question to ask LLM. |
| model | string | The id/name of LLM. |
| model_a | string | The id/name of model 1 of pairwise LLM to battle facing another on the same question. |
| model_b | string | The id/name of model 2 of pairwise LLM to battle facing another on the same question. |
| winner | string | The winner model valued as one of model_a, model_b, tie or tie(all bad) as outcome of one pairwise battle. |
| judger | string | The gpt name with version, such as gpt-4-turbo. |
| tstamp | string | The time battle happens, format as 2023-11-23 02:56:34.433226. |
| answer_a | string | The answer of model_a. |
| answer_b | string | The answer of model_b. |
| gpt_4_response | string | The reponse text of gpt-4 as judger to evaluate and score the better LLM. |
| gpt_4_score | string | The scores of model_a and model_b with json text, e.g., {'model_a': '0', 'model_b': '1'}. |
| is_valid | boolean | The row is valid or not. Set to false, when gpt-4 reject the eval because of policy. |
| elo_rating | float | The elo rating score of LLM. |
Citation
TODO
Contributors
Similar repos
README
RWQ battle records dataset
The dataset stores the battle records of 24 popular LLMs conduct Elo pairwise battles on RWQ questions and use GPT-4 as judger to determine the winner on each round of QA.
Columns
| Column Name | Data Type | Description |
|---|---|---|
| question | string | The question to ask LLM. |
| model | string | The id/name of LLM. |
| model_a | string | The id/name of model 1 of pairwise LLM to battle facing another on the same question. |
| model_b | string | The id/name of model 2 of pairwise LLM to battle facing another on the same question. |
| winner | string | The winner model valued as one of model_a, model_b, tie or tie(all bad) as outcome of one pairwise battle. |
| judger | string | The gpt name with version, such as gpt-4-turbo. |
| tstamp | string | The time battle happens, format as 2023-11-23 02:56:34.433226. |
| answer_a | string | The answer of model_a. |
| answer_b | string | The answer of model_b. |
| gpt_4_response | string | The reponse text of gpt-4 as judger to evaluate and score the better LLM. |
| gpt_4_score | string | The scores of model_a and model_b with json text, e.g., {'model_a': '0', 'model_b': '1'}. |
| is_valid | boolean | The row is valid or not. Set to false, when gpt-4 reject the eval because of policy. |
| elo_rating | float | The elo rating score of LLM. |
Citation
TODO