README
PandasPlotBench
PandasPlotBench is a benchmark to assess the capability of models in writing the code for visualizations given the description of the Pandas DataFrame.
🛠️ Task. Given the plotting task and the description of a Pandas DataFrame, write the code to build a plot.
The dataset is based on the MatPlotLib gallery.
The paper can be found in arXiv: https://arxiv.org/abs/2412.02764v1.
To score your model on this dataset, you can use the our GitHub repository.
📩 If you have any questions or requests concerning this dataset, please contact the author at timur.galimzyanov@jetbrains.com.
How-to
Loading the data
via load_dataset:
from datasets import load_dataset
dataset = load_dataset("JetBrains-Research/plot_bench", split="test")
Note that all of our data is considered to be in the test split.
Usage
You can find the benchmark code in our GitHub repository.
Scoring
We use the LLM as a Judge approach to score the results in two ways:
- Visual scoring. The Judge model is asked to compare two images on a scale of 0 to 100, focusing on the main idea of the images while ignoring styling details.
- Task-based scoring. The Judge model is asked to score the adherence of the resulting plot to the task description.
Datapoint Schema
Each example has the following fields:
| Field | Description |
|---|---|
id | Unique ID of the datapoint. |
code_plot | Ground truth code that plots the df DataFrame. Our basic data points use matplotlib. |
code_data | Code for loading the data. For the majority of the data points, it is import pandas as pd; df = pd.read_csv("data.csv") |
data_csv | CSV data to be plotted. Its content is saved to the dataset folder during benchmarking. |
task__plot_description | Main description of the plot (i.e., the task). |
task__plot_style | Description of the style of the plot. |
_task__plot_description_short | Synthetically shortened plot description (2-3 sentences). |
_task__plot_description_short_single | Synthetically shortened plot description (1 sentence). |
plots_gt | list of encoded (base64.b64encode(image_file.read()).decode("utf-8")) ground truth plots. Usually, this is a single plot. |
Similar repos
README
PandasPlotBench
PandasPlotBench is a benchmark to assess the capability of models in writing the code for visualizations given the description of the Pandas DataFrame.
🛠️ Task. Given the plotting task and the description of a Pandas DataFrame, write the code to build a plot.
The dataset is based on the MatPlotLib gallery.
The paper can be found in arXiv: https://arxiv.org/abs/2412.02764v1.
To score your model on this dataset, you can use the our GitHub repository.
📩 If you have any questions or requests concerning this dataset, please contact the author at timur.galimzyanov@jetbrains.com.
How-to
Loading the data
via load_dataset:
from datasets import load_dataset
dataset = load_dataset("JetBrains-Research/plot_bench", split="test")
Note that all of our data is considered to be in the test split.
Usage
You can find the benchmark code in our GitHub repository.
Scoring
We use the LLM as a Judge approach to score the results in two ways:
- Visual scoring. The Judge model is asked to compare two images on a scale of 0 to 100, focusing on the main idea of the images while ignoring styling details.
- Task-based scoring. The Judge model is asked to score the adherence of the resulting plot to the task description.
Datapoint Schema
Each example has the following fields:
| Field | Description |
|---|---|
id | Unique ID of the datapoint. |
code_plot | Ground truth code that plots the df DataFrame. Our basic data points use matplotlib. |
code_data | Code for loading the data. For the majority of the data points, it is import pandas as pd; df = pd.read_csv("data.csv") |
data_csv | CSV data to be plotted. Its content is saved to the dataset folder during benchmarking. |
task__plot_description | Main description of the plot (i.e., the task). |
task__plot_style | Description of the style of the plot. |
_task__plot_description_short | Synthetically shortened plot description (2-3 sentences). |
_task__plot_description_short_single | Synthetically shortened plot description (1 sentence). |
plots_gt | list of encoded (base64.b64encode(image_file.read()).decode("utf-8")) ground truth plots. Usually, this is a single plot. |