Dataset Card for statcodesearch

The StatCodeSearch dataset is a benchmark test set consisting of code comment pairs extracted from R programming language scripts authored mostly by researchers. The dataset is sourced from the Open Science Framework (OSF). It includes text and code samples from R projects that pertain to the fields of social science and psychology with a focus on the statistical analysis of research data. As part of the GenCodeSearchNet test suite, this dataset can be used to test programming language understanding on a low resource programming language.

Dataset Details

Dataset Sources [optional]

• Repository: https://github.com/drndr/gencodesearchnet
• Paper [optional]: https://arxiv.org/abs/2311.09707

Uses

• Semantic Code Search: using the comments as querries, return the matching code snippets from the set.

• Code Classification: using the labels, classify the code snippets in to four categories: Data Variable, Visualization, Statistical Modeling, Statistical Test

Dataset Structure

• Id: unique identifier for each item
• Comment: full string of a comment describing a code snippet
• Code: full string of a code snippet
• Label: class of a code snippet
• Source: the OSF repository of the extracted code-comment pair
• File: the name of the R file of the extracted code-comment pair from the OSF repository

Citation [optional]

BibTeX:

@inproceedings{diera2023gencodesearchnet,
  title={GenCodeSearchNet: A Benchmark Test Suite for Evaluating Generalization in Programming Language Understanding},
  author={Diera, Andor and Dahou, Abdelhalim and Galke, Lukas and Karl, Fabian and Sihler, Florian and Scherp, Ansgar},
  booktitle={Proceedings of the 1st GenBench Workshop on (Benchmarking) Generalisation in NLP},
  pages={12--24},
  year={2023}
}