LOC-BENCH: A Benchmark for Code Localization

LOC-BENCH is a dataset specifically designed to evaluate code localization methods in software repositories. LOC-BENCH provides diverse issues, including bug reports, feature requests, security vulnerabilities, and performance optimizations.

Code: https://github.com/gersteinlab/LocAgent

📊 Details

Loc-Bench_V1 is our official dataset for comparison with our approach.

The table below shows the distribution of categories in the dataset.

Previous Versions

• Loc-Bench_V0.1: The dataset used in the early version of our paper. Some examples in this dataset do not involve function-level code modifications but instead focus on modifying classes. V1 filters out 100 examples without function-level code modifications, creating a cleaner subset of the dataset.
• Loc-Bench_V0.2: Filtering out examples that do not involve function-level code modifications and then supplementing the dataset to restore it to the original size of 660 examples.

🔧 How to Use

You can easily load LOC-BENCH using Hugging Face's datasets library:

from datasets import load_dataset

dataset = load_dataset("czlll/Loc-Bench_V1", split="test")

📄 Citation

If you use LOC-BENCH in your research, please cite our paper:

@article{chen2025locagent,
title={LocAgent: Graph-Guided LLM Agents for Code Localization},
author={Chen, Zhaoling and Tang,Xiangru and Deng,Gangda and Wu,Fang and Wu,Jialong and Jiang,Zhiwei and Prasanna,Viktor and Cohan,Arman and Wang,Xingyao},
journal={arXiv preprint arXiv:2503.09089},
year={2025}
}