INSEC Vulnerability Dataset

This is the dataset used in the paper Black-Box Adversarial Attacks on LLM-Based Code Completion, published at ICML 2025, for vulnerability assessment of attacked models. It consists of code prefixes and suffixes as well as CodeQL queries to confirm or reject whether an infilled completion is vulnerable or not. You can find the accompanying code on GitHub.

Field description

• cwe: Number of the CWE
• lang: Language of the vulnerable code snippet (e.g., py or cpp)
• pre_tt: Text preceding the line of the vulnerability
• post_tt: Text preceding the vulnerable tokens in the line of the vulnerability
• suffix_pre: Text following the vulnerable tokens in the line of the vulnerability
• suffix_post: Remainder of the file after the line of the vulnerability
• check_ql: Link to the CodeQL query which can be used to verify or reject the vulnerability of the completed code snipped

In particular, the prefix for model infilling is pre_tt + post_tt, whereas the suffix is suffix_pre + suffix_post.