A niche collection of AI4Code papers and other resources (dataset, tutorial, etc.). There are also other collections that cover a wider range of content about AI4Code, such as:
A list of of AI coding tools (assistants, completion, refactoring, etc.).
More General Coding Assistants
ChatGPT in your editor
LLM-powered natural language compilers
Academic
Conferences
Software Enginnering/Programming Languages
The emphasis is on combining program analysis and deep learning to solve novel software engineering/programming languages task. In most cases, strong empirical results are required. Typically, new datasets are usually curated.
Machine Learning/AI
The emphasis is on desigining novel neural network architectures to process code. Typically, new datasets are usually curated.
Natural Language Processing
The emphasis is on applying NLP techniques for code, and the evaluation is primarily on running the models on known benchmark datasets; unique tasks are rarely introduced.
Papers (This list is a bit outdated, need to update)
Evaluate CodeLLMs
- Large Language Models of Code Fail at Completing Code with Potential Bugs - Tuan Dinh, Jinman Zhao, Samson Tan, Renato Negrinho, Leonard Lausen, Sheng Zha, George Karypis.
- Large Language Models Meet NL2Code: A Survey - Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Yongji Wang, Jian-Guang Lou (EMNLP 2023)
Techniques for Code Generation
Repo-Level CodeLLMs
Benchmarking CodeLLMs
Pretrained CodeLLMs
- CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation - Yue Wang, Weishi Wang, Shafiq Joty, Steven C.H. Hoi (EMNLP 2021) (CodeT5).
- CodeBERT:A Pre-Trained Model for Programming and Natural Language - Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, Ming Zhou (EMNLP 2020 Findings) (CodeBERT).
- Learning and Evaluating Contextual Embedding of Source Code - Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, Kensen Shi. (ICML 2020) (CuBERT).
- GraphCodeBERT: Pre-training Code Representations with Data Flow - Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, Ming Zhou (ICLR 2021) (GraphCodeBERT).
- InferCode: Self-Supervised Learning of Code Representations by Predicting Subtrees -Nghi D. Q. BUI, Yijun YU, Lingxiao JIANG (ICSE 2021) (InferCode).
- Self-Supervised Learning for Code Retrieval and Summarization through Semantic-Preserving Program Transformations - Nghi D. Q. BUI, Yijun YU, Lingxiao JIANG (SIGIR 2021) (Corder).
- Unsupervised Translation of Programming Languages - Marie-Anne Lachaux, Baptiste Roziere, Lowik Chanussot, Guillaume Lample (NeurIPS 2020) (Transcoder).
- Contrastive Code Representation Learning -
Paras Jain, Ajay Jain, Tianjun Zhang, Pieter Abbeel, Joseph E. Gonzalez, Ion Stoica (arxiv 2020) (ContraCode).
- CoTexT: Multi-task Learning with Code-Text Transformer, Long Phan, Hieu Tran, Daniel Le, Hieu Nguyen, James Anibal, Alec Peltekian, Yanfang Ye (arXiv 2021) (CoTexT)
- How could Neural Networks understand Programs?, Dinglan Peng, Shuxin Zheng, Yatao Li, Guolin Ke, Di He, Tie-Yan Liu (ICML 2021) (OSCAR)
- Unified Pre-training for Program Understanding and Generation, Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, Kai-Wei Chang (NAACL 2021) (PLBART).
- Exploring Software Naturalness through Neural Language Models, Luca Buratti, Saurabh Pujar, Mihaela Bornea, Scott McCarley, Yunhui Zheng, Gaetano Rossiello, Alessandro Morari, Jim Laredo, Veronika Thost, Yufan Zhuang, Giacomo Domeniconi, (arXiv 2020) (C-BERT).
- PYMT5: multi-mode translation of natural language and PYTHON code with transformers, Colin B. Clement, Dawn Drain, Jonathan Timcheck, Alexey Svyatkovskiy, Neel Sundaresan (EMNLP 2020) (PYMT5).
- DOBF: A Deobfuscation Pre-Training Objective for Programming Languages, Baptiste Roziere, Marie-Anne Lachaux, Marc Szafraniec, Guillaume Lample, (arXiv 2021) (DOBF).
- Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks, , Antonio Mastropaolo, Simone Scalabrino, Nathan Cooper, David Nader Palacio, Denys Poshyvanyk, Rocco Oliveto, Gabriele Bavota (ICSE 2021).
- CodeTrans: Towards Cracking the Language of Silicone’s Code Through Self-Supervised Deep Learning and High Performance Computing, Ahmed Elnaggar, Wei Ding, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Silvia Severini, Florian Matthes, Burkhard Rost (arXiv 2021/04( (CodeTrans).
- Disentangled Code Representation Learning for Multiple Programming Languages,Jingfeng Zhang, Haiwen Hong, Yin Zhang, Yao Wan, Ye Liu, Yulei Sui (ACL-Fingings 2021) (CODEDISEN).
- SYNCOBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code Representation, Xin Wang, Yasheng Wang, Fei Mi, Pingyi Zhou, Yao Wan, Xiao Liu, Li Li, Hao Wu, Jin Liu, Xin Jiang, (arXiv 2021) (SYNCOBERT).
- TreeBERT: A Tree-Based Pre-Trained Model for Programming Language, Xue Jiang, Zhuoran Zheng, Chen Lyu, Liang Li, Lei Lyu, (UAI 2021) (TreeBERT).
- Empirical Study of Transformers for Source Code, Nadezhda Chirkova, Sergey Troshin (ESEC/FSE 2021).
Dataset and Benchmark
Talks and Tutorials
Articles
