SEU-COIN/LLMPapers

Papers & Works for large languange models (OpenAI GPT-4, Meta Llama, etc.).

TeX

313

249 commits

updated Nov 15, 2025

See the code

README

Resources on ChatGPT and Large Language Models

Collection of papers and related works for Large Language Models (ChatGPT, GPT-3, Codex etc.).

Contributors

This repository is contributed by the following contributors.

Organizers: Guilin Qi (漆桂林), Xiaofang Qi (戚晓芳)
Paper Collectors: Zafar Ali, Sheng Bi (毕胜), Yongrui Chen (陈永锐), Zizhuo Chen (陈孜卓), Xinbang Dai (戴鑫邦), Huan Gao (高桓), Nan Hu (胡楠), Shilong Hu (胡世龙), Jingqi Kang (康婧淇), Jiaqi Li (李嘉琦), Dehai Min (闵德海), Guilin Qi (漆桂林), Yiming Tan (谭亦鸣), Tongtong Wu (吴桐桐), Songlin Zhai (翟松林), Shenyu Zhang (张沈昱), Yuxin Zhang (张裕欣)
Maintainers: Runzhe Wang (王润哲), Shenyu Zhang (张沈昱)

The automation script of this repo is powered by Auto-Bibfile. If you'd like to commit to this repo, please modify bibtex.bib or related_works.json and re-generate README.md using python scripts/run.py.

Papers

Outline

Hyperlinks

chatgpt

codex

gpt-3

large-language-models

Contributors

sid0527

124 commits

ZSY-SZ

124 commits

alphadl

1 commits

Languages

TeX

94.7%

Python

5.3%

SEU-COIN/LLMPapers

Papers & Works for large languange models (OpenAI GPT-4, Meta Llama, etc.).

TeX

313

249 commits

updated Nov 15, 2025

See the code

README

Resources on ChatGPT and Large Language Models

Collection of papers and related works for Large Language Models (ChatGPT, GPT-3, Codex etc.).

Contributors

This repository is contributed by the following contributors.

Organizers: Guilin Qi (漆桂林), Xiaofang Qi (戚晓芳)
Paper Collectors: Zafar Ali, Sheng Bi (毕胜), Yongrui Chen (陈永锐), Zizhuo Chen (陈孜卓), Xinbang Dai (戴鑫邦), Huan Gao (高桓), Nan Hu (胡楠), Shilong Hu (胡世龙), Jingqi Kang (康婧淇), Jiaqi Li (李嘉琦), Dehai Min (闵德海), Guilin Qi (漆桂林), Yiming Tan (谭亦鸣), Tongtong Wu (吴桐桐), Songlin Zhai (翟松林), Shenyu Zhang (张沈昱), Yuxin Zhang (张裕欣)
Maintainers: Runzhe Wang (王润哲), Shenyu Zhang (张沈昱)

Papers

Outline

Hyperlinks

Evaluation

<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs,
by Xiaoze Liu, Feijie Wu, Tianyang Xu, Zhuo Chen, Yichi Zhang, Xiaoqian Wang and Jing Gao
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity,
by Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji et al.
本文提出了一个使用公开数据集定量评估交互式LLM（如ChatGPT）的框架。我们使用涵盖8个不同的常见NLP应用任务的21个数据集对ChatGPT进行了广泛的技术评估。我们基于这些数据集和一个新设计的多模态数据集评估了ChatGPT的多任务、多语言和多模态方面。
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is ChatGPT a General-Purpose Natural Language Processing Task Solver?,
by Qin, Chengwei, Zhang, Aston, Zhang, Zhuosheng, Chen, Jiaao, Yasunaga, Michihiro and Yang, Diyi
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> ChatGPT versus Traditional Question Answering for Knowledge Graphs: Current Status and Future Directions Towards Knowledge Graph Chatbots,
by Reham Omar, Omij Mangukiya, Panos Kalnis and Essam Mansour
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Mathematical Capabilities of ChatGPT,
by Simon Frieder, Luca Pinchetti, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Christian Petersen, Alexis Chevalier and Julius Berner
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization,
by Xianjun Yang, Yan Li, Xinlu Zhang, Haifeng Chen and Wei Cheng
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective,
by Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> ChatGPT is not all you need. A State of the Art Review of large Generative AI models,
by Roberto Gozalo-Brizuela and Eduardo C. Garrido-Merch'an
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT,
by Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du and Dacheng Tao
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Evaluation of ChatGPT as a Question Answering System for Answering Complex Questions,
by Yiming Tan, Dehai Min, Yu Li, Wenbo Li, Nan Hu, Yongrui Chen and Guilin Qi
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models,
by Ning Bian, Xianpei Han, Le Sun, Hongyu Lin, Yaojie Lu and Ben He
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Through the Lens of Core Competency: Survey on Evaluation of Large Language Models,
by Ziyu Zhuang, Qiguang Chen, Longxuan Ma, Mingda Li, Yi Han, Yushan Qian, Haopeng Bai, Zixian Feng et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance,
by Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao Peng and Tushar Khot
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets,
by Md. Tahmid Rahman Laskar, M. Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty and Jimmy X. Huang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> GPTEval: A Survey on Assessments of ChatGPT and GPT-4,
by Rui Mao, Guanyi Chen, Xulang Zhang, Frank Guerin and Erik Cambria
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Holistic Evaluation of Language Models,
by Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Evaluating the Text-to-SQL Capabilities of Large Language Models,
by Nitarshan Rajkumar, Raymond Li and Dzmitry Bahdanau
<img src=https://img.shields.io/badge/COLING-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Are Visual-Linguistic Models Commonsense Knowledge Bases?,
by Hsiu-Yu Yang and Carina Silberer
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is GPT-3 a Psychopath? Evaluating Large Language Models from a Psychological Perspective,
by Xingxuan Li, Yutong Li, Linlin Liu, Lidong Bing and Shafiq R. Joty
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models,
by Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li and Kai-Wei Chang
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> RobustLR: A Diagnostic Benchmark for Evaluating Logical Robustness of Deductive Reasoners,
by Soumya Sanyal, Zeyi Liao and Xiang Ren
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Systematic Evaluation of Large Language Models of Code,
by Frank F. Xu, Uri Alon, Graham Neubig and Vincent J. Hellendoorn
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts,
by Arshiya Aggarwal, Jiao Sun and Nanyun Peng
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Evaluating Large Language Models Trained on Code,
by Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond'e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda et al.
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> GLGE: A New General Language Generation Evaluation Benchmark,
by Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu et al.
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Evaluating Pre-Trained Models for User Feedback Analysis in Software Engineering: A Study on Classification of App-Reviews,
by Mohammad Abdul Hadi and Fatemeh H. Fard
<img src=https://img.shields.io/badge/ACL_Findings-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Do Language Models Perform Generalizable Commonsense Inference?, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Peifeng Wang, Filip Ilievski, Muhao Chen and Xiang Ren
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms,
by Pei Zhou, Rahul Khanna, Seyeon Lee, Bill Yuchen Lin, Daniel Ho, Jay Pujara and Xiang Ren
<img src=https://img.shields.io/badge/CoRR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Evaluation of Text Generation: A Survey,
by Asli Celikyilmaz, Elizabeth Clark and Jianfeng Gao
<img src=https://img.shields.io/badge/CoRR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Neural Language Generation: Formulation, Methods, and Evaluation,
by Cristina Garbacea and Qiaozhu Mei
<img src=https://img.shields.io/badge/ICLR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> BERTScore: Evaluating Text Generation with BERT,
by Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger and Yoav Artzi

Survey

<img src=https://img.shields.io/badge/CoRR-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models,
by Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong et al.
<img src=https://img.shields.io/badge/CoRR-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> From System 1 to System 2: A Survey of Reasoning Large Language Models,
by Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng et al.
<img src=https://img.shields.io/badge/-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Context Engineering 2.0: The Context of Context Engineering,
by Qishuo Hua, Lyumanshan Ye, Dayuan Fu, Yang Xiao, Xiaojie Cai, Yunze Wu, Jifan Lin, Junfei Wang et al.
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Datasets for Large Language Models: A Comprehensive Survey,
by Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding and Lianwen Jin
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond,
by Qiushi Sun, Zhirui Chen, Fangzhi Xu, Kanzhi Cheng, Chang Ma, Zhangyue Yin, Jianing Wang, Chengcheng Han et al.
<img src=https://img.shields.io/badge/EMNLP-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models for Data Annotation and Synthesis: A Survey,
by Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey for In-context Learning,
by Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu et al.
This paper surveys and summarizes the progress and challenges of ICL, including ICL's formal definition, correlation to related studies, advanced techniques (training strategies, related analysis) and potential directions.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT,
by Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Complex QA and language models hybrid architectures, Survey,
by Xavier Daull, Patrice Bellot, Emmanuel Bruno, Vincent Martin and Elisabeth Murisasco
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Augmented Language Models: a Survey,
by Gr'egoire Mialon, Roberto Dess`\i, Maria Lomeli, Christoforos Nalmpantis, Ramakanth Pasunuru, Roberta Raileanu, Baptiste Rozi`ere, Timo Schick et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Life Cycle of Knowledge in Big Language Models: A Survey,
by Boxi Cao, Hongyu Lin, Xianpei Han and Le Sun
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey of Large Language Models,
by Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang et al.
<img src=https://img.shields.io/badge/ACM_Comput._Surv.-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Survey of Hallucination in Natural Language Generation,
by Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Aligning Large Language Models with Human: A Survey,
by Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey on Large Language Model based Autonomous Agents,
by Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang et al.
<img src=https://img.shields.io/badge/Internet_of_Things_and_Cyber--Physical_Systems-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope,
by Ray, Partha Pratim
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models for Software Engineering: A Systematic Literature Review,
by Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Unifying Large Language Models and Knowledge Graphs: A Roadmap,
by Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang and Xindong Wu
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models for Information Retrieval: A Survey,
by Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Zhicheng Dou and Ji-Rong Wen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey on Evaluation of Large Language Models,
by Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> AIGC for Various Data Modalities: A Survey,
by Lin Geng Foo, Hossein Rahmani and Jun Liu
<img src=https://img.shields.io/badge/arXiv_preprint_arXiv:2305.18703-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Domain specialization as the key to make large language models disruptive: A comprehensive survey,
by Ling, Chen, Zhao, Xujiang, Lu, Jiaying, Deng, Chengyuan, Zheng, Can, Wang, Junxiang, Chowdhury, Tanmoy, Li, Yun et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT,
by Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S. Yu and Lichao Sun
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Instruction Tuning for Large Language Models: A Survey,
by Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity,
by Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Jiayang Cheng, Yunzhi Yao, Wenyang Gao et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Deep Model Fusion: A Survey,
by Weishi Li, Yong Peng, Miao Zhang, Liang Ding, Han Hu and Li Shen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future,
by Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Explainability for Large Language Models: A Survey,
by Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin et al.
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models for Generative Information Extraction: A Survey,
by Derong Xu, Wei Chen, Wenjun Peng, Chao Zhang, Tong Xu, Xiangyu Zhao, Xian Wu, Yefeng Zheng et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey of Graph Meets Large Language Model: Progress and Future Directions,
by Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng and Jeffrey Xu Yu
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions,
by Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> When Neural Model Meets NL2Code: A Survey,
by Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Yongji Wang and Jian-Guang Lou
<img src=https://img.shields.io/badge/TKDE-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey on Knowledge-Enhanced Pre-trained Language Models,
by Chaoqi Zhen, Yanlei Shang, Xiangyu Liu, Yifei Li, Yong Chen and Dell Zhang
<img src=https://img.shields.io/badge/T--PAMI-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Continual Learning Survey: Defying Forgetting in Classification Tasks,
by Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory G. Slabaugh and Tinne Tuytelaars
<img src=https://img.shields.io/badge/JKSUCIS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> The survey: Text generation models in deep learning,
by Touseef Iqbal and Shaima Qureshi
<img src=https://img.shields.io/badge/KIS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> From distributed machine learning to federated learning: a survey,
by Ji Liu, Jizhou Huang, Yang Zhou, Xuhong Li, Shilei Ji, Haoyi Xiong and Dejing Dou
<img src=https://img.shields.io/badge/IJCAI-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code,
by Changan Niu, Chuanyi Li, Bin Luo and Vincent Ng
<img src=https://img.shields.io/badge/TKDE-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey on Knowledge Graph-Based Recommender Systems,
by Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong and Qing He
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems,
by Sagi Shaier, Lawrence Hunter and Katharina Kann
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Reasoning in Large Language Models: A Survey,
by Jie Huang and Kevin Chen-Chuan Chang
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Reasoning with Language Model Prompting: A Survey,
by Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey on Retrieval-Augmented Text Generation,
by Huayang Li, Yixuan Su, Deng Cai, Yan Wang and Lemao Liu
<img src=https://img.shields.io/badge/AAAI-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey,
by Prajjwal Bhargava and Vincent Ng
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Review on Language Models as Knowledge Bases,
by Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona T. Diab and Marjan Ghazvininejad
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Advances and Challenges in Conversational Recommender Systems: A Survey,
by Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke and Tat-Seng Chua
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey,
by Jinjie Ni, Tom Young, Vlad Pandelea, Fuzhao Xue, Vinay Adiga and Erik Cambria
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Relational World Knowledge Representation in Contextual Language Models: A Review,
by Tara Safavi and Danai Koutra
<img src=https://img.shields.io/badge/CoRR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Evaluation of Text Generation: A Survey,
by Asli Celikyilmaz, Elizabeth Clark and Jianfeng Gao

In-Context Learning

<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> In-context Learning with Retrieved Demonstrations for Language Models: A Survey,
by Man Luo, Xin Xu, Yue Liu, Panupong Pasupat and Mehran Kazemi
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> 'One size doesn't fit all': Learning how many Examples to use for In-Context Learning for Improved Text Classification,
by Manish Chandra, Debasis Ganguly, Yiwen Li and Iadh Ounis
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> "In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval",
by Andrew Parry, Debasis Ganguly and Manish Chandra
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> In-Context Editing: Learning Knowledge from Self-Induced Distributions,
by Siyuan Qi, Bangcheng Yang, Kailin Jiang, Xiaobo Wang, Jiaqi Li, Yifan Zhong, Yaodong Yang and Zilong Zheng
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey for In-context Learning,
by Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu et al.
This paper surveys and summarizes the progress and challenges of ICL, including ICL's formal definition, correlation to related studies, advanced techniques (training strategies, related analysis) and potential directions.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Explanation Selection Using Unlabeled Data for In-Context Learning,
by Xi Ye and Greg Durrett
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> In-Context Learning with Many Demonstration Examples,
by Mukai Li, Shansan Gong, Jiangtao Feng, Yiheng Xu, Jun Zhang, Zhiyong Wu and Lingpeng Kong
This paper proposes a LM named EvaLM to scale up the sequence length (trained with 8k tokens per batch line). Experiments based on EvaLM prove that in-context learning can achieve higher performance with more demonstrations under many-shot instruction tuning (8k) and further extending the length of instructions (16k) can further improve the upper bound of scaling in-context learning.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models Are Implicitly Topic Models: Explaining and Finding Good Demonstrations for In-Context Learning,
by Xinyi Wang, Wanrong Zhu and William Yang Wang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Finding Supporting Examples for In-Context Learning,
by Xiaonan Li and Xipeng Qiu
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Learnability of In-Context Learning,
by Noam Wies, Yoav Levine and Amnon Shashua
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> In-Context Instruction Learning,
by Seonghyeon Ye, Hyeonbin Hwang, Sohee Yang, Hyeongu Yun, Yireun Kim and Minjoon Seo
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> How Does In-Context Learning Help Prompt Tuning?,
by Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu and Mohit Iyyer
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fairness-guided Few-shot Prompting for Large Language Models,
by Huan Ma, Changqing Zhang, Yatao Bian, Lemao Liu, Zhirui Zhang, Peilin Zhao, Shu Zhang, Huazhu Fu et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Dr.ICL: Demonstration-Retrieved In-context Learning,
by Man Luo, Xin Xu, Zhuyun Dai, Panupong Pasupat, Seyed Mehran Kazemi, Chitta Baral, Vaiva Imbrasaite and Vincent Y. Zhao
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Unified Demonstration Retriever for In-Context Learning,
by Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang et al.
<img src=https://img.shields.io/badge/ESWC-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Exploring In-Context Learning Capabilities of Foundation Models for Generating Knowledge Graphs from Text,
by Hanieh Khorashadizadeh, Nandana Mihindukulasooriya, Sanju Tiwari, Jinghua Groppe and Sven Groppe
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Exploring the In-context Learning Ability of Large Language Model for Biomedical Concept Linking,
by Qinyong Wang, Zhenxiang Gao and Rong Xu
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> In-Context Demonstration Selection with Cross Entropy Difference,
by Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Yichong Xu and Chenguang Zhu
<img src=https://img.shields.io/badge/the_61st_Annual_Meeting_of_the_Association_for_Computational Linguistics_(Volume_2:Short_Papers),{ACL}_2023,_Toronto,_Canada, July_9--14,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models,
by Masoud Monajatipoor, Liunian Harold Li, Mozhdeh Rouhsedaghat, Lin Yang and Kai-Wei Chang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> SINC: Self-Supervised In-Context Learning for Vision-Language Tasks,
by Yi-Syuan Chen, Yun-Zhu Song, Cheng Yu Yeo, Bei Liu, Jianlong Fu and Hong-Han Shuai
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> How Many Demonstrations Do You Need for In-context Learning?,
by Jiuhai Chen, Lichang Chen, Chen Zhu and Tianyi Zhou
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Explaining Emergent In-Context Learning as Kernel Regression,
by Chi Han, Ziqi Wang, Han Zhao and Heng Ji
<img src=https://img.shields.io/badge/NeurIPS-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Meta-in-context learning in large language models,
by Julian Coda-Forno, Marcel Binz, Zeynep Akata, Matt M. Botvinick, Jane X. Wang and Eric Schulz
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks,
by Hao Peng, Xiaozhi Wang, Jianhui Chen, Weikai Li, Yunjia Qi, Zimu Wang, Zhili Wu, Kaisheng Zeng et al.
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Meta-learning via Language Model In-context Tuning, <img src=https://img.shields.io/badge/BERT-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/DeBERTa-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--2-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis and He He
<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> MetaICL: Learning to Learn In Context, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--2-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi
MetaICL proposes a supervised meta-training framework to enable LMs to more effectively learn a new task in context. In MetaICL, each meta-training example includes several training examples from one task that will be presented together as a single sequence to the LM, and the prediction of the final example is used to calculate the loss.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Selective Annotation Makes Language Models Better Few-Shot Learners, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/SBERT-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--J-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--Neo-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--3-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/Codex-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/OPT-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf et al.
This paper proposes a graph-based selective annotation method named vote-k to
(1) select a pool of examples to annotate from unlabeled data,
(2) retrieve prompts (contexts) from the annotated data pool for in-context learning.
Specifically, the selection method first selects a small set of unlabeled examples iteratively and then labels them to serve as contexts for LLMs to predict the labels of the rest unlabeled data. The method selects the predictions with highest confidence (log probability of generation output) to fill up the selective annotation pool.
<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improving In-Context Few-Shot Learning via Self-Supervised Training, <img src=https://img.shields.io/badge/MoE-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov and Zornitsa Kozareva
This paper proposes to use self-supervision (MLM, NSP, CL, etc.) between pre-training and downstream usage to teach the LM to perform in-context learning. Analysis reveals that:
(1) benefits of self-supervised depends on the amount of training data,
(2) semantic similarity between training and evaluation tasks matters,
(3) adding training objectives without diversity does not help,
(4) model performance improves when choosing similar templates for both self-supervised and downstream tasks,
(5) self-supervised tasks and human-annotated datasets are complementary,
(6) self-supervised-trained models are better at following task instructions.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Instruction Induction: From Few Examples to Natural Language Task Descriptions,
by Or Honovich, Uri Shaham, Samuel R. Bowman and Omer Levy
(1) 探索了利用LLM在几个样本的情况下归纳出任务指令的能力；
(2) 测量两个指标：1. 模型归纳指令与人类归纳的指令对比，2. 利用模型归纳的指令作为prompt进行预测的执行准确率；
(3) 相比于GPT-3，InstructGPT效果更好，理所当然。
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity, <img src=https://img.shields.io/badge/GPT--2-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--3-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel and Pontus Stenetorp
(1) This work demonstrates that few-shot prompts suffer from order sensitivity, in that for the same prompt the order in which samples are provided can make a difference to model performance.
(2) This work introduces a probing method which constructs an artificial development set by language models themselves to alleviate the order sensitivity problem.
<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning To Retrieve Prompts for In-Context Learning, <img src=https://img.shields.io/badge/GPT--3-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--Neo-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/Codex-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--J-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/SBERT-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/BERT-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Ohad Rubin, Jonathan Herzig and Jonathan Berant
This paper proposes a method to retrieve good contexts for in-context learning. Specifically, the method
(1) uses an unsupervised retriever (BM25/SBERT) to obtain a set of context candidates,
(2) passes the candidates to a scoring model (GPT-Neo/GPT-J/GPT-3/Codex) and select the top/bottom k as positive/negative examples,
(3) uses the examples to train a dense retriever (BERT-based).
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Active Example Selection for In-Context Learning, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--2-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--3-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Yiming Zhang, Shi Feng and Chenhao Tan
(1) This paper revisits the effect of example selection (re-ordering & calibration) for ICL, observing that a large variance across set of demonstration examples still exists.
(2) This paper applies reinforcement learning (Q-Learning) to optimize example selection by formulating this task as sequential decision-making problem, which is appropriate for example selection from unlabeled datasets.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Self-Generated In-Context Learning: Leveraging Auto-regressive Language Models as a Demonstration Generator,
by Hyuhng Joon Kim, Hyunsoo Cho, Junyeob Kim, Taeuk Kim, Kang Min Yoo and Sang-goo Lee
<img src=https://img.shields.io/badge/ISoLA-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Measuring Convergence Inertia: Online Learning in Self-adaptive Systems with Context Shifts,
by Elvin Alberts and Ilias Gerostathopoulos
<img src=https://img.shields.io/badge/ICLR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> An Explanation of In-context Learning as Implicit Bayesian Inference,
by Sang Michael Xie, Aditi Raghunathan, Percy Liang and Tengyu Ma
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?,
by Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi and Luke Zettlemoyer
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Impact of Symbolic Representations on In-context Learning for Few-shot Reasoning,
by Hanlin Zhang, Yi-Fan Zhang, Li Erran Li and Eric P. Xing
<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> What Makes Good In-Context Examples for GPT-3?, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/RoBERTa-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/T5-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/SBERT-yellow alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/GPT--3-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin and Weizhu Chen
(1) 探索了在in-context learning中什么样的demonstration example可以对GPT-3的效果取得帮助；
(2) 利用roberta对样本进行编码，并计算demonstration与test example的向量距离（欧氏距离），最终发现与test example越相近的demonstration越能取得较好的效果。
<img src=https://img.shields.io/badge/EMNLP_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again,
by Bernal Jimenez Gutierrez, Nikolas McNeal, Clayton Washington, You Chen, Lang Li, Huan Sun and Yu Su
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers,
by Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Zhifang Sui and Furu Wei
(1) 与The Dual Form of Neural Networks Revisited结合一起看，可以进一步理解in-context learning，通过与NN线性层对偶形式的类比，可以将ICL流程描述为：1. 基于Transformer的预训练语言模型作为元优化器；2. 通过正向计算，根据示范例子产生元梯度；3. 通过关注，将元梯度应用于原始语言模型，建立一个ICL模型；
(2)与Fine-tune类似，ICL也是在zero-shot learning参数的基础上，提供了一个更新量。
<img src=https://img.shields.io/badge/ICML-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention,
by Kazuki Irie, R'obert Csord'as and J"urgen Schmidhuber
(1) 很有意思的一篇，回顾神经网络（NN）线性层Y=WX（省略偏置b）的原始形式与对偶形式，两种形式完全等价；
(2) 从对偶形式中可以发现，通过反向传播训练的NN线性层的输出主要是该层在训练期间的训练误差信号et的线性组合，其中权重是通过比较测试查询x和每个训练输入计算出来的；进一步可以得出，如果测试时输入的x和训练时的输入是正交的，那么梯度下降所得到的参数更新对于该样本x完全没有影响。
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Self-adaptive In-context Learning,
by Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye and Lingpeng Kong
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Careful Data Curation Stabilizes In-context Learning,
by Ting-Yun Chang and Robin Jia
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale,
by Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, Sravan Bodapati, Katrin Kirchhoff and Dan Roth

Instruction Tuning

<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning or Self-aligning? Rethinking Instruction Fine-tuning,
by Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Guanglu Wan, Xunliang Cai et al.
<img src=https://img.shields.io/badge/AAAI-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can Large Language Models Understand Real-World Complex Instructions?,
by Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Jiaqing Liang et al.
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Model Instruction Following: A Survey of Progresses and Challenges,
by Renze Lou, Kai Zhang and Wenpeng Yin
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models Can Be Easily Distracted by Irrelevant Context,
by Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Sch"arli and Denny Zhou
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Capacity for Moral Self-Correction in Large Language Models,
by Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamile Lukosiute, Anna Chen, Anna Goldie, Azalia Mirhoseini et al.
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Exploring the Benefits of Training Expert Language Models over Instruction Tuning,
by Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee and Minjoon Seo
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Chinese Open Instruction Generalist: A Preliminary Release,
by Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Instruction Tuning with GPT-4,
by Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley and Jianfeng Gao
<img src=https://img.shields.io/badge/NeurIPS-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Visual Instruction Tuning,
by Haotian Liu, Chunyuan Li, Qingyang Wu and Yong Jae Lee
<img src=https://img.shields.io/badge/NeurIPS-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning,
by Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung et al.
<img src=https://img.shields.io/badge/NeurIPS-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction,
by Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li and Ying Shan
<img src=https://img.shields.io/badge/NeurIPS-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark,
by Zhenfei Yin, Jiong Wang, Jianjian Cao, Zhelun Shi, Dingning Liu, Mukai Li, Xiaoshui Huang, Zhiyong Wang et al.
<img src=https://img.shields.io/badge/ICLR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Finetuned Language Models are Zero-Shot Learners,
by Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> LaMDA: Language Models for Dialog Applications,
by Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Scaling Instruction-Finetuned Language Models,
by Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang et al.
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks,
by Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Self-Instruct: Aligning Language Model with Self Generated Instructions,
by Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi and Hannaneh Hajishirzi
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> How Many Data Samples is an Additional Instruction Worth?,
by Ravsehaj Singh Puri, Swaroop Mishra, Mihir Parmar and Chitta Baral

RLHF

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Capacity for Moral Self-Correction in Large Language Models,
by Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamile Lukosiute, Anna Chen, Anna Goldie, Azalia Mirhoseini et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Aligning Text-to-Image Models using Human Feedback,
by Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Secrets of RLHF in Large Language Models Part I: PPO,
by Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei Shen, Binghai Wang, Yan Liu, Senjie Jin et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback,
by Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, J'er'emy Scheurer, Javier Rando, Rachel Freedman, Tomasz Korbak et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback,
by Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune and Abhinav Rastogi
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback,
by Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is Reinforcement Learning (Not) for Natural Language Processing?: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization,
by Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kiant'e Brantley, Jack Hessel, Rafet Sifa, Christian Bauckhage, Hannaneh Hajishirzi and Yejin Choi
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Teaching language models to support answers with verified quotes,
by Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, H. Francis Song, Martin Chadwick, Mia Glaese, Susannah Young et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improving alignment of dialogue agents via targeted human judgements,
by Amelia Glaese, Nat McAleese, Maja Trebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Scaling Laws for Reward Model Overoptimization,
by Gao, Leo, Schulman, John and Hilton, Jacob
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned,
by Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning,
by Deborah Cohen, Moonkyung Ryu, Yinlam Chow, Orgad Keller, Ido Greenberg, Avinatan Hassidim, Michael Fink, Yossi Matias et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Training language models to follow instructions with human feedback,
by Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal et al.
<img src=https://img.shields.io/badge/IJRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences,
by Erdem Biyik, Dylan P. Losey, Malayandi Palan, Nicholas C. Landolfi, Gleb Shevchuk and Dorsa Sadigh
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> WebGPT: Browser-assisted question-answering with human feedback,
by Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain et al.
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Recursively Summarizing Books with Human Feedback,
by Jeff Wu, Long Ouyang, Daniel M. Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike and Paul F. Christiano
<img src=https://img.shields.io/badge/NeurIPS-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning to summarize with human feedback,
by Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei et al.
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Dialogue Response Ranking Training with Large-Scale Human Feedback Data,
by Xiang Gao, Yizhe Zhang, Michel Galley, Chris Brockett and Bill Dolan
<img src=https://img.shields.io/badge/CoRR-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fine-Tuning Language Models from Human Preferences,
by Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul F. Christiano and Geoffrey Irving
<img src=https://img.shields.io/badge/NeurIPS-2017-blue alt="img" style="zoom:100%; vertical-align: middle" /> Deep Reinforcement Learning from Human Preferences,
by Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg and Dario Amodei

Pre-Training Techniques

<img src=https://img.shields.io/badge/OpenAI-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> GPT-4 Technical Report, <img src=https://img.shields.io/badge/GPT--4-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by OpenAI
<img src=https://img.shields.io/badge/OpenAI-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> GPT-4 System Card, <img src=https://img.shields.io/badge/GPT--4-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by OpenAI
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> OPT: Open Pre-trained Transformer Language Models, <img src=https://img.shields.io/badge/OPT-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona T. Diab et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> WeLM: A Well-Read Pre-trained Language Model for Chinese, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Hui Su, Xiao Zhou, Houjin Yu, Yuwen Chen, Zilin Zhu, Yang Yu and Jie Zhou
<img src=https://img.shields.io/badge/NeurIPS-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Language Models are Few-Shot Learners, <img src=https://img.shields.io/badge/GPT--3-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam et al.
<img src=https://img.shields.io/badge/ICLR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators, <img src=https://img.shields.io/badge/ELECTRA-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Kevin Clark, Minh-Thang Luong, Quoc V. Le and Christopher D. Manning
<img src=https://img.shields.io/badge/EMNLP_Findings-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Revisiting Pre-Trained Models for Chinese Natural Language Processing,
by Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang and Guoping Hu
<img src=https://img.shields.io/badge/CoRR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> DeBERTa: Decoding-enhanced BERT with Disentangled Attention, <img src=https://img.shields.io/badge/DeBERTa-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Pengcheng He, Xiaodong Liu, Jianfeng Gao and Weizhu Chen
<img src=https://img.shields.io/badge/JMLR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer, <img src=https://img.shields.io/badge/T5-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li et al.
<img src=https://img.shields.io/badge/TACL-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Primer in BERTology: What We Know About How BERT Works,
by Anna Rogers, Olga Kovaleva and Anna Rumshisky
<img src=https://img.shields.io/badge/OpenAI-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Language Models are Unsupervised Multitask Learners, <img src=https://img.shields.io/badge/GPT--2-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Radford, Alec, Wu, Jeffrey, Child, Rewon, Luan, David, Amodei, Dario and Sutskever, Ilya
<img src=https://img.shields.io/badge/NAACL-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, <img src=https://img.shields.io/badge/BERT-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova
<img src=https://img.shields.io/badge/CoRR-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> RoBERTa: A Robustly Optimized BERT Pretraining Approach, <img src=https://img.shields.io/badge/RoBERTa-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis et al.
<img src=https://img.shields.io/badge/EMNLP-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks, <img src=https://img.shields.io/badge/SBERT-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Nils Reimers and Iryna Gurevych
<img src=https://img.shields.io/badge/OpenAI-2018-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improving language understanding by generative pre-training, <img src=https://img.shields.io/badge/GPT--1-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Radford, Alec, Narasimhan, Karthik, Salimans, Tim, Sutskever, Ilya and others

Mixtures of Experts

<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Efficient Large Scale Language Modeling with Mixtures of Experts, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" /> <img src=https://img.shields.io/badge/MoE-yellow alt="img" style="zoom:100%; vertical-align: middle" />
by Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du et al.
<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation,
by Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He, Tuo Zhao and Weizhu Chen
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints,
by Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts,
by Tao Zhong, Zhixiang Chi, Li Gu, Yang Wang, Yuanhao Yu and Jin Tang

Knowledge Enhanced

<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> ConcEPT: Concept-Enhanced Pre-Training for Language Models,
by Xintao Wang, Zhouhong Gu, Jiaqing Liang, Dakuan Lu, Yanghua Xiao and Wei Wang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Knowledge-enhanced Neural Machine Reasoning: A Review,
by Tanmoy Chowdhury, Chen Ling, Xuchao Zhang, Xujiang Zhao, Guangji Bai, Jian Pei, Haifeng Chen and Liang Zhao
<img src=https://img.shields.io/badge/NeurIPS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Deep Bidirectional Language-Knowledge Graph Pretraining,
by Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D. Manning, Percy Liang and Jure Leskovec
<img src=https://img.shields.io/badge/TKDE-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Survey on Knowledge-Enhanced Pre-trained Language Models,
by Chaoqi Zhen, Yanlei Shang, Xiangyu Liu, Yifei Li, Yong Chen and Dell Zhang
<img src=https://img.shields.io/badge/FCST-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Review of Knowledge-Enhanced Pre-trained Language Models,
by Yi, HAN, Linbo, QIAO, Dongsheng, LI and Xiangke, LIAO
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems,
by Sagi Shaier, Lawrence Hunter and Katharina Kann
<img src=https://img.shields.io/badge/COLING-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Domain Knowledge Enhanced Pre-Trained Language Model for Vertical Search: Case Study on Medicinal Products,
by Kesong Liu, Jianhui Jiang and Feifei Lyu
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Knowledge Prompting in Pre-trained Language Model for Natural Language Understanding,
by Jianing Wang, Wenkang Huang, Minghui Qiu, Qiuhui Shi, Hongbin Wang, Xiang Li and Ming Gao
<img src=https://img.shields.io/badge/ACL_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Dict-BERT: Enhancing Language Model Pre-training with Dictionary,
by Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng and Meng Jiang
<img src=https://img.shields.io/badge/ICLR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> GreaseLM: Graph REASoning Enhanced Language Models,
by Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D. Manning and Jure Leskovec
<img src=https://img.shields.io/badge/CIKM-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> SPOT: Knowledge-Enhanced Language Representations for Information Extraction,
by Jiacheng Li, Yannis Katsis, Tyler Baldwin, Ho-Cheol Kim, Andrew Bartko, Julian J. McAuley and Chun-Nan Hsu
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Relation-aware Language-Graph Transformer for Question Answering,
by Jinyoung Park, Hyeong Kyu Choi, Juyeon Ko, Hyeon-Jin Park, Ji-Hoon Kim, Jisu Jeong, Kyung-Min Kim and Hyunwoo J. Kim
<img src=https://img.shields.io/badge/SIGIR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Knowledge-based Review Generation by Coherence Enhanced Text Planning,
by Junyi Li, Wayne Xin Zhao, Zhicheng Wei, Nicholas Jing Yuan and Ji-Rong Wen
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Three-Stage Learning Framework for Low-Resource Knowledge-Grounded Dialogue Generation,
by Shilei Liu, Xiaofeng Zhao, Bochao Li, Feiliang Ren, Longhui Zhang and Shujuan Yin
<img src=https://img.shields.io/badge/NAACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledge,
by Bodhisattwa Prasad Majumder, Sudha Rao, Michel Galley and Julian J. McAuley
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation,
by Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen et al.
<img src=https://img.shields.io/badge/NAACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improving Biomedical Pretrained Language Models with Knowledge,
by Zheng Yuan, Yijia Liu, Chuanqi Tan, Songfang Huang and Fei Huang
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation,
by Wenhu Chen, Yu Su, Xifeng Yan and William Yang Wang
<img src=https://img.shields.io/badge/TACL-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation,
by Jian Guan, Fei Huang, Minlie Huang, Zhihao Zhao and Xiaoyan Zhu
<img src=https://img.shields.io/badge/CIKM-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Knowledge-Enhanced Personalized Review Generation with Capsule Graph Neural Network,
by Junyi Li, Siqing Li, Wayne Xin Zhao, Gaole He, Zhicheng Wei, Nicholas Jing Yuan and Ji-Rong Wen
<img src=https://img.shields.io/badge/EMNLP-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models,
by Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar and Bryan Catanzaro
<img src=https://img.shields.io/badge/ACL-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Barack's Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling,
by Robert L. Logan IV, Nelson F. Liu, Matthew E. Peters, Matt Gardner and Sameer Singh
<img src=https://img.shields.io/badge/NeurIPS-2009-blue alt="img" style="zoom:100%; vertical-align: middle" /> Zero-shot Learning with Semantic Output Codes,
by Mark Palatucci, Dean Pomerleau, Geoffrey E. Hinton and Tom M. Mitchell

Knowledge Distillation

<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes,
by Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee et al.
<img src=https://img.shields.io/badge/ASE-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Compressing Pre-trained Models of Code into 3 MB,
by Jieke Shi, Zhou Yang, Bowen Xu, Hong Jin Kang and David Lo
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Less is More: Task-aware Layer-wise Distillation for Language Model Compression,
by Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen and Tuo Zhao
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts,
by Tao Zhong, Zhixiang Chi, Li Gu, Yang Wang, Yuanhao Yu and Jin Tang
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> CN-AutoMIC: Distilling Chinese Commonsense Knowledge from Pretrained Language Models,
by Chenhao Wang, Jiachun Li, Yubo Chen, Kang Liu and Jun Zhao
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation,
by Dongha Choi, Hongseok Choi and Hyunju Lee
<img src=https://img.shields.io/badge/Neurocomputing-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Preparing lessons: Improve knowledge distillation with better supervision, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Tiancheng Wen, Shenqi Lai and Xueming Qian
<img src=https://img.shields.io/badge/ACL_Findings-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains,
by Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong and Furu Wei
<img src=https://img.shields.io/badge/ACL-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation,
by Shizhe Diao, Ruijia Xu, Hongjin Su, Yilei Jiang, Yan Song and Tong Zhang
<img src=https://img.shields.io/badge/ACL-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Distilling Knowledge Learned in BERT for Text Generation,
by Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu and Jingjing Liu
<img src=https://img.shields.io/badge/AAAI-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improved Knowledge Distillation via Teacher Assistant, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Seyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa and Hassan Ghasemzadeh
<img src=https://img.shields.io/badge/CVPR-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Regularizing Class-Wise Predictions via Self-Knowledge Distillation, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Sukmin Yun, Jongjin Park, Kimin Lee and Jinwoo Shin
<img src=https://img.shields.io/badge/CVPR-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Relational Knowledge Distillation, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Wonpyo Park, Dongju Kim, Yan Lu and Minsu Cho
<img src=https://img.shields.io/badge/CoRR-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Revisit Knowledge Distillation: a Teacher-free Framework, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Li Yuan, Francis E. H. Tay, Guilin Li, Tao Wang and Jiashi Feng
<img src=https://img.shields.io/badge/ICCV-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Knowledge Distillation via Route Constrained Optimization, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Xiao Jin, Baoyun Peng, Yichao Wu, Yu Liu, Jiaheng Liu, Ding Liang, Junjie Yan and Xiaolin Hu
<img src=https://img.shields.io/badge/CoRR-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improving Generalization and Robustness with Noisy Collaboration in Knowledge Distillation, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Elahe Arani, Fahad Sarfraz and Bahram Zonooz
<img src=https://img.shields.io/badge/CoRR-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> Distilling Task-Specific Knowledge from BERT into Simple Neural Networks, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Raphael Tang, Yao Lu, Linqing Liu, Lili Mou, Olga Vechtomova and Jimmy Lin
<img src=https://img.shields.io/badge/ICLR-2019-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Jonathan Frankle and Michael Carbin
<img src=https://img.shields.io/badge/ICML-2018-blue alt="img" style="zoom:100%; vertical-align: middle" /> Born-Again Neural Networks, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti and Anima Anandkumar
<img src=https://img.shields.io/badge/ICLR-2017-blue alt="img" style="zoom:100%; vertical-align: middle" /> Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Sergey Zagoruyko and Nikos Komodakis
<img src=https://img.shields.io/badge/ICLR-2017-blue alt="img" style="zoom:100%; vertical-align: middle" /> Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Antti Tarvainen and Harri Valpola
<img src=https://img.shields.io/badge/CoRR-2017-blue alt="img" style="zoom:100%; vertical-align: middle" /> Deep Mutual Learning, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Ying Zhang, Tao Xiang, Timothy M. Hospedales and Huchuan Lu
<img src=https://img.shields.io/badge/CoRR-2016-blue alt="img" style="zoom:100%; vertical-align: middle" /> Deep Model Compression: Distilling Knowledge from Noisy Teachers, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Bharat Bhusan Sau and Vineeth N. Balasubramanian
<img src=https://img.shields.io/badge/CoRR-2015-blue alt="img" style="zoom:100%; vertical-align: middle" /> Distilling the Knowledge in a Neural Network, <img src=https://img.shields.io/badge/Code-skyblue alt="img" style="zoom:100%; vertical-align: middle" />
by Geoffrey E. Hinton, Oriol Vinyals and Jeffrey Dean

Knowledge Generation

<img src=https://img.shields.io/badge/EACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Crawling the Internal Knowledge-Base of Language Models,
by Roi Cohen, Mor Geva, Jonathan Berant and Amir Globerson
本文提出一种从语言模型中提取结构化知识图谱的方法；使用专门设计的提示来控制提取过程中的精度和召回率；在GPT-3上进行了评估，显示了高精确度的结果。
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Understanding Finetuning for Factual Knowledge Extraction from Language Models,
by Mehran Kazemi, Sid Mittal and Deepak Ramachandran
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> LLMs4OL: Large Language Models for Ontology Learning,
by Hamed Babaei Giglou, Jennifer D'Souza and S"oren Auer
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Tab2KG: Semantic Table Interpretation with Lightweight Semantic Profiles,
by Simon Gottschalk and Elena Demidova
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Generative Knowledge Graph Construction: A Review,
by Hongbin Ye, Ningyu Zhang, Hui Chen and Huajun Chen
<img src=https://img.shields.io/badge/EMNLP_Findings-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Calibrating Factual Knowledge in Pretrained Language Models,
by Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui and Lei Li
<img src=https://img.shields.io/badge/ICLR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> P-Adapters: Robustly Extracting Factual Information from Language Models with Diverse Prompts,
by Benjamin Newman, Prafulla Kumar Choubey and Nazneen Rajani
<img src=https://img.shields.io/badge/ACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Generated Knowledge Prompting for Commonsense Reasoning,
by Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi and Hannaneh Hajishirzi
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering,
by Jiacheng Liu, Skyler Hallinan, Ximing Lu, Pengfei He, Sean Welleck, Hannaneh Hajishirzi and Yejin Choi
<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Symbolic Knowledge Distillation: from General Language Models to Commonsense Models,
by Peter West, Chandra Bhagavatula, Jack Hessel, Jena D. Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation,
by Chandra Bhagavatula, Jena D. Hwang, Doug Downey, Ronan Le Bras, Ximing Lu, Keisuke Sakaguchi, Swabha Swayamdipta, Peter West et al.

Knowledge Editing

<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is it Possible to Edit Large Language Models Robustly?,
by Xinbei Ma, Tianjie Ju, Jiyang Qiu, Zhuosheng Zhang, Hai Zhao, Lifeng Liu and Yulong Wang
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Updating Language Models with Unstructured Facts: Towards Practical Knowledge Editing,
by Xiaobao Wu, Liangming Pan, William Yang Wang and Anh Tuan Luu
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Event-level Knowledge Editing,
by Hao Peng, Xiaozhi Wang, Chunyang Li, Kaisheng Zeng, Jiangshan Duo, Yixin Cao, Lei Hou and Juanzi Li
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Robustness of edited neural networks,
by Davis Brown, Charles Godfrey, Cody Nizinski, Jonathan Tu and Henry Kvinge
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Transformer-Patcher: One Mistake worth One Neuron,
by Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong and Zhang Xiong
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Editing Language Model-based Knowledge Graph Embeddings,
by Siyuan Cheng, Ningyu Zhang, Bozhong Tian, Zelin Dai, Feiyu Xiong, Wei Guo and Huajun Chen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> PMET: Precise Model Editing in a Transformer,
by Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma and Jie Yu
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions,
by Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning, Christopher Potts and Danqi Chen
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark,
by Jason Hoelscher-Obermaier, Julia Persson, Esben Kran, Ioannis Konstas and Fazl Barez
<img src=https://img.shields.io/badge/EACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models,
by Peter Hase, Mona T. Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal and Srinivasan Iyer
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> MedEdit: Model Editing for Medical Question Answering with External Knowledge Bases,
by Yucheng Shi, Shaochen Xu, Zhengliang Liu, Tianming Liu, Xiang Li and Ninghao Liu
<img src=https://img.shields.io/badge/ICLR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fast Model Editing at Scale,
by Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn and Christopher D. Manning
<img src=https://img.shields.io/badge/ICML-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Memory-Based Model Editing at Scale,
by Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning and Chelsea Finn
<img src=https://img.shields.io/badge/NeurIPS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Locating and editing factual associations in gpt,
by Meng, Kevin, Bau, David, Andonian, Alex J and Belinkov, Yonatan
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors,
by Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim and Marzyeh Ghassemi
<img src=https://img.shields.io/badge/EMNLP-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Editing Factual Knowledge in Language Models,
by Nicola De Cao, Wilker Aziz and Ivan Titov

Reasoning

<img src=https://img.shields.io/badge/CoRR-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Reasoning with Reinforced Functional Token Tuning,
by Kongcheng Zhang, Qi Yao, Baisheng Lai, Jiaxing Huang, Wenkai Fang, Dacheng Tao, Mingli Song and Shunyu Liu
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Do Large Language Models Latently Perform Multi-Hop Reasoning?,
by Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva and Sebastian Riedel
<img src=https://img.shields.io/badge/ACL-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models,
by Anonymous Submission
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> SoFA: Shielded On-the-fly Alignment via Priority Rule Following,
by Xinyu Lu, Bowen Yu, Yaojie Lu, Hongyu Lin, Haiyang Yu, Le Sun, Xianpei Han and Yongbin Li
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization,
by Boshi Wang, Xiang Yue, Yu Su and Huan Sun
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoning,
by Debrup Das, Debopriyo Banerjee, Somak Aditya and Ashish Kulkarni
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning,
by Eli Schwartz, Leshem Choshen, Joseph Shtok, Sivan Doveh, Leonid Karlinsky and Assaf Arbelle
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can LLM Graph Reasoning Generalize beyond Pattern Memorization?,
by Yizhuo Zhang, Heng Wang, Shangbin Feng, Zhaoxuan Tan, Xiaochuang Han, Tianxing He and Yulia Tsvetkov
<img src=https://img.shields.io/badge/ACL-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models,
by Mihir Parmar, Nisarg Patel, Neeraj Varshney, Mutsumi Nakamura, Man Luo, Santosh Mashetty, Arindam Mitra and Chitta Baral
<img src=https://img.shields.io/badge/Findings_of_the_Association_for_Computational_Linguistics:_{NAACL} 2024,_Mexico_City,_Mexico,_June_16--21,_2024-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Language Models can be Deductive Solvers,
by Jiazhan Feng, Ruochen Xu, Junheng Hao, Hiteshi Sharma, Yelong Shen, Dongyan Zhao and Weizhu Chen
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs,
by Haritz Puerto, Martin Tutek, Somak Aditya, Xiaodan Zhu and Iryna Gurevych
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> ThoughtSource: A central hub for large language model reasoning data,
by Simon Ott, Konstantin Hebenstreit, Valentin Li'evin, Christoffer Egeberg Hother, Milad Moradi, Maximilian Mayrhauser, Robert Praas, Ole Winther et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Knowledge-enhanced Neural Machine Reasoning: A Review,
by Tanmoy Chowdhury, Chen Ling, Xuchao Zhang, Xujiang Zhao, Guangji Bai, Jian Pei, Haifeng Chen and Liang Zhao
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning,
by Yunhu Ye, Binyuan Hui, Min Yang, Binhua Li, Fei Huang and Yongbin Li
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Specializing Smaller Language Models towards Multi-Step Reasoning,
by Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal and Tushar Khot
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> MathPrompter: Mathematical Reasoning using Large Language Models,
by Imani, Shima, Du, Liang and Shrivastava, Harsh
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Using Language Models For Knowledge Acquisition in Natural Language Reasoning Problems,
by Fangzhen Lin, Ziyi Shou and Chengcai chen
<img src=https://img.shields.io/badge/ICLR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Complexity-Based Prompting for Multi-step Reasoning,
by Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark and Tushar Khot
<img src=https://img.shields.io/badge/EACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Penguins Don't Fly: Reasoning about Generics through Instantiations and Exceptions,
by Emily Allaway, Jena D. Hwang, Chandra Bhagavatula, Kathleen R. McKeown, Doug Downey and Yejin Choi
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> NeuroComparatives: Neuro-Symbolic Distillation of Comparative Knowledge,
by Phillip Howard, Junlin Wang, Vasudev Lal, Gadi Singer, Yejin Choi and Swabha Swayamdipta
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge,
by Jiangjie Chen, Wei Shi, Ziquan Fu, Sijie Cheng, Lei Li and Yanghua Xiao
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond,
by Fangzhi Xu, Qika Lin, Jiawei Han, Tianzhe Zhao, Jun Liu and Erik Cambria
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning To Teach Large Language Models Logical Reasoning,
by Meiqi Chen, Yubo Ma, Kaitao Song, Yixin Cao, Yan Zhang and Dongsheng Li
<img src=https://img.shields.io/badge/NeurIPS-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Schema-learning and rebinding as mechanisms of in-context learning and emergence,
by Sivaramakrishnan Swaminathan, Antoine Dedieu, Rajkumar Vasudeva Raju, Murray Shanahan, Miguel L'azaro-Gredilla and Dileep George
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Are LLMs Rigorous Logical Reasoner? Empowering Natural Language Proof Generation with Contrastive Stepwise Decoding,
by Ying Su, Xiaojin Fu, Mingwen Liu and Zhijiang Guo
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning,
by Tianhua Zhang, Jiaxin Ge, Hongyin Luo, Yung-Sung Chuang, Mingye Gao, Yuan Gong, Xixin Wu, Yoon Kim et al.
<img src=https://img.shields.io/badge/Advances_in_Neural_Information_Processing_Systems_36:_Annual_Conference on_Neural_Information_Processing_Systems_2023,_NeurIPS_2023,_New_Orleans, LA,_USA,December_10--_16,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Information,
by Mehran Kazemi, Quan Yuan, Deepti Bhatia, Najoung Kim, Xin Xu, Vaiva Imbrasaite and Deepak Ramachandran
<img src=https://img.shields.io/badge/Findings_of_the_Association_for_Computational_Linguistics:_{EMNLP} 2023,_Singapore,_December_6--10,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning,
by Liangming Pan, Alon Albalak, Xinyi Wang and William Yang Wang
<img src=https://img.shields.io/badge/Advances_in_Neural_Information_Processing_Systems_36:_Annual_Conference on_Neural_Information_Processing_Systems_2023,_NeurIPS_2023,_New_Orleans, LA,_USA,December_10--_16,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Faith and Fate: Limits of Transformers on Compositionality,
by Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West et al.
<img src=https://img.shields.io/badge/ICML-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improved logical reasoning of language models via differentiable symbolic programming,
by Zhang, Hanlin, Li, Ziyang, Huang, Jiani, Naik, Mayur and Xing, Eric
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> LILA: A Unified Benchmark for Mathematical Reasoning,
by Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord et al.
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations,
by Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras and Yejin Choi
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation,
by Swarnadeep Saha, Xinyan Velocity Yu, Mohit Bansal, Ramakanth Pasunuru and Asli Celikyilmaz
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks,
by Wenhu Chen, Xueguang Ma, Xinyi Wang and William W. Cohen
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Impact of Symbolic Representations on In-context Learning for Few-shot Reasoning,
by Hanlin Zhang, Yi-Fan Zhang, Li Erran Li and Eric P. Xing
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Reasoning in Large Language Models: A Survey,
by Jie Huang and Kevin Chen-Chuan Chang
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression,
by Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen and Xiaodan Liang
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Least-to-Most Prompting Enables Complex Reasoning in Large Language Models,
by Denny Zhou, Nathanael Sch"arli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet et al.
(1) 两阶段的prompt，第一阶段问题分解（通过in-context learning实现，context中包含了其他问题的分解示例），对于每个问题，分解出回答该问题需要先回答什么子问题；
(2) 在第二阶段中，从后往前依次解决子问题，同样通过in-context learing得到，每次LLM的回答会参与组成下一个问题的prompt。
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rationale-Augmented Ensembles in Language Models,
by Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi and Denny Zhou
<img src=https://img.shields.io/badge/NeurIPS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> The unreliability of explanations in few-shot prompting for textual reasoning,
by Ye, Xi and Durrett, Greg
<img src=https://img.shields.io/badge/KDD-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding,
by Wayne Xin Zhao, Kun Zhou, Zheng Gong, Beichen Zhang, Yuanhang Zhou, Jing Sha, Zhigang Chen, Shijin Wang et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> ThinkSum: Probabilistic reasoning over sets using large language models,
by Batu Ozturkler, Nikolay Malkin, Zhen Wang and Nebojsa Jojic
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> RobustLR: A Diagnostic Benchmark for Evaluating Logical Robustness of Deductive Reasoners,
by Soumya Sanyal, Zeyi Liao and Xiang Ren
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Solving Quantitative Reasoning Problems with Language Models,
by Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay V. Ramasesh, Ambrose Slone, Cem Anil et al.
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> LogicNMR: Probing the Non-monotonic Reasoning Ability of Pre-trained Language Models,
by Yeliang Xiu, Zhanhao Xiao and Yongmei Liu
<img src=https://img.shields.io/badge/EMNLP_Findings-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Thinking Like a Skeptic: Defeasible Inference in Natural Language,
by Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith and Yejin Choi

Chain of Thought

<img src=https://img.shields.io/badge/-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> LightThinker: Thinking Step-by-Step Compression,
by Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen et al.
<img src=https://img.shields.io/badge/CoRR-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Is Depth All You Need? An Exploration of Iterative Reasoning in LLMs,
by Zongqian Wu, Tianyu Li, Baoduo Xu, Jiaying Yang, Mengmeng Zhan, Xiaofeng Zhu and Lei Feng
<img src=https://img.shields.io/badge/-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in Large Language Models,
by Feiyang Li, Peng Fang, Zhan Shi, Arijit Khan, Fang Wang, Dan Feng, Weihao Wang, Xin Zhang et al.
<img src=https://img.shields.io/badge/the_63rd_Annual_Meeting_of_the_Association_for_Computational Linguistics_(Volume_1:Long_Papers),{ACL}_2025,_Vienna,Austria, July_27--_August_1,_2025-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMs,
by Xiang Zhang, Juntai Cao, Chenyu You and Dujian Ding
<img src=https://img.shields.io/badge/the_63rd_Annual_Meeting_of_the_Association_for_Computational Linguistics_(Volume_1:Long_Papers),{ACL}_2025,_Vienna,Austria, July_27--_August_1,_2025-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspective,
by Yiyao Yu, Yuxiang Zhang, Dongdong Zhang, Xiao Liang, Hengyuan Zhang, Xingxing Zhang, Mahmoud Khademi, Hany Hassan Awadalla et al.
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts,
by Maciej Besta, Florim Memedi, Zhenyu Zhang, Robert Gerstenberger, Nils Blach, Piotr Nyczyk, Marcin Copik, Grzegorz Kwaśniewski et al.
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation,
by Zihao Wang, Anji Liu, Haowei Lin, Jiaqi Li, Xiaojian Ma and Yitao Liang
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models,
by Wenshan Wu, Shaoguang Mao, Yadong Zhang, Yan Xia, Li Dong, Lei Cui and Furu Wei
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Boosting of Thoughts: Trial-and-Error Problem Solving with Large Language Models,
by Sijia Chen, Baochun Li and Di Niu
<img src=https://img.shields.io/badge/AAAI-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Visual Chain-of-Thought Prompting for Knowledge-Based Visual Reasoning,
by Zhenfang Chen, Qinhong Zhou, Yikang Shen, Yining Hong, Zhiqing Sun, Dan Gutfreund and Chuang Gan
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphs,
by Minh-Vuong Nguyen, Linhao Luo, Fatemeh Shiri, Dinh Phung, Yuan-Fang Li, Thuy-Trang Vu and Gholamreza Haffari
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future,
by Zheng Chu, Jingchang Chen, Qianglong Chen, Weijiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu et al.
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models,
by Shibo Hao, Yi Gu, Haotian Luo, Tianyang Liu, Xiyan Shao, Xinyuan Wang, Shuhua Xie, Haodi Ma et al.
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Flow of Reasoning: Efficient Training of LLM Policy with Divergent Thinking,
by Fangxu Yu, Lai Jiang, Haoqiang Kang, Shibo Hao and Lianhui Qin
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts,
by Maciej Besta, Florim Memedi, Zhenyu Zhang, Robert Gerstenberger, Nils Blach, Piotr Nyczyk, Marcin Copik, Grzegorz Kwasniewski et al.
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Deductive Beam Search: Decoding Deducible Rationale for Chain-of-Thought Reasoning,
by Tinghui Zhu, Kai Zhang, Jian Xie and Yu Su
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Generating Chain-of-Thoughts with a Direct Pairwise-Comparison Approach to Searching for the Most Promising Intermediate Thought,
by Zhen-Yu Zhang, Siwei Han, Huaxiu Yao, Gang Niu and Masashi Sugiyama
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Chain-of-Thought Reasoning Without Prompting,
by Xuezhi Wang and Denny Zhou
<img src=https://img.shields.io/badge/Findings_of_the_Association_for_Computational_Linguistics,_{ACL}_2024, Bangkok,_Thailand_and_virtual_meeting,_August_11--16,_2024-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs,
by Bowen Jin, Chulin Xie, Jiawei Zhang, Kashob Kumar Roy, Yu Zhang, Zheng Li, Ruirui Li, Xianfeng Tang et al.
<img src=https://img.shields.io/badge/Thirty--Eighth_{AAAI}Conference_on_Artificial_Intelligence,{AAAI} 2024,Thirty--Sixth_Conference_on_Innovative_Applications_of_Artificial Intelligence,{IAAI}_2024,Fourteenth_Symposium_on_Educational_Advances in_Artificial_Intelligence,{EAAI}_2014,_February_20--27,_2024,_Vancouver, Canada-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning,
by Debjyoti Mondal, Suraj Modi, Subhadarshi Panda, Rituraj Singh and Godawari Sudhakar Rao
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> M(^\mbox3)CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought,
by Qiguang Chen, Libo Qin, Jin Zhang, Zhi Chen, Xiao Xu and Wanxiang Che
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models,
by Tongxuan Liu, Wenjiang Xu, Weizhe Huang, Xingyu Wang, Jiaxing Wang, Hailong Yang and Jing Li
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> AlignedCoT: Prompting Large Language Models via Native-Speaking Demonstrations,
by Zhicheng Yang, Yinya Huang, Jing Xiong, Liang Feng, Xiaodan Liang, Yiwei Wang and Jing Tang
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rethinking Chain-of-Thought from the Perspective of Self-Training,
by Zongqian Wu, Baoduo Xu, Ruochen Cui, Mengmeng Zhan, Xiaofeng Zhu and Lei Feng
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Multimodal Chain-of-Thought Reasoning in Language Models,
by Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis and Alex Smola
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rethinking with Retrieval: Faithful Large Language Model Inference,
by Hangfeng He, Hongming Zhang and Dan Roth
本文通过用GPT-3在三个复杂的推理任务：常识推理，时间推理和表格推理上进行大量实验来评估RR的有效性。结果表明，RR可以产生更忠实的解释，并提高LLM的性能。
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Active Prompting with Chain-of-Thought for Large Language Models,
by Shizhe Diao, Pengcheng Wang, Yong Lin and Tong Zhang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data,
by Kashun Shum, Shizhe Diao and Tong Zhang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Graph of Thoughts: Solving Elaborate Problems with Large Language Models,
by Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski et al.
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models,
by Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee and Ee-Peng Lim
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Model Guided Tree-of-Thought,
by Jieyi Long
<img src=https://img.shields.io/badge/ICLR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Language models are multilingual chain-of-thought reasoners,
by Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Large Language Models,
by Yao Yao, Zuchao Li and Hai Zhao
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Thinking Like an Expert: Multimodal Hypergraph-of-Thought (HoT) Reasoning to boost Foundation Modals,
by Fanglong Yao, Changyuan Tian, Jintao Liu, Zequn Zhang, Qing Liu, Li Jin, Shuchao Li, Xiaoyu Li et al.
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models,
by Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji and Ajay Divakaran
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought,
by Bin Lei, Pei-Hung Lin, Chunhua Liao and Caiwen Ding
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Tree of Thoughts: Deliberate Problem Solving with Large Language Models,
by Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao and Karthik Narasimhan
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Tree-of-Mixed-Thought: Combining Fast and Slow Thinking for Multi-hop Visual Reasoning,
by Pengbo Hu, Ji Qi, Xingyu Li, Hong Li, Xinqi Wang, Bing Quan, Ruiyu Wang and Yi Zhou
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Making Large Language Models Better Reasoners with Alignment,
by Peiyi Wang, Lei Li, Liang Chen, Feifan Song, Binghuai Lin, Yunbo Cao, Tianyu Liu and Zhifang Sui
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Making Large Language Models Better Reasoners with Step-Aware Verifier,
by Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou and Weizhu Chen
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Meta-CoT: Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models,
by Anni Zou, Zhuosheng Zhang, Hai Zhao and Xiangru Tang
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Chain of Thought Prompting Elicits Knowledge Augmentation,
by Dingjun Wu, Jing Zhang and Xinmei Huang
<img src=https://img.shields.io/badge/NeurIPS-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models,
by Ge Zheng, Bin Yang, Jiajin Tang, Hong-Yu Zhou and Sibei Yang
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Chain of Thought Prompt Tuning in Vision Language Models,
by Jiaxin Ge, Hongyin Luo, Siyuan Qian, Yulu Gan, Jie Fu and Shanghang Zhang
<img src=https://img.shields.io/badge/NeurIPS-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning,
by Yingcong Li, Kartik Sreenivasan, Angeliki Giannou, Dimitris Papailiopoulos and Samet Oymak
<img src=https://img.shields.io/badge/IJCNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Faithful Chain-of-Thought Reasoning,
by Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki and Chris Callison-Burch
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoning,
by Jinyuan Wang, Junlong Li and Hai Zhao
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents,
by Zhuosheng Zhang, Yao Yao, Aston Zhang, Xiangru Tang, Xinbei Ma, Zhiwei He, Yiming Wang, Mark Gerstein et al.
<img src=https://img.shields.io/badge/Advances_in_Neural_Information_Processing_Systems_36:_Annual_Conference on_Neural_Information_Processing_Systems_2023,_NeurIPS_2023,_New_Orleans, LA,_USA,December_10--_16,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Deductive Verification of Chain-of-Thought Reasoning,
by Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic and Hao Su
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Instruction Induction: From Few Examples to Natural Language Task Descriptions,
by Or Honovich, Uri Shaham, Samuel R. Bowman and Omer Levy
(1) 探索了利用LLM在几个样本的情况下归纳出任务指令的能力；
(2) 测量两个指标：1. 模型归纳指令与人类归纳的指令对比，2. 利用模型归纳的指令作为prompt进行预测的执行准确率；
(3) 相比于GPT-3，InstructGPT效果更好，理所当然。
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Iteratively Prompt Pre-trained Language Models for Chain of Thought,
by Boshi Wang, Xiang Deng and Huan Sun
(1) 提出了一种迭代式的prompt-tuning方法，他们认为soft prompt应该带有语境，即在自回归解码时不同时刻应该有不同的prompt向量；
(2) 利用BERT为encoder-decoder架构的PLM生成prompt，在每个解码时刻BERT都会根据先前时刻的上下文生成一组新的prompt向量，提供给PLM生成新的上下文，迭代往复。
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Complexity-Based Prompting for Multi-Step Reasoning,
by Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark and Tushar Khot
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Measuring and Narrowing the Compositionality Gap in Language Models,
by Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith and Mike Lewis
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Automatic Chain of Thought Prompting in Large Language Models,
by Zhuosheng Zhang, Aston Zhang, Mu Li and Alex Smola
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Chain of Thought Prompting Elicits Reasoning in Large Language Models,
by Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le and Denny Zhou
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Self-Consistency Improves Chain of Thought Reasoning in Language Models,
by Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi and Denny Zhou
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango,
by Aman Madaan and Amir Yazdanbakhsh
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters,
by Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer and Huan Sun
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> PaLM: Scaling Language Modeling with Pathways,
by Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> LAMBADA: Backward Chaining for Automated Reasoning in Natural Language,
by Seyed Mehran Kazemi, Najoung Kim, Deepti Bhatia, Xin Xu and Deepak Ramachandran
<img src=https://img.shields.io/badge/NeurIPS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Star: Self-taught reasoner bootstrapping reasoning with reasoning,
by Zelikman, Eric, Mu, Jesse, Goodman, Noah D and Wu, Yuhuai Tony
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought,
by Abulhair Saparov and He He
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models are Zero-Shot Reasoners,
by Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo and Yusuke Iwasawa
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning,
by Antonia Creswell, Murray Shanahan and Irina Higgins
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Emergent Abilities of Large Language Models,
by Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma et al.
<img src=https://img.shields.io/badge/KDD-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding,
by Wayne Xin Zhao, Kun Zhou, Zheng Gong, Beichen Zhang, Yuanhang Zhou, Jing Sha, Zhigang Chen, Shijin Wang et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models Are Reasoning Teachers,
by Namgyu Ho, Laura Schmid and Se-Young Yun
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models are reasoners with Self-Verification,
by Yixuan Weng, Minjun Zhu, Shizhu He, Kang Liu and Jun Zhao
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Reasoning with Language Model Prompting: A Survey,
by Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> PAL: Program-aided Language Models,
by Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan and Graham Neubig
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models are few(1)-shot Table Reasoners,
by Wenhu Chen
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Large Language Models Can Self-Improve,
by Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu and Jiawei Han

Multi-Step Reasoning

<img src=https://img.shields.io/badge/CoRR-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Reasoning with Reinforced Functional Token Tuning,
by Kongcheng Zhang, Qi Yao, Baisheng Lai, Jiaxing Huang, Wenkai Fang, Dacheng Tao, Mingli Song and Shunyu Liu
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Specializing Smaller Language Models towards Multi-Step Reasoning,
by Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal and Tushar Khot
<img src=https://img.shields.io/badge/ICLR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Complexity-Based Prompting for Multi-step Reasoning,
by Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark and Tushar Khot
<img src=https://img.shields.io/badge/Advances_in_Neural_Information_Processing_Systems_36:_Annual_Conference on_Neural_Information_Processing_Systems_2023,_NeurIPS_2023,_New_Orleans, LA,_USA,December_10--_16,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Faith and Fate: Limits of Transformers on Compositionality,
by Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation,
by Swarnadeep Saha, Xinyan Velocity Yu, Mohit Bansal, Ramakanth Pasunuru and Asli Celikyilmaz
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rationale-Augmented Ensembles in Language Models,
by Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi and Denny Zhou

Arithmetic Reasoning

<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoning,
by Debrup Das, Debopriyo Banerjee, Somak Aditya and Ashish Kulkarni
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> LILA: A Unified Benchmark for Mathematical Reasoning,
by Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks,
by Wenhu Chen, Xueguang Ma, Xinyi Wang and William W. Cohen
<img src=https://img.shields.io/badge/KDD-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> JiuZhang: A Chinese Pre-trained Language Model for Mathematical Problem Understanding,
by Wayne Xin Zhao, Kun Zhou, Zheng Gong, Beichen Zhang, Yuanhang Zhou, Jing Sha, Zhigang Chen, Shijin Wang et al.
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Solving Quantitative Reasoning Problems with Language Models,
by Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay V. Ramasesh, Ambrose Slone, Cem Anil et al.

Symbolic Reasoning

<img src=https://img.shields.io/badge/ACL-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models,
by Anonymous Submission
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> SoFA: Shielded On-the-fly Alignment via Priority Rule Following,
by Xinyu Lu, Bowen Yu, Yaojie Lu, Hongyu Lin, Haiyang Yu, Le Sun, Xianpei Han and Yongbin Li
<img src=https://img.shields.io/badge/-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization,
by Boshi Wang, Xiang Yue, Yu Su and Huan Sun
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning,
by Eli Schwartz, Leshem Choshen, Joseph Shtok, Sivan Doveh, Leonid Karlinsky and Assaf Arbelle
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Can LLM Graph Reasoning Generalize beyond Pattern Memorization?,
by Yizhuo Zhang, Heng Wang, Shangbin Feng, Zhaoxuan Tan, Xiaochuang Han, Tianxing He and Yulia Tsvetkov
<img src=https://img.shields.io/badge/ACL-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models,
by Mihir Parmar, Nisarg Patel, Neeraj Varshney, Mutsumi Nakamura, Man Luo, Santosh Mashetty, Arindam Mitra and Chitta Baral
<img src=https://img.shields.io/badge/Findings_of_the_Association_for_Computational_Linguistics:_{NAACL} 2024,_Mexico_City,_Mexico,_June_16--21,_2024-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Language Models can be Deductive Solvers,
by Jiazhan Feng, Ruochen Xu, Junheng Hao, Hiteshi Sharma, Yelong Shen, Dongyan Zhao and Weizhu Chen
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Code Prompting Elicits Conditional Reasoning Abilities in Text+Code LLMs,
by Haritz Puerto, Martin Tutek, Somak Aditya, Xiaodan Zhu and Iryna Gurevych
<img src=https://img.shields.io/badge/EACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Penguins Don't Fly: Reasoning about Generics through Instantiations and Exceptions,
by Emily Allaway, Jena D. Hwang, Chandra Bhagavatula, Kathleen R. McKeown, Doug Downey and Yejin Choi
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> NeuroComparatives: Neuro-Symbolic Distillation of Comparative Knowledge,
by Phillip Howard, Junlin Wang, Vasudev Lal, Gadi Singer, Yejin Choi and Swabha Swayamdipta
<img src=https://img.shields.io/badge/ACL-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge,
by Jiangjie Chen, Wei Shi, Ziquan Fu, Sijie Cheng, Lei Li and Yanghua Xiao
<img src=https://img.shields.io/badge/-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond,
by Fangzhi Xu, Qika Lin, Jiawei Han, Tianzhe Zhao, Jun Liu and Erik Cambria
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Learning To Teach Large Language Models Logical Reasoning,
by Meiqi Chen, Yubo Ma, Kaitao Song, Yixin Cao, Yan Zhang and Dongsheng Li
<img src=https://img.shields.io/badge/NeurIPS-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Schema-learning and rebinding as mechanisms of in-context learning and emergence,
by Sivaramakrishnan Swaminathan, Antoine Dedieu, Rajkumar Vasudeva Raju, Murray Shanahan, Miguel L'azaro-Gredilla and Dileep George
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Are LLMs Rigorous Logical Reasoner? Empowering Natural Language Proof Generation with Contrastive Stepwise Decoding,
by Ying Su, Xiaojin Fu, Mingwen Liu and Zhijiang Guo
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning,
by Tianhua Zhang, Jiaxin Ge, Hongyin Luo, Yung-Sung Chuang, Mingye Gao, Yuan Gong, Xixin Wu, Yoon Kim et al.
<img src=https://img.shields.io/badge/Advances_in_Neural_Information_Processing_Systems_36:_Annual_Conference on_Neural_Information_Processing_Systems_2023,_NeurIPS_2023,_New_Orleans, LA,_USA,December_10--_16,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Information,
by Mehran Kazemi, Quan Yuan, Deepti Bhatia, Najoung Kim, Xin Xu, Vaiva Imbrasaite and Deepak Ramachandran
<img src=https://img.shields.io/badge/Findings_of_the_Association_for_Computational_Linguistics:_{EMNLP} 2023,_Singapore,_December_6--10,_2023-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning,
by Liangming Pan, Alon Albalak, Xinyi Wang and William Yang Wang
<img src=https://img.shields.io/badge/ICML-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improved logical reasoning of language models via differentiable symbolic programming,
by Zhang, Hanlin, Li, Ziyang, Huang, Jiani, Naik, Mayur and Xing, Eric
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations,
by Jaehun Jung, Lianhui Qin, Sean Welleck, Faeze Brahman, Chandra Bhagavatula, Ronan Le Bras and Yejin Choi
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> The Impact of Symbolic Representations on In-context Learning for Few-shot Reasoning,
by Hanlin Zhang, Yi-Fan Zhang, Li Erran Li and Eric P. Xing
<img src=https://img.shields.io/badge/EMNLP-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expression,
by Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen and Xiaodan Liang
<img src=https://img.shields.io/badge/EMNLP_Findings-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Thinking Like a Skeptic: Defeasible Inference in Natural Language,
by Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith and Yejin Choi

Chain of Verification

<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Chain-of-Verification Reduces Hallucination in Large Language Models,
by Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz and Jason Weston

Knowledge Graph Embedding

<img src=https://img.shields.io/badge/ESWC-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> Navigating Ontology Development with Large Language Models,
by Mohammad Javad Saeedizade and Eva Blomqvist
<img src=https://img.shields.io/badge/CVPR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles,
by Ye, Shuquan, Xie, Yujia, Chen, Dongdong, Xu, Yichong, Yuan, Lu, Zhu, Chenguang and Liao, Jing
<img src=https://img.shields.io/badge/CoRR-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> Towards Foundation Models for Knowledge Graph Reasoning,
by Mikhail Galkin, Xinyu Yuan, Hesham Mostafa, Jian Tang and Zhaocheng Zhu
<img src=https://img.shields.io/badge/EMNLP-2023-blue alt="img" style="zoom:100%; vertical-align: middle" /> KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion,
by Yanbin Wei, Qiushi Huang, Yu Zhang and James T. Kwok
<img src=https://img.shields.io/badge/NeurIPS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Deep Bidirectional Language-Knowledge Graph Pretraining,
by Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D. Manning, Percy Liang and Jure Leskovec

Slow Thinking

<img src=https://img.shields.io/badge/CoRR-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models,
by Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen et al.
<img src=https://img.shields.io/badge/CoRR-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Generating, Fast and Slow: Scalable Parallel Video Generation with Video Interface Networks,
by Bhishma Dedhia, David Bourgin, Krishna Kumar Singh, Yuheng Li, Yan Kang, Zhan Xu, Niraj K. Jha and Yuchen Liu
<img src=https://img.shields.io/badge/CoRR-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking,
by Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao and Ji-Rong Wen
<img src=https://img.shields.io/badge/CoRR-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> ThinkPatterns-21k: A Systematic Study on the Impact of Thinking Patterns in LLMs,
by Pengcheng Wen, Jiaming Ji, Chi-Min Chan, Juntao Dai, Donghai Hong, Yaodong Yang, Sirui Han and Yike Guo
<img src=https://img.shields.io/badge/CoRR-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> Step Back to Leap Forward: Self-Backtracking for Boosting Reasoning of Language Models,
by Xiao-Wen Yang, Xuan-Yi Zhu, Wen-Da Wei, Dingchu Zhang, Jie-Jing Shao, Zhi Zhou, Lan-Zhe Guo and Yufeng Li
<img src=https://img.shields.io/badge/Submitted_to_ACL_Rolling_Review_--_December_2024-2025-blue alt="img" style="zoom:100%; vertical-align: middle" /> An Empirical Study of Activating Slow-thinking Capability of Large Language Models,
by Anonymous
<img src=https://img.shields.io/badge/CoRR-2024-blue alt="img" style="zoom:100%; vertical-align: middle" /> What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective,
by Ming Li, Yanhong Li and Tianyi Zhou

Federated Learning

<img src=https://img.shields.io/badge/JIS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fairness and accuracy in horizontal federated learning,
by Wei Huang, Tianrui Li, Dexian Wang, Shengdong Du, Junbo Zhang and Tianqiang Huang
<img src=https://img.shields.io/badge/TNSE-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Federated Learning Meets Multi-Objective Optimization,
by Zeou Hu, Kiarash Shaloudegi, Guojun Zhang and Yaoliang Yu
<img src=https://img.shields.io/badge/KIS-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> From distributed machine learning to federated learning: a survey,
by Ji Liu, Jizhou Huang, Yang Zhou, Xuhong Li, Shilei Ji, Haoyi Xiong and Dejing Dou
<img src=https://img.shields.io/badge/IJCAI-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Meta-Learning Based Knowledge Extrapolation for Knowledge Graphs in the Federated Setting,
by Mingyang Chen, Wen Zhang, Zhen Yao, Xiangnan Chen, Mengxiao Ding, Fei Huang and Huajun Chen
<img src=https://img.shields.io/badge/CIKM-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Mitigating Biases in Student Performance Prediction via Attention-Based Personalized Federated Learning,
by Yun-Wei Chu, Seyyedali Hosseinalipour, Elizabeth Tenorio, Laura M. Cruz Castro, Kerrie A. Douglas, Andrew Lan and Christopher G. Brinton
<img src=https://img.shields.io/badge/NAACL-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Pretrained Models for Multilingual Federated Learning,
by Orion Weller, Marc Marone, Vladimir Braverman, Dawn J. Lawrie and Benjamin Van Durme
<img src=https://img.shields.io/badge/CVPR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning,
by Liangqiong Qu, Yuyin Zhou, Paul Pu Liang, Yingda Xia, Feifei Wang, Ehsan Adeli, Li Fei-Fei and Daniel L. Rubin
<img src=https://img.shields.io/badge/TIST-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> FedBERT: When Federated Learning Meets Pre-training,
by Yuanyishu Tian, Yao Wan, Lingjuan Lyu, Dezhong Yao, Hai Jin and Lichao Sun
<img src=https://img.shields.io/badge/CoRR-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning,
by John Nguyen, Jianyu Wang, Kshitiz Malik, Maziar Sanjabi and Michael Rabbat
<img src=https://img.shields.io/badge/ICML-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Ditto: Fair and Robust Federated Learning Through Personalization,
by Tian Li, Shengyuan Hu, Ahmad Beirami and Virginia Smith
<img src=https://img.shields.io/badge/CoRR-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Fine-tuning is Fine in Federated Learning,
by Gary Cheng, Karan N. Chadha and John C. Duchi
<img src=https://img.shields.io/badge/IJCAI-2021-blue alt="img" style="zoom:100%; vertical-align: middle" /> Federated Learning with Fair Averaging,
by Zheng Wang, Xiaoliang Fan, Jianzhong Qi, Chenglu Wen, Cheng Wang and Rongshan Yu
<img src=https://img.shields.io/badge/FLPI-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Collaborative Fairness in Federated Learning,
by Lingjuan Lyu, Xinyi Xu, Qian Wang and Han Yu
<img src=https://img.shields.io/badge/ECCV-2020-blue alt="img" style="zoom:100%; vertical-align: middle" /> Federated Visual Classification with Real-World Data Distribution,
by Tzu-Ming Harry Hsu, Hang Qi and Matthew Brown

Distributed AI

<img src=https://img.shields.io/badge/EDBT-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Distributed Training of Knowledge Graph Embedding Models using Ray,
by Nasrullah Sheikh, Xiao Qin, Yaniv Gur and Berthold Reinwald
<img src=https://img.shields.io/badge/IEEE_-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Distributed Learning With Sparsified Gradient Differences,
by Yicheng Chen, Rick S. Blum, Martin Tak'ac and Brian M. Sadler
<img src=https://img.shields.io/badge/AIIoT-2022-blue alt="img" style="zoom:100%; vertical-align: middle" /> Graph Attention Neural Network Distributed Model Training,
by *Esmaeilzadeh, Arm

Truncated — view the full README on GitHub.

chatgpt

codex

gpt-3

large-language-models

Contributors

sid0527

124 commits

ZSY-SZ

124 commits

alphadl

1 commits

Languages

TeX

94.7%

Python

5.3%