《Awesome-LLM》

Updates

• [04/26/2023]: Add Open-source Instruction Data
• [04/22/2023]: Add Open-source Projects
• [04/21/2023]: Add ChatGPT-related Papers

Table of Contents

• Introduction
• Open-source Models
• Open-source Instruction Data
• Papers

Introduction

This repository collects awesome projects and resources related to large language model (LLM).

Open-source Models

StableLM

StableLM: Stability AI Language Models

GitHub: https://github.com/Stability-AI/StableLM

Colossal-AI

Colossal-AI: Making large AI models cheaper, faster, and more accessible

GitHub: https://github.com/hpcaitech/ColossalAI

ChatGLM-6B

ChatGLM-6B: An Open Bilingual Dialogue Language Model

GitHub: https://github.com/THUDM/ChatGLM-6B

Moss

Moss: An open-source tool-augmented conversational language model from Fudan University GitHub: https://github.com/OpenLMLab/MOSS

LLaMA

LLaMA: Inference code for LLaMA models

GitHub: https://github.com/facebookresearch/llama

Alpaca

Alpaca: The current Alpaca model is fine-tuned from a 7B LLaMA model on 52K instruction-following data generated by the techniques in the Self-Instruct paper

GitHub: https://github.com/tatsu-lab/stanford_alpaca

BELLE

BELLE: Be Everyone's Large Language model Engine GitHub: https://github.com/LianjiaTech/BELLE

Vicuna

The release repo for "Vicuna: An Open Chatbot Impressing GPT-4"

GitHub: https://github.com/lm-sys/FastChat

Dolly

Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform

GitHub: https://github.com/databrickslabs/dolly

OpenAssistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

GitHub: https://github.com/LAION-AI/Open-Assistant

LLM Zoo

LLM Zoo: democratizing ChatGPT

GitHub: https://github.com/FreedomIntelligence/LLMZoo

Chinese-LLaMA-Alpaca

Chinese LLaMA & Alpaca LLMs

GitHub: https://github.com/ymcui/Chinese-LLaMA-Alpaca

BayLing

BayLing: Bridging Cross-lingual Alignment and Instruction Following through Interactive Translation for Large Language Models

GitHub: https://github.com/ictnlp/BayLing

Open-source Instruction Data

PromptSource

PromptSource is a toolkit for creating, sharing and using natural language prompts. [2k][English]

Link: https://github.com/bigscience-workshop/promptsource

T0/P3

Multitask Prompted Training Enables Zero-Shot Task Generalization[2k][English]

Link: https://github.com/bigscience-workshop/t-zero

xP3

Crosslingual Generalization through Multitask Finetuning [Multilingual] [NMT]

Link: https://github.com/bigscience-workshop/xmtf

Super-Natural-Instruct v2

SUPER-NATURALINSTRUCTIONS: Generalization via Declarative Instructions on 1600+ NLP Tasks [1.6k][Multilingual]

Link: https://instructions.apps.allenai.org/

FLAN

Finetuned Language Models are Zero-Shot Learners/Flan Collection [18k][English]

Link: https://github.com/google-research/flan

CrossFit

The CrossFit Challenge 🏋️ and The NLP Few-shot Gym 💦 [English]

Link: https://github.com/INK-USC/CrossFit

Self-Instruct

Self-Instruct: Aligning Language Model with Self Generated Instructions [52K] [English] [Generated]

Link: https://github.com/yizhongw/self-instruct

Unnatural Instructions

Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor [240K] [English] [Generated]

Link: https://github.com/orhonovich/unnatural-instructions

Stanford Alpaca

Stanford Alpaca: An Instruction-following LLaMA Model [51.9K] [English] [Generated]

Link: https://github.com/tatsu-lab/stanford_alpaca

Camel

CAMEL: Communicative Agents for “Mind” Exploration of Large Scale Language Model Society [115K] [English] [Generated]

Link: https://github.com/lightaime/camel

Dolly

The training data on which dolly-v2-12b is instruction tuned represents natural language instructions generated by Databricks employees [15K] [English]

Link: https://github.com/databrickslabs/dolly

GuanacoDataset

The dataset for the Guanaco model is designed to enhance the multilingual capabilities and address various linguistic tasks. [534K] [Multilingual]

Link: https://huggingface.co/datasets/JosephusCheung/GuanacoDataset

Chinese-ChatLLaMA

本项目向社区提供中文对话模型 ChatLLama 、中文基础模型 LLaMA-zh 及其训练数据。 [Multilingual]

Link: https://github.com/ydli-ai/Chinese-ChatLLaMA

OIG

Open Instruction Generalist (OIG) [43M] [English]

Link: https://laion.ai/blog/oig-dataset/

GPTeacher

GPTeacher: A collection of modular datasets generated by GPT-4, General-Instruct - Roleplay-Instruct - Code-Instruct - and Toolformer [English] [Generated]

Link: https://github.com/teknium1/GPTeacher

CSL

Chinese Scientific Literature Dataset [396K] [Chinese]

Link: https://github.com/ydli-ai/CSL

GLM-130B

GLM-130B: An Open Bilingual Pre-Trained Model [Multilingual (eng, zh)]

Link: https://github.com/THUDM/GLM-130B

Firefly

Firefly(流萤): 中文对话式大语言模型 [1.1M] [Chinese]

Link: https://github.com/yangjianxin1/Firefly

BELLE

BELLE: Be Everyone's Large Language model Engine [1.5M] [Chinese]

Link: https://github.com/LianjiaTech/BELLE

Chinese-Vicuna

Chinese-Vicuna: A Chinese Instruction-following LLaMA-based Model —— 一个中文低资源的llama+lora方案 [1M] [Chinese] Link: https://github.com/Facico/Chinese-Vicuna

HC3

How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection [37k] [Multilingual]

Link: https://github.com/Hello-SimpleAI/chatgpt-comparison-detection

Luotuo

骆驼(Luotuo): Chinese-alpaca-lora [51.6K] [Chinese]

Link: https://github.com/LC1332/Chinese-alpaca-lora

COIG

Chinese Open Instruction Generalist: A Preliminary Release [Chinese]

Link: https://github.com/BAAI-Zlab/COIG

ShareGPT52K

This dataset is a collection of approximately 52,000 90,000 conversations scraped via the ShareGPT API before it was shut down. These conversations include both user prompts and responses from OpenAI's ChatGPT. [90K] [Multiligual]

Link: https://huggingface.co/datasets/RyokoAI/ShareGPT52K

Open-source RLHF Data

Stack-Exchange-Preferences

Huggingface H4 Stack Exchange Preferences Dataset [10M] [English]

Link: https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences

HH-RLHF

Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback [169K] [English]

Link: https://github.com/anthropics/hh-rlhf

SHP

Stanford Human Preferences Dataset (SHP) [385K] [English]

Link: https://huggingface.co/datasets/stanfordnlp/SHP

OASST

OpenAssistant Conversations -- Democratizing Large Language Model Alignment [161K] [Multilingual]

Link: https://huggingface.co/datasets/OpenAssistant/oasst1

GPT4All

GPT4All: Training an Assistant-style Chatbot with Large Scale Data Distillation from GPT-3.5-Turbo [165K] [Multilingual]

Link: https://github.com/nomic-ai/gpt4all

InstructWild

Instruction in the Wild: A User-based Instruction Dataset [104K] [Multilingual] [Generated]

Link: https://github.com/XueFuzhao/InstructionWild

Papers

Survey

1. A Survey of Large Language Models. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, Ji-Rong Wen
2. A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT. Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, Hao Peng, Jianxin Li, Jia Wu, Ziwei Liu, Pengtao Xie, Caiming Xiong, Jian Pei, Philip S. Yu, Lichao Sun
3. A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT. Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S. Yu, Lichao Sun
4. ChatGPT is not all you need. A State of the Art Review of large Generative AI models. Roberto Gozalo-Brizuela, Eduardo C. Garrido-Merchan
5. ChatGPT: Applications, Opportunities, and Threats. Aram Bahrini, Mohammadsadra Khamoshifar, Hossein Abbasimehr, Robert J. Riggs, Maryam Esmaeili, Rastin Mastali Majdabadkohne, Morteza Pasehvar

Machine Translation

1. Exploring Human-Like Translation Strategy with Large Language Models . Zhiwei He, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang, Rui Wang, Zhaopeng Tu, Shuming Shi, Xing Wang
2. Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis. Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Jiajun Chen, Lei Li, Shujian Huang.
3. ParroT: Translating During Chat Using Large Language Models. Wenxiang Jiao, Jen-tse Huang, Wenxuan Wang, Xing Wang, Shuming Shi, Zhaopeng Tu
4. Document-Level Machine Translation with Large Language Models. Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, Zhaopeng Tu
5. Unleashing the Power of ChatGPT for Translation: An Empirical Study. Yuan Gao, Ruili Wang, Feng Hou
6. Linguistically Informed ChatGPT Prompts to Enhance Japanese-Chinese Machine Translation: A Case Study on Attributive Clauses. Wenshi Gu
7. Towards Making the Most of ChatGPT for Machine Translation. Keqin Peng, Liang Ding, Qihuang Zhong, Li Shen, Xuebo Liu, Min Zhang, Yuanxin Ouyang, Dacheng Tao
8. How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation. Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, Hany Hassan Awadalla
9. Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine. Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, Zhaopeng Tu
10. How to Design Translation Prompts for ChatGPT: An Empirical Study. Yuan Gao, Ruili Wang, Feng Hou

Sentiment Analysis

1. Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study. Zengzhi Wang, Qiming Xie, Zixiang Ding, Yi Feng, Rui Xia
2. Investigating Chain-of-thought with ChatGPT for Stance Detection on Social Media. Bowen Zhang, Xianghua Fu, Daijun Ding, Hu Huang, Yangyang Li, Liwen Jing
3. How would Stance Detection Techniques Evolve after the Launch of ChatGPT? Bowen Zhang, Daijun Ding, Liwen Jing
4. Is ChatGPT Equipped with Emotional Dialogue Capabilities? Weixiang Zhao, Yanyan Zhao, Xin Lu, Shilong Wang, Yanpeng Tong, Bing Qin

Multi-Lingual

1.Phoenix: Democratizing ChatGPT across Languages. Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang, Juhao Liang, Chen Zhang, Zhiyi Zhang, Jianquan Li, Xiang Wan, Benyou Wang, Haizhou Li

Dialogue

1. A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding. Wenbo Pan, Qiguang Chen, Xiao Xu, Wanxiang Che, Libo Qin.
2. Language-Driven Representation Learning for Robotics. Siddharth Karamcheti, Suraj Nair,Annie Chen, Thomas Kollar, Chelsea Finn, Dorsa Sadigh, Percy Liang

Summarization

1. Extractive Summarization via ChatGPT for Faithful Summary Generation. Haopeng Zhang, Xiao Liu, Jiawei Zhang
2. Human-like Summarization Evaluation with ChatGPT. Mingqi Gao, Jie Ruan, Renliang Sun, Xunjian Yin, Shiping Yang, Xiaojun Wan
3. ChatGPT as a Factual Inconsistency Evaluator for Abstractive Text Summarization. Zheheng Luo, Qianqian Xie, Sophia Ananiadou
4. Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization. Xianjun Yang, Yan Li, Xinlu Zhang, Haifeng Chen, Wei Cheng
5. Cross-Lingual Summarization via ChatGPT. Jiaan Wang, Yunlong Liang, Fandong Meng, Zhixu Li, Jianfeng Qu, Jie Zhou

Robot

1. ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application. Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi, Jun Takamatsu, Katsushi Ikeuchi

Logical Reasoning

1. Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4. Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, Yue Zhang

Medical AI

1. On the Evaluations of ChatGPT and Emotion-enhanced Prompting for Mental Health Analysis. Kailai Yang, Shaoxiong Ji, Tianlin Zhang, Qianqian Xie, Sophia Ananiadou
2. DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task. Honglin Xiong, Sheng Wang, Yitao Zhu, Zihao Zhao, Yuxiao Liu, Qian Wang, Dinggang Shen
3. Zero-shot Clinical Entity Recognition using ChatGPT. Yan Hu, Iqra Ameer, Xu Zuo, Xueqing Peng, Yujia Zhou, Zehan Li, Yiming Li, Jianfu Li, Xiaoqian Jiang, Hua Xu
4. Evaluation of ChatGPT for NLP-based Mental Health Applications. Bishal Lamichhane
5. ChatDoctor: A Medical Chat Model Fine-tuned on LLaMA Model using Medical Domain Knowledge. Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, You Zhang
6. DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4. Zhengliang Liu, Xiaowei Yu, Lu Zhang, Zihao Wu, Chao Cao, Haixing Dai, Lin Zhao, Wei Liu, Dinggang Shen, Quanzheng Li, Tianming Liu, Dajiang Zhu, Xiang Li
7. Exploring the Cognitive Dynamics of Artificial Intelligence in the Post-COVID-19 and Learning 3.0 Era: A Case Study of ChatGPT. Lingfei Luan, Xi Lin, Wenbiao Li
8. HuaTuo (华驼): Tuning LLaMA Model with Chinese Medical Knowledge. Haochun Wang , Chi Liu, Nuwa Xi, Zewen Qiang, Sendong Zhao, Bing Qin and Ting Liu

Commonsense

1. ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models. Ning Bian, Xianpei Han, Le Sun, Hongyu Lin, Yaojie Lu, Ben He

Grammatical Error Correction

1. Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation. Tao Fang, Shu Yang, Kaixin Lan, Derek F. Wong, Jinpeng Hu, Lidia S. Chao, Yue Zhang
2. ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark. Haoran Wu, Wenxuan Wang, Yuxuan Wan, Wenxiang Jiao, Michael Lyu

Text-to-SQL

1. A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability. Aiwei Liu, Xuming Hu, Lijie Wen, Philip S. Yu

Question Answering

1. Evaluation of ChatGPT as a Question Answering System for Answering Complex Questions. Yiming Tan, Dehai Min, Yu Li, Wenbo Li, Nan Hu, Yongrui Chen, Guilin Qi

Keyphrase Generator

1. Is ChatGPT A Good Keyphrase Generator? A Preliminary Study. Mingyang Song, Haiyun Jiang, Shuming Shi, Songfang Yao, Shilong Lu, Yi Feng, Huafeng Liu, Liping Jing

Code Intelligence

1. Self-collaboration Code Generation via ChatGPT. Yihong Dong, Xue Jiang, Zhi Jin, Ge Li
2. How Secure is Code Generated by ChatGPT? Raphaël Khoury, Anderson R. Avila, Jacob Brunelle, Baba Mamadou Camara

NLG

1. Is ChatGPT a Good NLG Evaluator? A Preliminary Study. Jiaan Wang, Yunlong Liang, Fandong Meng, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, Jie Zhou

Event Extraction

1. Exploring the Feasibility of ChatGPT for Event Extraction. Jun Gao, Huan Zhao, Changlong Yu, Ruifeng Xu
2. Zero-Shot Information Extraction via Chatting with ChatGPT. Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, Yong Jiang, Wenjuan Han

Information Extraction

1. Code4Struct: Code Generation for Few-Shot Structured Prediction from Natural Language. Xingyao Wang, Sha Li, Heng Ji
2. Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples! Yubo Ma, Yixin Cao, YongChing Hong, Aixin Sun
3. Thinking about GPT-3 in-context learning for biomedical IE? Bernal Jiménez Gutiérrez, Nikolas McNeal, Clay Washington, You Chen, Lang Li, Huan Sun, Yu Su
4. Yes but.. Can ChatGPT Identify Entities in Historical Documents? Carlos-Emiliano González-Gallardo, Emanuela Boros, Nancy Girdhar, Ahmed Hamdi, Jose G. Moreno, Antoine Doucet

Data Augmentation

1. AugGPT: Leveraging ChatGPT for Text Data Augmentation. Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Yihan Cao, Zihao Wu, Lin Zhao, Shaochen Xu, Wei Liu, Ninghao Liu, Sheng Li, Dajiang Zhu, Hongmin Cai, Lichao Sun, Quanzheng Li, Dinggang Shen, Tianming Liu, Xiang Li
2. Testing the Reliability of ChatGPT for Text Annotation and Classification: A Cautionary Remark Michael V. Reiss
3. Is a prompt and a few samples all you need? Using GPT-4 for data augmentation in low-resource classification tasks. Anders Giovanni Møller, Jacob Aarup Dalsgaard, Arianna Pera, Luca Maria Aiello

Keyphrase Generation

1. ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Task. Roberto Martínez-Cruz, Alvaro J. López-López, José Portela

Industrial Engineering

1. Industrial Engineering with Large Language Models: A case study of ChatGPT's performance on Oil & Gas problems. Oluwatosin Ogundare, Srinath Madasu, Nathanial Wiggins

Mathematical Word Problem

1. An Independent Evaluation of ChatGPT on Mathematical Word Problems (MWP). Paulo Shakarian, Abhinav Koyyalamudi, Noel Ngu, Lakshmivihari Mareedu

Recommendation

1. Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System. Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, Jiawei Zhang

2. Is ChatGPT a Good Recommender? A Preliminary Study. Junling Liu, Chao Liu, Renjie Lv, Kang Zhou, Yan Zhang

3. Uncovering ChatGPT’s Capabilities in Recommender Systems. Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongxiang Sun, Xiao Zhang, Jun Xu

Safety

1. The Capacity for Moral Self-Correction in Large Language Models. Deep Ganguli , Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamile Lukošiute, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, Dawn Drain, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jackson Kernion, Jamie Kerr, Jared Mueller, Joshua Landau, Kamal Ndousse, Karina Nguyen, Liane Lovitt, Michael Sellitto, Nelson Elhage, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robert Lasenby, Robin Larson, Sam Ringer, Sandipan Kundu, Saurav Kadavath, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, Christopher Olah, Jack Clark, Samuel R. Bowman, Jared Kaplan
2. Toxicity in ChatGPT: Analyzing Persona-assigned Language Models. Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan

Application

1. Tool Learning with Foundation Models. Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, Yi Ren Fung, Yusheng Su, Huadong Wang, Cheng Qian, Runchu Tian, Kunlun Zhu, Shihao Liang, Xingyu Shen, Bokai Xu, Zhen Zhang, Yining Ye, Bowen Li, Ziwei Tang, Jing Yi, Yuzhang Zhu, Zhenning Dai, Lan Yan, Xin Cong, Yaxi Lu, Weilin Zhao, Yuxiang Huang, Junxi Yan, Xu Han, Xian Sun, Dahai Li, Jason Phang, Cheng Yang, Tongshuang Wu, Heng Ji, Zhiyuan Liu, Maosong Sun.
2. Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models. Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, Nan Duan

AGI

1. Sparks of Artificial General Intelligence: Early experiments with GPT-4. Sebastien Bubeck Varun Chandrasekaran Ronen Eldan Johannes Gehrke Eric Horvitz Ece Kamar Peter Lee Yin Tat Lee Yuanzhi Li Scott Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang

Analysis, Challenge and Future Work

1. Comparative Analysis of CHATGPT and the evolution of language models. Oluwatosin Ogundare, Gustavo Quiros Araya

2. Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing. Walid Hariri

3. Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models. Yiheng Liu, Tianle Han, Siyuan Ma, Jiayue Zhang, Yuanyuan Yang, Jiaming Tian, Hao He, Antong Li, Mengshen He, Zhengliang Liu, Zihao Wu, Dajiang Zhu, Xiang Li, Ning Qiang, Dingang Shen, Tianming Liu, Bao Ge

4. Can we trust the evaluation on ChatGPT? Rachith Aiyappa, Jisun An, Haewoon Kwak, Yong-Yeol Ahn

5. A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need? Chaoning Zhang, Chenshuang Zhang, Sheng Zheng, Yu Qiao, Chenghao Li, Mengchun Zhang, Sumit Kumar Dam, Chu Myaet Thwal, Ye Lin Tun, Le Luang Huy, Donguk kim, Sung-Ho Bae, Lik-Hang Lee, Yang Yang, Heng Tao Shen, In So Kweon, Choong Seon Hong

6. A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models. Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, Jie Zhou, Siming Chen, Tao Gui, Qi Zhang, Xuanjing Huang

7. ChatGPT: A Meta-Analysis after 2.5 Months. Christoph Leiter, Ran Zhang, Yanran Chen, Jonas Belouadi, Daniil Larionov, Vivian Fresen, Steffen Eger

8. On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective. Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang, Wei Ye, Xiubo Geng, Binxin Jiao, Yue Zhang, Xing Xie

9. Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT. Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, Dacheng Tao

10. Is ChatGPT a General-Purpose Natural Language Processing Task Solver? Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, Diyi Yang

11. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, Pascale Fung

12. A Categorical Archive of ChatGPT Failures. Ali Borji

13. ChatGPT and Software Testing Education: Promises & Perils. Sajed Jalil, Suzzana Rafi, Thomas D. LaToza, Kevin Moran, Wing Lam

14. Exploring AI Ethics of ChatGPT: A Diagnostic Analysis. Terry Yue Zhuo, Yujin Huang, Chunyang Chen, Zhenchang Xing

15. How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, Yupeng Wu

16. Can ChatGPT and Bard Generate Aligned Assessment Items? A Reliability Analysis against Human Performance. Abdolvahab Khademi

17. Large Language Models Can Be Easily Distracted by Irrelevant Context Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Scharli, Denny Zhou

18. GPT as Knowledge Worker: A Zero-Shot Evaluation of (AI)CPA Capabilities. Jillian Bommarito, Michael J Bommarito II, Jessica Katz, Daniel Martin Katz

19. Exploring ChatGPT's Ability to Rank Content: A Preliminary Study on Consistency with Human Preferences. Yunjie Ji, Yan Gong, Yiping Peng, Chao Ni, Peiyan Sun, Dongyu Pan, Baochang Ma*, Xiangang Li

20. ChatGPT versus Traditional Question Answering for Knowledge Graphs: Current Status and Future Directions Towards Knowledge Graph Chatbots. Reham Omar, Omij Mangukiya, Panos Kalnis, Essam Mansour

21. Conversational Process Modelling: State of the Art, Applications, and Implications in Practice. Nataliia Klievtsova, Janik-Vasily Benzin, Timotheus Kampik, Juergen Mangler, Stefanie Rinderle-Ma

22. Can ChatGPT-like Generative Models Guarantee Factual Accuracy? On the Mistakes of New Generation Search Engines. Ruochen Zhao, Xingxuan Li, Yew Ken Chia, Bosheng Ding, Lidong Bing

23. ChatLog: Recording and Analyzing ChatGPT Across Time. Shangqing Tu, Chunyang Li, Jifan Yu, Xiaozhi Wang, Lei Hou, Juanzi Li

24. The Dark Side of ChatGPT: Legal and Ethical Challenges from Stochastic Parrots and Hallucination. Zihao Li