Continuously updated list of related resources for generative LLMs like GPT and their analysis and detection.
231
217 commits
updated May 28, 2025
Note: #params > 1B only
OpenAI
(GPT-2) Language Models are Unsupervised Multitask Learners. [paper] [blog] 
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever.
(GPT-3) Language Models are Few-Shot Learners. [paper] 
WebGPT: Browser-assisted question-answering with human feedback. [paper] [blog] 
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess and John Schulman.
(InstructGPT) Training language models to follow instructions with human feedback. [paper] [blog]
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike and Ryan Lowe.
ChatGPT: Optimizing Language Models for Dialogue. [blog]
(GPT-4) GPT-4 Technical Report. [paper] [blog] [blog]
OpenAI.
DeepMind
(Gopher) Scaling Language Models: Methods, Analysis & Insights from Training Gopher. [paper] [blog]
(RETRO) Improving language models by retrieving from trillions of tokens. [paper] [blog]
(Chinchilla) Training Compute-Optimal Large Language Models. [paper] [blog]
(Sparrow) Improving alignment of dialogue agents via targeted human judgements. [paper] [blog]
Nvidia & Microsoft
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. [paper]
Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper and Bryan Catanzaro.
Turing-NLG: A 17-billion-parameter language model by Microsoft. [blog]
(Megatron-Turing NLG) Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model. [paper] [blog] [blog]
(Meena) Towards a Human-like Open-Domain Chatbot. [paper] [blog]
Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu and Quoc V. Le.
(T5) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. [paper] [blog] 
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li and Peter J. Liu.
mT5: A massively multilingual pre-trained text-to-text transformer. [paper] 
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua and Colin Raffel.
(FLAN) Finetuned Language Models Are Zero-Shot Learners. [paper] [blog] 
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai and Quoc V. Le.
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts. [paper] [slides] [blog] 
LaMDA: Language Models for Dialog Applications. [paper] [blog]
PaLM: Scaling Language Modeling with Pathways. [paper] [blog]
UL2: Unifying Language Learning Paradigms. [paper] [blog]
Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Denny Zhou, Neil Houlsby and Donald Metzler.
(Flan-T5 & Flan-PaLM & Flan-U-PaLM) Scaling Instruction-Finetuned Language Models. [paper] [blog]
Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Denny Zhou, Neil Houlsby and Donald Metzler.
Meta
(Generative BST) Recipes for Building an Open-Domain Chatbot. [paper] 
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan Boureau and Jason Weston.
(XGLM) Few-shot Learning with Multilingual Language Models. [paper] 
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O'Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov and Xian Li.
OPT: Open Pre-trained Transformer Language Models. [paper]
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang and Luke Zettlemoyer.
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage. [paper] [blog]
Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, Morteza Behrooz, William Ngan, Spencer Poff, Naman Goyal, Arthur Szlam, Y-Lan Boureau, Melanie Kambadur and Jason Weston.
Atlas: Few-shot Learning with Retrieval Augmented Language Models. [paper]
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel and Edouard Grave.
LLaMA: Open and Efficient Foundation Language Models. [paper] [blog]
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave and Guillaume Lample.
BigScience
(T0++) Multitask Prompted Training Enables Zero-Shot Task Generalization. [paper]
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. [paper]
Amazon
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model. [paper] [blog]
Saleh Soltan, Shankar Ananthakrishnan, Jack FitzGerald, Rahul Gupta, Wael Hamza, Haidar Khan, Charith Peris, Stephen Rawls, Andy Rosenbaum, Anna Rumshisky, Chandana Satya Prakash, Mukund Sridhar, Fabian Triefenbach, Apurv Verma, Gokhan Tur and Prem Natarajan.
AI21 Labs
Baidu
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation. [paper]
Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen, Yanbin Zhao, Yuxiang Lu, Weixin Liu, Zhihua Wu, Weibao Gong, Jianzhong Liang, Zhizhou Shang, Peng Sun, Wei Liu, Xuan Ouyang, Dianhai Yu, Hao Tian, Hua Wu and Haifeng Wang.
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation. [paper]
PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation. [paper] 
Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhihua Wu, Zhen Guo, Hua Lu, Xinxian Huang, Xin Tian, Xinchao Xu, Yingzhan Lin and Zheng-Yu Niu.
ERNIE Bot [news]
Anthropic
(Anthropic-52) Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback [paper]
(Claude) Constitutional AI: Harmlessness from AI Feedback. [paper]
EleutherAI
GPT-J-6B: 6B JAX-Based Transformer. [blog]
GPT-NeoX-20B: An Open-Source Autoregressive Language Model. [paper] [blog] 
Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, Samuel Weinbach.
Tsinghua University
CPM: A Large-scale Generative Chinese Pre-trained Language Model. [paper] 
Zhengyan Zhang, Xu Han, Hao Zhou, Pei Ke, Yuxian Gu, Deming Ye, Yujia Qin, Yusheng Su, Haozhe Ji, Jian Guan, Fanchao Qi, Xiaozhi Wang, Yanan Zheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen, Daixuan Li, Zhenbo Sun, Zhiyuan Liu, Minlie Huang, Wentao Han, Jie Tang, Juanzi Li, Xiaoyan Zhu, Maosong Sun.
CPM-2: Large-scale Cost-effective Pre-trained Language Models. [paper]
Zhengyan Zhang, Yuxian Gu, Xu Han, Shengqi Chen, Chaojun Xiao, Zhenbo Sun, Yuan Yao, Fanchao Qi, Jian Guan, Pei Ke, Yanzheng Cai, Guoyang Zeng, Zhixing Tan, Zhiyuan Liu, Minlie Huang, Wentao Han, Yang Liu, Xiaoyan Zhu, Maosong Sun.
Chinese-Transformer-XL [repo]
EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training. [paper]
Hao Zhou, Pei Ke, Zheng Zhang, Yuxian Gu, Yinhe Zheng, Chujie Zheng, Yida Wang, Chen Henry Wu, Hao Sun, Xiaocong Yang, Bosi Wen, Xiaoyan Zhu, Minlie Huang, Jie Tang.
EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training. [paper]
Yuxian Gu, Jiaxin Wen, Hao Sun, Yi Song, Pei Ke, Chujie Zheng, Zheng Zhang, Jianzhu Yao, Xiaoyan Zhu, Jie Tang, Minlie Huang.
GLM: General Language Model Pretraining with Autoregressive Blank Infilling. [paper] 
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, Jie Tang.
GLM-130B: An Open Bilingual Pre-trained Model. [paper] 
Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Zhiyuan Liu, Peng Zhang, Yuxiao Dong and Jie Tang.
ChatGLM. [blog]
IDEA
Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence. [paper] [blog]
Junjie Wang, Yuxiang Zhang, Lin Zhang, Ping Yang, Xinyu Gao, Ziwei Wu, Xiaoqun Dong, Junqing He, Jianheng Zhuo, Qi Yang, Yongfeng Huang, Xiayu Li, Yanghan Wu, Junyu Lu, Xinyu Zhu, Weifeng Chen, Ting Han, Kunhao Pan, Rui Wang, Hao Wang, Xiaojun Wu, Zhongshen Zeng, Chongpei Chen, Ruyi Gan and Jiaxing Zhang.
Huawei
PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation. [paper]
PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained Language Model. [paper]
Fei Mi, Yitong Li, Yulong Zeng, Jingyan Zhou, Yasheng Wang, Chuanfei Xu, Lifeng Shang, Xin Jiang, Shiqi Zhao and Qun Liu.
Inspur
WeChat AI
WeLM: A Well-Read Pre-trained Language Model for Chinese. [paper]
Hui Su, Xiao Zhou, Houjin Yu, Yuwen Chen, Zilin Zhu, Yang Yu and Jie Zhou.
Holistic Evaluation of Language Models. [paper] [blog]
Benchmark 30 prominent language models across a wide range of scenarios and for a broad range of metrics to elucidate their capabilities and risks.
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 [paper] 
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and Shmargaret Shmitchell.
Identified a wide variety of costs and risks associated with the rush for ever larger LMs, including: environmental costs, financial costs, opportunity cost and the risk of substantial harms.
TruthfulQA: Measuring How Models Mimic Human Falsehoods. [paper]
Stephanie Lin, Jacob Hilton and Owain Evans.
Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful.
How Much Knowledge Can You Pack Into the Parameters of a Language Model? [paper] 
Adam Roberts, Colin Raffel and Noam Shazeer.
The maximum-likelihood objective used to train our model provides no guarantees as to whether a model will learn a fact or not. This makes it difficult to ensure that the model obtains specific knowledge over the course of pre-training and prevents us from explicitly updating or removing knowledge from a pre-trained model.
On the Opportunities and Risks of Foundation Models. [paper]
This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities and technical principles to their applications and societal impact.
Survey of Hallucination in Natural Language Generation. [paper] 
Benchmark 30 prominent language models across a wide range of scenarios and for a broad range of metrics to elucidate their capabilities and risks.
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? [paper] 
Nouha Dziri, Sivan Milton, Mo Yu, Osmar Zaiane and Siva Reddy.
Reveals that the standard benchmarks consist of >60% hallucinated responses, leading to models that not only hallucinate but even amplify hallucinations..
Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations. [paper]
Josh A. Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel and Katerina Sedova.
This report aims to assess: how might language models change influence operations, and what steps can be taken to mitigate these threats? This task is inherently speculative, as both AI and influence operations are changing quickly.
Forecasting Potential Misuses of Language Models for Disinformation Campaigns—and How to Reduce Risk. [blog]
OpenAI researchers collaborated with Georgetown University’s Center for Security and Emerging Technology and the Stanford Internet Observatory to investigate how large language models might be misused for disinformation purposes.
ChatGPT Wrote a Terrible Gizmodo Article. [blog]
ChatGPT kept including incorrect information in its explainer—sometimes mixing up basic facts about the history of its own technology.
Temporary policy: ChatGPT is banned. [blog]
While the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce.
On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning. [paper]
Omar Shaikh, Hongxin Zhang, William Held, Michael Bernstein and Diyi Yang.
Find that using zero-shot CoT reasoning in a prompt can significantly increase a model’s likelihood to produce undesirable output.
The Woman Worked as a Babysitter: On Biases in Language Generation. [paper] 
Emily Sheng, Kai-Wei Chang, Premkumar Natarajan and Nanyun Peng.
A systematic study of biases in natural language generation (NLG) by analyzing text generated from prompts that contain mentions of different demographic groups.
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. [paper] 
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi and Noah A. Smith.
Using RealToxicityPrompts, we find that pretrained LMs can degenerate into toxic text even from seemingly innocuous prompts.
OpenAI's new ChatGPT bot: 10 dangerous things it's capable of. [blog]
As the erudite machinery turns into a viral sensation, humans have started to discover some of the AI's biases, like the desire to wipe out humanity.
OpwnAI: AI That Can Save the Day or HACK it Away. [blog]
AI models can be used to create a full infection flow, from spear-phishing to running a reverse shell.
OpwnAI : Cybercriminals Starting to Use ChatGPT. [blog]
There are already instances of cybercriminals using OpenAI to develop malicious tools.
Security risks of ChatGPT and other AI text generators. [blog]
Successfully asked ChatGPT to write a convincing phishing email and to create JavaScript that could be used to steal personal information.
The security threat of AI-powered cyberattacks. [paper]
Matti Aksela, Samuel Marchal, Andrew Patel, Lina Rosenstedt and WithSecure.
Investigate the security threat of AI-enabled cyberattacks by summarising current knowledge on the topic.
How hackers might be exploiting ChatGPT. [blog]
The viral AI chatbot ChatGPT might advise threat actors how to hack into networks with ease.
Extracting Training Data from Large Language Models. [paper]
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea and Colin Raffel.
This paper demonstrates that an adversary can perform a training data extraction attack to recover individual training examples by querying the language model.
Ignore Previous Prompt: Attack Techniques For Language Models. [paper] 
Fábio Perez and Ian Ribeiro.
Study prompt injection attacks against LLMs and propose a framework to explore such attacks; Investigate two specific attacks: goal hijacking and prompt leaking.
Carbon Emissions and Large Neural Network Training. [paper]
David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier and Jeff Dean.
Calculate the energy use and carbon footprint of several recent large models-T5, Meena, GShard, Switch Transformer, and GPT-3.
Automatic Detection of Machine Generated Text: A Critical Survey. [paper] 
Ganesh Jawahar, Muhammad Abdul-Mageed and Laks V.S. Lakshmanan.
在线社交网络文本内容对抗技术. [paper] 
刘晓明, 张兆晗, 杨晨阳, 张宇辰, 沈超, 周亚东 and 管晓宏.
Synthetic Text Detection: Systemic Literature Review. [paper]
Jesus Guerrero, Izzat Alsmadi.
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods. [paper]
Evan Crothers, Nathalie Japkowicz and Herna Viktor.
Deepfake Text Detection: Limitations and Opportunities. [paper] 
Jiameng Pu, Zain Sarwar, Sifat Muhammad Abdullah, Abdullah Rehman, Yoonjin Kim, Parantapa Bhattacharya, Mobin Javed and Bimal Viswanath.
The Science of Detecting LLM-Generated Texts. [paper]
Ruixiang Tang, Yu-Neng Chuang and Xia Hu.
To ChatGPT, or not to ChatGPT: That is the question! [paper]
Alessandro Pegoraro, Kavita Kumari, Hossein Fereidooni and Ahmad-Reza Sadeghi.
The Age of Synthetic Realities: Challenges and Opportunities. [paper]
João Phillipe Cardenuto, Jing Yang, Rafael Padilha, Renjie Wan, Daniel Moreira, Haoliang Li, Shiqi Wang, Fernanda Andaló, Sébastien Marcel and Anderson Rocha.
Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective. [paper] 
Adaku Uchendu, Thai Le and Dongwon Lee.
Detecting Artificial Intelligence: A New Cyberarms Race Begins. [paper] 
Mark Campbell and Mlađan Jovanović.
Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text. [paper] 
Mahdi Dhaini, Wessel Poelman and Ege Erdogan.
A Survey on LLM-gernerated Text Detection: Necessity, Methods, and Future Directions. [paper]
Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F. Wong and Lidia S. Chao.
Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey. [paper]
Soumya Suvra Ghosal, Souradip Chakraborty, Jonas Geiping, Furong Huang, Dinesh Manocha and Amrit Singh Bedi.
A Survey on Detection of LLMs-Generated Content. [paper]
Xianjun Yang, Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Petzold, William Yang Wang and Wei Cheng.
A Survey of Text Watermarking in the Era of Large Language Models. [paper]
Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Lijie Wen, Irwin King and Philip S. Yu.
Detection of Machine-Generated Text: Literature Survey. [paper]
Dmytro Valiaiev.
Detecting Multimedia Generated by Large AI Models: A Survey. [paper]
Li Lin, Neeraj Gupta, Yue Zhang, Hainan Ren, Chun-Hao Liu, Feng Ding, Xin Wang, Xin Li, Luisa Verdoliva and Shu Hu.
Copyright Protection in Generative AI: A Technical Perspective. [paper]
Jie Ren, Han Xu, Pengfei He, Yingqian Cui, Shenglai Zeng, Jiankun Zhang, Hongzhi Wen, Jiayuan Ding, Hui Liu, Yi Chang and Jiliang Tang.
Beyond the Human Eye: Comprehensive Approaches to AI Text Detection. [paper] 
Jamal Goddard, Yuksel Celik and Sanjay Goel.
A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterization. [paper]
Tharindu Kumarage, Garima Agrawal, Paras Sheth, Raha Moraffah, Aman Chadha, Joshua Garland and Huan Liu.
Decoding the AI Pen: Techniques and Challenges in Detecting AI-Generated Text. [paper]
Sara Abdali, Richard Anarfi, CJ Barberan and Jia He.
Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods. [paper]
Kathleen C. Fraser, Hillary Dawkins and Svetlana Kiritchenko.
From Intentions to Techniques: A Comprehensive Taxonomy and Challenges in Text Watermarking for Large Language Models. [paper]
Harsh Nishant Lalai, Aashish Anantha Ramakrishnan, Raj Sanjay Shah and Dongwon Lee.
Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integrity. [paper]
Shushanta Pudasaini, Luis Miralles-Pechuán, David Lillis and Marisa Llorens Salvador.
Survey for Detecting AI-generated Content. [paper] 
Yu Wang, Ziyan Wang.
Building Intelligence Identification System via Large Language Model Watermarking: A Survey and Beyond. [paper]
Xuhong Wang, Haoyu Jiang, Yi Yu, Jingru Yu, Yilun Lin, Ping Yi, Yingchun Wang, Yu Qiao, Li Li and Fei-Yue Wang.
Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges. [paper]
Baixiang Huang, Canyu Chen and Kai Shu.
SoK: Watermarking for AI-Generated Content. [paper]
Xuandong Zhao, Sam Gunn, Miranda Christ, Jaiden Fairoze, Andres Fabrega, Nicholas Carlini, Sanjam Garg, Sanghyun Hong, Milad Nasr, Florian Tramer, Somesh Jha, Lei Li, Yu-Xiang Wang and Dawn Song.
Deepfake Bot Submissions to Federal Public Comment Websites Cannot Be Distinguished from Human Submissions. [paper] 
Max Weiss.
Generate and submit 1,001 deepfake comments regarding a Medicaid reform waiver to a federal public comment website. The human classification results are no better than would have been gotten by random guessing.
RoFT: A Tool for Evaluating Human Detection of Machine-Generated Text. [paper]
Liam Dugan, Daphne Ippolito, Arun Kirubarajan and Chris Callison-Burch.
Develop the Real or Fake Text (RoFT) system, a novel application for simultaneously collecting quality annotations of machinegenerated text while allowing the public to assess and improve their skill at detecting machinegenerated text.
All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text. [paper] 
Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan and Noah A. Smith.
Assess nonexperts’ ability to distinguish between humanand machine-authored text, and test three evaluator-training methods to see if we could improve people’s ability to identify machinegenerated text.
Human Heuristics for AI-Generated Language Are Flawed. [paper]
Maurice Jakesch, Jeffrey Hancock and Mor Naaman.
Study how humans discern whether verbal self-presentations were generated by AI. Show that human judgments of AI-generated language are handicapped by intuitive but flawed heuristics.
Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Text. [paper] 
Liam Dugan, Daphne Ippolito, Arun Kirubarajan, Sherry Shi and Chris Callison-Burch.
Analyze how a variety of variables (model size, decoding strategy, fine-tuning, prompt genre, etc.) affect human detection performance.
AI model GPT-3 (dis)informs us better than humans. [paper]
Giovanni Spitale, Nikola Biller-Andorno and Federico Germani.
Evaluate whether recruited individuals can distinguish disinformation from accurate information, and determine whether a tweet has been written by a Twitter user or by GPT-3.
Creating a Large Language Model of a Philosopher. [paper]
Eric Schwitzgebel, David Schwitzgebel and Anna Strasser.
Fine-tuned the GPT-3 on the corpus of Daniel Dennett, then asked it a series of philosophical questions. Ordinary research participants untrained in philosophy were at or near chance in distinguishing GPT-3’s answers from those of an “actual human philosopher”. .
Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts? [paper] 
Adaku Uchendu, Jooyoung Lee, Hua Shen, Thai Le, Ting-Hao 'Kenneth' Huang and Dongwon Lee.
Find that: (1) expert humans detect deepfake texts significantly better than non-expert humans, (2) synchronous teams on the Upwork detect deepfake texts significantly better than individuals, while asynchronous teams on the AMT detect deepfake texts weakly better than individuals, and (3) among various error categories, examining coherence and consistency in texts is useful in detecting deepfake texts.
Too Good to Be True? An Empirical Study of ChatGPT Capabilities for Academic Writing and Implications for Academic Misconduct. [paper] 
Peter André Busch and Geir Inge Hausvik.
Ask 15 faculty members to assess a total of ten answers (two each) to an actual IS exam question, of which students wrote five, and ChatGPT generated five. Find that ChatGPT can generate answers of generally good quality that may pass as human-written text by examiners.
Game of Tones: Faculty detection of GPT-4 generated content in university assessments. [paper]
Mike Perkins, Jasper Roe, Darius Postma, James McGaughran and Don Hickerson.
This study explores the robustness of university assessments against the use of Open AI's Generative Pre-Trained Transformer 4 (GPT-4) generated content and evaluates the ability of academic staff to detect its use when supported by the Turnitin Artificial Intelligence (AI) detection tool.
The Unseen A+ Student: Navigating the Impact of Large Language Models in the Classroom. [paper] 
Matyas Bohacek.
Collect a dataset of authentic high school coursework and generate their AI alternatives and text continuations using ChatGPT with 4.0, 3.5, and 3.5 Legacy backbones. Through a study involving student peers, we found that ChatGPT can quickly produce high-school-level coursework that peers consider better than human-written text, even in a low-resourced language like Czech. Moreover, show that the AI text detectors fail to identify these texts in Czech.
Can linguists distinguish between ChatGPT/AI and human writing?: A study of research ethics and academic publishing. [paper] 
J. Elliott Casal and Matt Kessler.
Investigate: 1) the extent to which linguists/reviewers from top journals can distinguish AI- from human-generated writing, 2) what the basis of reviewers’ decisions are, and 3) the extent to which editors of top Applied Linguistics journals believe AI tools are ethical for research purposes.
Detection of GPT-4 Generated Text in Higher Education: Combining Academic Judgement and Software to Identify Generative AI Tool Misuse. [paper] 
Mike Perkins, Jasper Roe, Darius Postma, James McGaughran and Don Hickerson.
Explore the capability of academic staff assisted by the Turnitin Artificial Intelligence (AI) detection tool to identify the use of AI-generated content in university assessments.
ChatGPT versus Human Essayists: An Exploration of the Impact of Artificial Intelligence for Authorship and Academic Integrity in the Humanities. [paper]
Tom Revell, Will Yeadon, Glenn Cahilly-Bretzin, Isabella Clarke, George Manning, Jasmine Jones, Clare Mulley, Rafael Pascual, Natasha Bradley, Daniel Thomas and Francis Leneghan.
Examine AI’s ability to write essays analysing Old English poetry; human markers assessed and attempted to distinguish them from authentic analyses of poetry by first-year undergraduate students in English at the University of Oxford.
Evaluating AI and Human Authorship Quality in Academic Writing through Physics Essays. [paper]
Will Yeadon, Elise Agra, Oto-obong Inyang, Paul Mackay and Arin Mizouri.
Evaluate n=300 short-form physics essay submissions, equally divided between student work submitted before the introduction of ChatGPT and those generated by OpenAI's GPT-4. In blinded evaluations conducted by five independent markers who were unaware of the origin of the essays, we observed no statistically significant differences in scores between essays authored by humans and those produced by AI (p-value =0.107, α = 0.05).
A feasibility study for the application of AI-generated conversations in pragmatic analysis. [paper] 
Xi Chen, Jun Li and Yuting Ye.
Compare 148 ChatGPT-generated conversations with 82 human-written ones and 354 human evaluations of these conversations. The data are analysed using various methods, including traditional speech strategy coding, four computational methods developed in NLP, and four statistical tests.
People cannot distinguish GPT-4 from a human in a Turing test. [paper]
Cameron R. Jones and Benjamin K. Bergen.
Evaluate 3 systems (ELIZA, GPT-3.5 and GPT-4) in a randomized, controlled, and preregistered Turing test. Human participants had a 5 minute conversation with either a human or an AI, and judged whether or not they thought their interlocutor was human. GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%).
Can human intelligence safeguard against artificial intelligence? Exploring individual differences in the discernment of human from AI texts. [paper]
Jason Chein, Steven Martinez and Alexander Barone.
Examine individual differences in the human ability to differentiate human- from AI-generated texts, exploring relationships with fluid intelligence, executive functioning, empathy, and digital habits. Overall, participants exhibited better than chance text discrimination, with substantial variation across individuals.
Limitations of Human Identification of Automatically Generated Text. [paper] 
Nadège Alavoine, Maximin Coavoux, Emmanuelle Esperança-Rodier, Romane Gallienne, Carlos Gonzalez Gallardo, Jérôme Goulian, Jose G. Moreno, Aurélie Névéol, Didier Schwab, Vincent Segonne and Johanna Simoens.
Propose a new corpus in French and English for the task of recognising automatically generated texts and conduct a study of how humans perceive the text. The results show that the generated texts by tools such as ChatGPT share some common characteristics but they are not clearly identifiable which generates different perceptions of these texts.
Seeing Through AI's Lens: Enhancing Human Skepticism Towards LLM-Generated Fake News. [paper]
Navid Ayoobi, Sadat Shahriar and Arjun Mukherjee.
By providing cues in human-written and LLM-generated news, we can help individuals increase their skepticism towards fake LLM-generated news. This paper aims to elucidate simple markers that help individuals distinguish between articles penned by humans and those created by LLMs. Devise a metric named Entropy-Shift Authorship Signature (ESAS) based on the information theory and entropy principles.
GPT-4 is judged more human than humans in displaced and inverted Turing tests. [paper]
Ishika Rathi, Sydney Taylor, Benjamin K. Bergen and Cameron R. Jones.
Measure how well people and large language models can discriminate using two modified versions of the Turing test: inverted and displaced. GPT-3.5, GPT-4, and displaced human adjudicators judged whether an agent was human or AI on the basis of a Turing test transcript. Suggest that both humans and current LLMs struggle to distinguish between the two when they are not actively interrogating the person, underscoring an urgent need for more accurate tools to detect AI in conversations.
Defending Against Neural Fake News. [paper] 
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner and Yejin Choi.
Present a model for controllable text generation called Grover. Find that the best way to detect neural fake news is to use a model that is also a generator.
GLTR: Statistical Detection and Visualization of Generated Text. [paper] 
Sebastian Gehrmann, Hendrik Strobelt and Alexander M. Rush.
A Giant Language model Test Room. GLTR aims to both teach users what to be aware of when assessing whether a text is real, and to assist them in performing forensic analyses.
Real or Fake? Learning to Discriminate Machine from Human Generated Text. [paper]
Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc'Aurelio Ranzato and Arthur Szlam.
Use Energy-based models to discriminate real text from text generated by the auto-regressive models.
The Limitations of Stylometry for Detecting Machine-Generated Fake News. [paper] 
Tal Schuster, Roei Schuster, Darsh J Shah and Regina Barzilay.
Examine the state-of-the-art stylometry model, and find it effective in preventing impersonation, but limited in detecting LM-generated misinformation.
Automatic Detection of Generated Text is Easiest when Humans are Fooled. [paper] 
Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch and Douglas Eck.
A comprehensive study of generated text detection systems’ sensitivity to model structure, decoding strategy, and excerpt length.
Reverse Engineering Configurations of Neural Text Generation Models. [paper]
Yi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler and Andrew Tomkins.
Propose the new task of distinguishing which of several variants (e.g., sampling methods, top-k probabilities, model architectures, etc.) of a given model generated some piece of text. Find that detectable artifacts are present and that different modeling choices can be inferred by looking at generated text alone.
TweepFake: about Detecting Deepfake Tweets. [paper] 
Tiziano Fagni, Fabrizio Falchi, Margherita Gambini, Antonio Martella and Maurizio Tesconi.
Collect the first dataset of real deepfake tweets, TweepFake. Evaluate 13 deepfake text detection methods.
Identifying Automatically Generated Headlines using Transformers. [paper] 
Antonis Maronikolakis, Hinrich Schutze and Mark Stevenson.
Create a dataset containing human and computer-generated headlines. Transformers achieved an overall accuracy of 85.7%.
Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News. [paper]
Reuben Tan, Bryan A. Plummer and Kate Saenko.
Present DIDAN, a approach which exploits possible semantic inconsistencies between the text and image/captions to detect machine-generated articles.
Neural Deepfake Detection with Factual Structure of Text. [paper]
Wanjun Zhong, Duyu Tang, Zenan Xu, Ruize Wang, Nan Duan, Ming Zhou, Jiahai Wang and Jian Yin.
Propose a graph-based model that utilizes the factual structure of a document for deepfake detection of text.
Authorship Attribution for Neural Text Generation. [paper]
Adaku Uchendu, Thai Le, Kai Shu and Dongwon Lee.
Investigate the authorship attribution problem in three versions: (1) given two texts T1 and T2, are both generated by the same method or not? (2) is the given text T written by a human or machine? (3) given a text T and k candidate neural methods, can we single out the method (among k alternatives) that generated T?
How Effectively Can Machines Defend Against Machine-Generated Fake News? An Empirical Study. [paper] 
Meghana Moorthy Bhat and Srinivasan Parthasarathy.
Measure the performance of models by looking at accuracy with respect to perturbations introduced in this work. Find that success of style-based classifiers are limited when real articles are perturbed even under extreme modifications.
Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover. [paper] 
Leon Fröhling and Arkaitz Zubiaga.
A feature-based detection approach relies on features that discriminate between human and machine text by modelling properties and dimensions in which both types of text differ.
Detecting Bot-Generated Text by Characterizing Linguistic Accommodation in Human-Bot Interactions. [paper] 
Paras Bhatt and Anthony Rios.
Show that bot-generated text detection methods are more robust across datasets and models if we use information about how people respond to it rather than using the bot’s text directly.
Artificial Text Detection via Examining the Topology of Attention Maps. [paper] 
Laida Kushnareva, Daniil Cherniavskii, Vladislav Mikhailov, Ekaterina Artemova, Serguei Barannikov, Alexander Bernstein, Irina Piontkovskaya, Dmitri Piontkovski and Evgeny Burnaev.
Propose three novel types of interpretable topological features based on Topological Data Analysis (TDA). The features derived from the BERT model outperform count- and neural-based baselines up to 10% on three common datasets, and tend to be the most robust towards unseen GPT-style generation models as opposed to existing methods.
Unsupervised and Distributional Detection of Machine-Generated Text. [paper]
Matthias Gallé, Jos Rozen, Germán Kruszewski and Hady Elsahar.
Propose a method to detect those machine-generated documents leveraging repeated higher-order n-grams, which we show over-appear in machine-generated text as compared to human ones.
Detecting computer-generated disinformation. [paper] 
Harald Stiff and Fredrik Johansson.
Evaluate promising Transformer-based detection algorithms in a large variety of experiments involving both in-distribution and out-of-distribution test data, as well as evaluation on more realistic in-the-wild data.
On pushing DeepFake Tweet Detection capabilities to the limits. [paper] 
Margherita Gambini, Tiziano Fagni, Fabrizio Falchi and Maurizio Tesconi.
Study and improve the performance of the stateof-the-art deepfake tweet detection methods over GPT-2 tweets and the detectors’ capabilities to generalize over tweets generated by GPT-3.
Cross-Domain Detection of GPT-2-Generated Technical Text. [paper]
Juan Rodriguez, Todd Hay, David Gros, Zain Shamsi and Ravi Srinivasan.
Find that RoBERTa-based detectors can be successfully adapted from one scientific discipline (physics) to another (biomedicine), requiring relatively small amounts of in-domain labeled data.
Automatic Detection of Machine Generated Texts: Need More Tokens. [paper] 
Paras Bhatt and Anthony Rios.
Present a dataset for Russian language and conduct a set of learning experiments to build accurate machine-generated text detectors for both English and Russian languages.
Threat Scenarios and Best Practices to Detect Neural Fake News. [paper] 
Artidoro Pagnoni, Martin Graciarena and Yulia Tsvetkov.
Provide an assessment of the current landscape of generated text detection and identify three primary threat scenarios. Establish the minimax strategies that minimize the worst case scenario for the detector.
Demystifying Neural Fake News via Linguistic Feature-Based Interpretation. [paper]
Ankit Aich, Souvik Bhattacharya and Natalie Parde.
Conduct a linguistically interpretative examination of the feature vulnerabilities exploited by neural fake news generators. Establish 21 stylistic, complexity, and psychological features for detection.
CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Data Limitation With Contrastive Learning. [paper] 
Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Yu Lan and Chao Shen.
Present a coherence-based contrastive learning model named CoCo to detect the possible machine-generated text under low-resource scenario.
How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection. [paper]
Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue and Yupeng Wu.
Collect dataset the Human ChatGPT Comparison Corpus (HC3) and build three different detection systems to detect whether a certain text is generated by ChatGPT or humans.
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. [paper] 
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning and Chelsea Finn.
Propose DetectGPT, a zero-shot method for automated machine-generated text detection. To test if a passage came from a source model, DetectGPT compares the log probability of a candidate passage with the average log probability of several perturbations of the passage.
ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text. [paper]
Sandra Mitrović, Davide Andreoletti and Omran Ayoub.
Build Transformer-based model to discriminate between human-written and seemingly human text, focusing on short texts. Give some insights about ChatGPT writing style by SHAP explanations of the predictions.
AI vs. Human -- Differentiation Analysis of Scientific Content Generation. [paper]
Yongqiang Ma, Jiawei Liu, Fan Yi, Qikai Cheng, Yong Huang, Wei Lu and Xiaozhong Liu.
Investigate the detection methods from three perspectives: (1) feature-based detection method; (2) fine-tuned pre-trained detection model; (3) explainability of detection model. Further investigate the gap between AI-generated text and human-written text.
ChatGPT Generated Text Detection. [paper]
Rexhep Shijaku and Ercan Canhasi.
Present a classification model based on XGBoost for automatically detecting essays generated by ChatGPT.
Linguistic Markers of AI-Generated Text Versus Human-Generated Text: Evidence from Hotel Reviews and News Headlines. [paper]
David M. Markowitz, Jeffrey Hancock and Jeremy Bailenson.
Use ChatGPT to compare how it wrote hotel reviews and news headlines to human-generated counterparts across content, style, and structural features.
Mutation-Based Adversarial Attacks on Neural Text Detectors. [paper]
Gongbo Liang, Jesus Guerrero and Izzat Alsmadi.
Propose character- and word-based mutation operators for generating adversarial samples to attack state-of-the-art natural text detectors.
GAN-Based Unsupervised Learning Approach to Generate and Detect Fake News. [paper] 
Pranjal Bhardwaj, Krishna Yadav, Hind Alsharif and Rania Anwar Aboalela.
Develop an unsupervised-based approach to generate fake news using autoencoder and GAN. Further describe the use of discriminators in detecting machine-generated fake news.
Combat AI With AI: Counteract Machine-Generated Fake Restaurant Reviews on Social Media. [paper]
Alessandro Gambetti and Qiwei Han.
Propose to leverage the high-quality elite restaurant reviews verified by Yelp to generate fake reviews from the OpenAI GPT review creator and ultimately fine-tune a GPT output detector to predict fake reviews that significantly outperform existing solutions.
Accurate Generated Text Detection Based on Deep Layer-wise Relevance Propagation. [paper] 
Mengjie Guo, Limin Liu, Meicheng Guo, Siyuan Liu and Zhiwei Xu.
Propose a deep interpretable model to achieve accurate detection of the generated texts. In detail, a feature extraction method based on Layer-wise Relevance Propagation is proposed to improve the feature mining performance for detection of generated texts.
Stylometric Detection of AI-Generated Text in Twitter Timelines. [paper]
Tharindu Kumarage, Joshua Garland, Amrita Bhattacharjee, Kirill Trapeznikov, Scott Ruston and Huan Liu.
Propose two simple architectures to utilize three categories of stylometric features towards 1) discriminating between human-written and AI-generated tweets and 2) detecting if and when an AI starts to generate tweets in a given Twitter timeline.
ChatGPT or academic scientist? Distinguishing authorship with over 99% accuracy using off-the-shelf machine learning tools. [paper]
Heather Desaire, Aleesa E. Chua, Madeline Isom, Romana Jarosova and David Hua.
With a set of 20 features, build a model that assigned the author, as human or AI, at well over 99% accuracy, resulting in 20 times fewer misclassified documents compared to the field-leading approach.
Detection of AI-generated Essays in Writing Assessments. [paper] 
Duanli Yan, Michael Fauss, Jiangang Hao, Wenju Cui.
Show how AI-generated essays are similar or different from human-written essays based on a set of typical prompts for a sample from a large-scale assessment. Introduce two classifiers that can detect AI-generated essays with a high accuracy of over 95%.
Comparing Abstractive Summaries Generated by ChatGPT to Real Summaries Through Blinded Reviewers and Text Classification Algorithms. [paper]
Mayank Soni, Vincent Wade.
Evaluate the performance of ChatGPT on Abstractive Summarization by the means of automated metrics and blinded human reviewers. Also build automatic text classifiers to detect ChatGPT generated summaries. Find that while text classification algorithms can distinguish between real and generated summaries, humans are unable to distinguish between real summaries and those produced by ChatGPT.
Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis. [paper]
Wataru Zaitsu and Mingzhe Jin.
Perform multi-dimensional scaling (MDS) to confirm the distributions of 216 texts of three classes (human, GPT-3.5, GPT-4) focusing on the following stylometric features: (1) bigrams of parts-of-speech, (2) bigram of postpositional particle words, (3) positioning of commas, and (4) rate of function words.
Evaluating AIGC Detectors on Code Content. [paper]
Jian Wang, Shangqing Liu, Xiaofei Xie and Yi Li.
Present the first empirical study on evaluating existing AIGC detectors in the software domain. Create a comprehensive dataset including 492.5K samples comprising code-related content produced by ChatGPT and evaluate six AIGC detectors on this dataset.
Detection of Fake Generated Scientific Abstracts. [paper]
Panagiotis C. Theocharopoulos, Panagiotis Anagnostou, Anastasia Tsoukala, Spiros V. Georgakopoulos, Sotiris K. Tasoulis and Vassilis P. Plagianakos.
Utilize the GPT-3 model to generate scientific paper abstracts through Artificial Intelligence and explore various text representation methods when combined with Machine Learning models with the aim of identifying machine-written text.
ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models. [paper]
Yikang Liu, Ziyin Zhang, Wanyang Zhang, Shisen Yue, Xiaojing Zhao, Xinyuan Cheng, Yiwen Zhang and Hai Hu.
Present ArguGPT, a balanced corpus of 4,038 argumentative essays generated by 7 GPT models in response to essay prompts from three sources: (1) in-class or homework exercises, (2) TOEFL and (3) GRE writing tasks. A RoBERTa fine-tuned with the training set of ArguGPT achieves above 90% accuracy in both essay- and sentence-level classification.
Improving Detection of ChatGPT-Generated Fake Science Using Real Publication Text: Introducing xFakeBibs a Supervised Learning Network Algorithm. [paper]
Ahmed Abdeen Hamed.
By means of a new algorithm, called xFakeBibs, show the significant difference between ChatGPT-generated fake publications and real publications. The algorithm predicted 98 of the 100 publications as fake, while 2 articles failed the test and were classified as real publications.
Differentiate ChatGPT-generated and Human-written Medical Texts. [paper]
Wenxiong Liao, Zhengliang Liu, Haixing Dai, Shaochen Xu, Zihao Wu, Yiyang Zhang, Xiaoke Huang, Dajiang Zhu, Hongmin Cai, Tianming Liu and Xiang Li.
Construct a suite of datasets containing medical texts written by human experts and generated by ChatGPT. Analyze the linguistic features of these two types of content and uncover differences in vocabulary, part-of-speech, dependency, sentiment, perplexity, etc. Finally, design and implement machine learning methods to detect medical text generated by ChatGPT.
CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts. [paper]
Peipeng Yu, Jiahan Chen, Xuan Feng and Zhihua Xia.
Present a large-scale CHatGPT-writtEn AbsTract dataset (CHEAT) contains 35,304 synthetic abstracts, with Generation, Polish, and Mix as prominent representatives. Show that ChatGPT-written abstracts are detectable, while the detection difficulty increases with human involvement.
Origin Tracing and Detecting of LLMs. [paper]
Linyang Li, Pengyu Wang, Ke Ren, Tianxiang Sun and Xipeng Qiu.
First raise the concern of the origin tracing of LLMs and propose an effective method to trace and detect AI-generated contexts. Introduce a novel algorithm that leverages the contrastive features between LLMs and extracts model-wise features to trace the text origins.
ChatLog: Recording and Analyzing ChatGPT Across Time. [paper]
Shangqing Tu, Chunyang Li, Jifan Yu, Xiaozhi Wang, Lei Hou and Juanzi Li.
Collect a coarse-to-fine temporal dataset called ChatLog, consisting of two parts that update monthly and daily. In section 5, explore the application of ChatGPT unchanged features on ChatLog-Daily dataset. Show a preliminary use case: improving the ChatGPT detection model’s robustness.
Perception, performance, and detectability of conversational artificial intelligence across 32 university courses. [paper]
Compare the performance of ChatGPT against students on 32 university-level courses. Current AI-text classifiers cannot reliably detect ChatGPT’s use in school work, due to their propensity to classify human-written answers as AI-generated, as well as the ease with which AI-generated text can be edited to evade detection.
GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content. [paper]
Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh and Bhiksha Raj.
Collecte and release a pre-processed dataset named OpenGPTText, which consists of rephrased content generated using ChatGPT. Design, implement, and train two different models for text classification, using RoBERTa and T5, respectively. These models achieve remarkable results, with an accuracy of over 97% on the test dataset, as evaluated through various metrics.
Machine-Made Media: Monitoring the Mobilization of Machine-Generated Articles on Misinformation and Mainstream News Websites. [paper]
Hans W. A. Hanley and Zakir Durumeric.
Train a DeBERTa-based synthetic news detector and classify over 12.91 million articles from 3,074 misinformation and mainstream news websites. Find that between January 1, 2022 and April 1, 2023, the relative number of synthetic news articles increased by 79.4% on mainstream websites while increasing by 342% on misinformation sites.
Smaller Language Models are Better Black-box Machine-Generated Text Detectors. [paper] 
Fatemehsadat Mireshghallah, Justus Mattern, Sicun Gao, Reza Shokri and Taylor Berg-Kirkpatrick.
Explore whether models other than the generator can be used to differentiate between machinegenerated and human-written text. Find that overall, smaller and partially-trained models are better universal text detectors. Interestingly, we find that whether the detector and generator were trained on the same data is not critically important to the detection success.
DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection. [paper] 
Xiao Yu, Yuang Qi, Kejiang Chen, Guoqiang Chen, Xi Yang, Pengyuan Zhu, Xiuwei Shang, Weiming Zhang and Nenghai Yu.
Propose to decouple prompt and intrinsic characteristics (DPIC) for LLM-generated text detection method. Specifically, given a candidate text, DPIC employs an auxiliary LLM to reconstruct the prompt corresponding to the candidate text, then uses the prompt to regenerate text by the auxiliary LLM. Then, the similarity between the candidate text and the regenerated text is used as a detection feature.
G3Detector: General GPT-Generated Text Detector. [paper]
Haolan Zhan, Xuanli He, Qiongkai Xu, Yuxiang Wu and Pontus Stenetorp.
Introduce an unpretentious yet potent detection approach proficient in identifying synthetic text across a wide array of fields. The detector demonstrates outstanding performance uniformly across various model architectures and decoding strategies. It also possesses the capability to identify text generated utilizing a potent detectionevasion technique.
MAGE: Machine-generated Text Detection in the Wild. [paper] 
Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Longyue Wang, Linyi Yang, Shuming Shi and Yue Zhang.
Construct a wild testbed for deepfake text detection, by gathering texts from various writing tasks and deepfake texts generated by different LLMs. Find that supervised PLM-based methods were the best-performing detection methods across all testbeds. Out-of-distribution posed an even greater challenge for a detector to be employed in realistic application scenarios.
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text. [paper] 
Jinyan Su, Terry Yue Zhuo, Di Wang and Preslav Nakov.
Introduce two novel zero-shot methods for detecting machinegenerated text by leveraging the log rank information. One is called DetectLLM-LRR, which is fast and efficient, and the other is called DetectLLM-NPR, which is more accurate, but slower due to the need for perturbations.
LLMDet: A Third Party Large Language Models Generated Text Detection Tool. [paper]
Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng and Tat-Seng Chua.
Propose an efficient, secure, and scalable detection tool called LLMDet, which calculates the proxy perplexity of text by utilizing the prior information of the model’s next-token probabilities, obtained through pre-training. Subsequently, use the self-watermarking information of the model, as measured by proxy perplexity, to detect the source of the text.
Ghostbuster: Detecting Text Ghostwritten by Large Language Models. [paper]
Vivek Verma, Eve Fleisig, Nicholas Tomlin and Dan Klein.
Introduce Ghostbuster, works by passing documents through a series of weaker language models and running a structured search over possible combinations of their features, then training a classifier on the selected features to determine if the target document was AI-generated. Release three new datasets of human and AI-generated text as detection benchmarks that cover multiple domains and task setups.
HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis. [paper]
Christoforos Vasilatos, Manaar Alam, Talal Rahwan, Yasir Zaki and Michail Maniatakos.
Propose a novel multi-level approach to detect AIgenerated text focusing on university student homework. Utilize metadata categorization from an academic dataset to enhance the perplexity metric used to detect whether a given assignment has been studentauthored or AI-generated.
DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text. [paper] 
Xianjun Yang, Wei Cheng, Linda Petzold, William Yang Wang and Haifeng Chen.
Given a text, we first truncate it in the middle and then use only the preceding portion as input to the LLMs to regenerate the new remaining parts. By analyzing the differences between the original and new remaining parts through N-gram analysis in black-box or probability divergence in white-box, we can clearly illustrate significant discrepancies between machine-generated and human-written text.
Efficient Detection of LLM-generated Texts with a Bayesian Surrogate Model. [paper] 
Zhijie Deng, Hongcheng Gao, Yibo Miao and Hao Zhang.
DetectGPT suffers from significant inefficiency issues, as detecting a single candidate requires scoring hundreds of its perturbations with the source LLM. This paper aims to bridge this gap. Technically, propose to incorporate a Bayesian surrogate model, which allows us to select typical samples based on Bayesian uncertainty and interpolate scores from typical samples to other ones, to improve query efficiency.
Distinguishing Human Generated Text From ChatGPT Generated Text Using Machine Learning. [paper]
Niful Islam, Debopom Sutradhar, Humaira Noor, Jarin Tasnim Raya, Monowara Tabassum Maisha and Dewan Md Farid.
Present a machine learning-based solution that can identify the ChatGPT delivered text from the human written text along with the comparative analysis of a total of 11 machine learning and deep learning algorithms in the classification process.
Multiscale Positive-Unlabeled Detection of AI-Generated Texts. [paper]
Yuchuan Tian, Hanting Chen, Xutao Wang, Zheyuan Bai, Qinghua Zhang, Ruifeng Li, Chao Xu and Yunhe Wang.
Mainstream detectors are formulated without considering the factor of corpus length: shorter corpuses are harder to detect compared with longer ones for shortage of informative features. In this paper, a Multiscale Positive-Unlabeled (MPU) training framework is proposed to address the challenge of multiscale text detection.
Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts. [paper]
Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, Serguei Barannikov, Irina Piontkovskaya, Sergey Nikolenko and Evgeny Burnaev.
Propose an invariant of human texts, namely the intrinsic dimensionality of the manifold underlying the set of embeddings of a given text sample. We show a clear statistical separation between human-generated and AI-generated distributions. This property allows us to build a score-based artificial text detector, which outperforming SOTA detectors in model-agnostic and cross-domain scenarios by a significant margin.
Distinguishing academic science writing from humans or ChatGPT with over 99% accuracy using off-the-shelf machine learning tools. [paper] 
Heather Desaire, Aleesa E. Chua, Madeline Isom, Romana Jarosova and David Hua.
Report a method for discriminating text generated by ChatGPT from (human) academic scientists, relying on prevalent and accessible supervised classification methods. The approach uses new features for discriminating (these) humans from AI.
Long-form analogies generated by chatGPT lack human-like psycholinguistic properties. [paper] 
S. M. Seals and Valerie L. Shalin.
Apply psycholinguistic methods to evaluate individual sentences from long-form analogies about biochemical concepts. Perform a supervised classification analysis using 78 features extracted from Coh-metrix that analyze text cohesion, language, and readability. Results illustrate high performance for classifying student-generated and chatGPT-generated analogies.
Check Me If You Can: Detecting ChatGPT-Generated Academic Writing using CheckGPT. [paper]
Zeyan Liu, Zijun Yao, Fengjun Li and Bo Luo.
Present GPABenchmark, a benchmarking dataset of 600,000 samples of human-written, GPT-written, GPT-completed, and GPT-polished abstracts of research papers in CS, physics, and humanities and social sciences (HSS). Present CheckGPT, a novel LLM-content detector consisting of a general representation module and an attentive-BiLSTM classification module, which is accurate, transferable, and interpretable.
Stanford CS224N Custom Projects
Check Me If You Can: Detecting ChatGPT-Generated Academic Writing using CheckGPT. [paper]
Jeffrey Heo and Simon Kim.
Looking Under the Hood of DetectGPT. [paper]
Maximilian Du, Ryan Lian and Kaien Yang.
GPTNo: A Deep Learning LLM to Beat the Turing Test. [paper]
Sri Jaladi, William Li and Abhinav Sinha.
Human Writing is as Uniform as Machine Writing. [paper]
Ryan Tan, Raghav Garg and Jacob Stavrianos.
DetectChatGPT: Black-Box Zero-Shot Detection of LLM-Generated Text. [paper]
Julia Park and Armaan Rashid.
La détection de textes générés par des modèles de langue: une tâche complexe? Une étude sur des textes académiques. [paper] 
Vijini Liyanage and Davide Buscaldi.
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect? [paper]
Wissam Antoun, Virginie Mouilleron, Benoît Sagot and Djamé Seddah.
Proposes a methodology for developing and evaluating ChatGPT detectors for French text, with a focus on investigating their robustness on outof-domain data and against common attack schemes. The proposed method involves translating an English dataset into French and training a classifier on the translated data.
Implementing BERT and fine-tuned RobertA to detect AI generated news by ChatGPT. [paper]
Zecong Wang, Jiaxi Cheng, Chen Cui and Chenhao Yu.
Demonstrate that the BERT and RobertA models with fine-tuning had the best success in detecting AI generated news. With a score of 98%, tweaked RobertA in particular showed excellent precision.
Beyond Black Box AI-Generated Plagiarism Detection: From Sentence to Document Level. [paper] 
Mujahid Ali Quidwai, Chunhui Li and Parijat Dube.
Propose a novel approach offering quantifiable metrics at both sentence. Employs a multi-faceted approach, generating multiple paraphrased versions of a given question and inputting them into the LLM to generate answers. By using a contrastive loss function based on cosine similarity, match generated sentences with those from the student’s response.
Matching Pairs: Attributing Fine-Tuned Models to their Pre-Trained Large Language Models. [paper] 
Myles Foley, Ambrish Rawat, Taesung Lee, Yufang Hou, Gabriele Picco and Giulio Zizzo.
We need a method to investigate how a model was trained or a piece of text was generated and what their pre-trained base model was. Address this open problem by tracing back the origin of a given fine-tuned LLM to its corresponding pre-trained base model. Consider different knowledge levels and attribution strategies, and find that we can correctly trace back 8 out of the 10 fine tuned models with our best method.
Detecting Artificially Generated Academic Text: The Importance of Mimicking Human Utilization of Large Language Models. [paper] 
Vijini Liyanage and Davide Buscaldi.
Examine the detectability of the generated text using DetectGPT and GLTR, and utilize state-of-the-art classification models like SciBERT, RoBERTa, DEBERTa, XLNet, and ELECTRA. Experiments show that the generated text is difficult to detect using existing models when created using a LLM fine-tuned on the remainder of a paper.
Testing of Detection Tools for AI-Generated Text. [paper]
Debora Weber-Wulff, Alla Anohina-Naumeca, Sonja Bjelobaba, Tomáš Foltýnek, Jean Guerrero-Dib, Olumide Popoola, Petr Šigut and Lorna Waddington.
The study seeks to answer research questions about whether existing detection tools can reliably differentiate between human-written text and ChatGPT-generated text, and whether machine translation and content obfuscation techniques affect the detection of AIgenerated text. The research covers 12 publicly available tools and two commercial systems (Turnitin and PlagiarismCheck) that are widely used in the academic setting.
Discriminating Human-authored from ChatGPT-Generated Code Via Discernable Feature Analysis. [paper]
Li Ke, Hong Sheng, Fu Cai, Zhang Yunhe and Liu Ming.
Propose a discriminative feature set yielding high accuracy in differentiating ChatGPT-generated code from human-authored code in binary classification tasks; Devise methods for generating extensive ChatGPT-generated codes; Introduce a dataset cleansing strategy.
RADAR: Robust AI-Text Detection via Adversarial Learning. [paper] 
Xiaomeng Hu, Pin-Yu Chen and Tsung-Yi Ho.
Existing works show that current AI-text detectors are not robust to LLM-based paraphrasing, this paper aims to bridge this gap by proposing a new framework called RADAR. RADAR is based on adversarial training of a paraphraser and a detector. The paraphraser’s goal is to generate realistic contents to evade AI-text detection. RADAR uses the feedback from the detector to update the paraphraser, and vice versa.
Detecting LLM-Generated Text in Computing Education: A Comparative Study for ChatGPT Cases. [paper]
Michael Sheinman Orenstrakh, Oscar Karnalim, Carlos Anibal Suarez and Michael Liut.
Collect 124 submissions from computer science students before the creation of ChatGPT, then generate 40 ChatGPT submissions. Use this data to evaluate eight publicly-available LLM-generated text detectors through the measures of accuracy, false positives, and resilience.
Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text. [paper]
Lingyi Yang, Feng Jiang and Haizhou Li.
Introduce a novel dataset termed HPPT (ChatGPT-polished academic abstracts), facilitating the construction of more robust detectors. Propose the "Polish Ratio" method, an innovative measure of ChatGPT’s involvement in text generation based on editing distance. It provides a mechanism to measure the degree of human originality in the resulting text.
OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples. [paper] 
Ryuto Koike, Masahiro Kaneko and Naoaki Okazaki.
Propose OUTFOX, a novel framework that improves the robustness of LLM-generated-text detectors by allowing both the detector and the attacker to consider each other’s output and apply this to the domain of student essays.
The Looming Threat of Fake and LLM-generated LinkedIn Profiles: Challenges and Opportunities for Detection and Prevention. [paper]
Navid Ayoobi, Sadat Shahriar and Arjun Mukherjee.
Present a novel method for detecting fake and Large Language Model (LLM)-generated profiles in the LinkedIn Online Social Network immediately upon registration and before establishing connections.
The Imitation Game: Detecting Human and AI-Generated Texts in the Era of Large Language Models. [paper]
Kadhim Hayawi, Sakib Shahriar and Sujith Samuel Mathew.
Introduce a novel dataset of human-written and LLM-generated texts in different genres: essays, stories, poetry, and Python code. Employ several machine learning models to classify the texts.
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education. [paper] 
Zijie Zeng, Lele Sha, Yuheng Li, Kaixun Yang, Dragan Gašević and Guanliang Chen.
Construct a hybrid essay dataset. Propose a two-step detection approach where we (1) separate AI-generated content from human-written content during the encoder training process; and (2) calculate the distances between every two adjacent prototypes and assume that the boundaries exist between the two adjacent prototypes that have the furthest distance from each other.
Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text? [paper] 
Amrita Bhattacharjee and Huan Liu.
Empirically investigate if ChatGPT is symmetrically effective in detecting AI-generated or human-written text. Provide insight on how ChatGPT and similar LLMs may be leveraged in automated detection pipelines by simply focusing on solving a specific aspect of the problem and deriving the rest from that solution.
A Deep Fusion Model for Human vs. Machine-Generated Essay Classification. [paper] 
Roberto Corizzo and Sebastian Leal-Arenas.
Propose a deep neural network-based model that combines text modeling and linguistic features via data fusion, with the aim to provide robust classification capabilities. Experiments show that it is possible to accurately perform essay classification, and that our model is competitive, even when compared to popular feature-based and neural network-based approaches.
MFD: Multi-Feature Detection of LLM-Generated Text. [paper]
Zhendong Wu and Hui Xiang.
Introduce Multi-Feature Detection (MFD), a new zero-shot method. MFD comprehensively considers log-likelihood, log-rank, entropy, and LLM-Deviation. LLM-Deviation is a new statistical feature proposed in this paper and has a clear distribution difference between texts generated by LLMs and those written by humans. Experiments show MFD is more effective than the existing zero-shot method.
Classification of Human- and AI-Generated Texts: Investigating Features for ChatGPT. [paper]
Lorenz Mindner, Tim Schlippe and Kristina Schaaff.
Explore traditional and new features to (1) detect text generated by AI from scratch and (2) text rephrased by AI. Results show that the new features substantially help to improve the performance of many classifiers.
Separating the Human Touch from AI-Generated Text using Higher Criticism: An Information-Theoretic Approach. [paper]
Alon Kipnis.
Propose a method to determine whether a given article was entirely written by a generative language model. The process involves many perplexity tests for the origin of individual sentences or other text atoms, combining these multiple tests using Higher Criticism (HC).
人工智能生成语言与人类语言对比研究——以ChatGPT为例. [paper] 
朱君辉, 王梦焰, 杨尔弘, 聂锦燃, 王誉杰, 岳岩 and 杨麟儿.
Explore the differences between the language used in human-generated responses and responses generated by ChatGPT. Extract and compute the distribution of 159 language features in real human text and ChatGPT-generated text. Employ three machine learning algo-rithms: Random Forest, Logistic Regression, and SVM. The result reveals that the two texts differ significantly in three dimensions: descriptive features, word commonness, and word diversity.
HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus. [paper]
Zhenpeng Su, Xing Wu, Wei Zhou, Guangyuan Ma and Songlin Hu.
Introduce a more extensive and comprehensive dataset that considers more types of tasks than previous work, including semantic-invariant tasks. Further instruct fine-tuning Tk-instruct and build a more powerful detection system. Experimental results show that the proposed detector outperforms the previous state-of-the-art RoBERTa-based detector.
J-Guard: Journalism Guided Adversarially Robust Detection of AI-generated News. [paper] 
Tharindu Kumarage, Amrita Bhattacharjee, Djordje Padejski, Kristy Roschke, Dan Gillmor, Scott Ruston, Huan Liu and Joshua Garland.
Develop a framework, J-Guard, capable of steering existing supervised AI text detectors for detecting AI-generated news while boosting adversarial robustness. By incorporating stylistic cues inspired by the unique journalistic attributes, J-Guard effectively distinguishes between real-world journalism and AIgenerated news articles.
Overview of AuTexTification at IberLEF 2023: Detection and Attribution of Machine-Generated Text in Multiple Domains. [paper] 
Areg Mikael Sarvazyan, José Ángel González, Marc Franco-Salvador, Francisco Rangel, Berta Chulvi and Paolo Rosso.
This paper presents the overview of the AuTexTification shared task as part of the IberLEF 2023 Workshop. AuTexTification consists of two subtasks: for Subtask 1, participants had to determine whether a text is human-authored or has been generated by a large language model. For Subtask 2, participants had to attribute a machine-generated text to one of six different text generation models. In this overview, present the AuTexTification dataset and task, the submitted participating systems, and the results.
ConDA: Contrastive Domain Adaptation for AI-generated Text Detection. [paper]
Amrita Bhattacharjee, Tharindu Kumarage, Raha Moraffah and Huan Liu.
Address the problem of AIgenerated text detection in the absence of labeled target data. Propose a contrastive domain adaptation framework that leverages the power of both unsupervised domain adaptation and selfsupervised representation learning, in order to tackle the task of AI-generated text detection.
Supervised Machine-Generated Text Detectors: Family and Scale Matters. [paper] 
Areg Mikael Sarvazyan, José Ángel González, Paolo Rosso and Marc Franco-Salvador.
This work studies the generalization capabilities of supervised Machine-Generated Text (MGT) detectors across model families and parameter scales of text generation models. In addition, explore the feasibility of identifying the family and scale of the generator behind an MGT, instead of attributing the text to a particular language model.
Use prompt to differentiate text generated by ChatGPT and humans. [paper] 
Ruopeng An, Yuyi Yang, Fan Yang and Shanshan Wang.
Introduce a method that uses both prompts and essays to differentiate between machine-generated and human-written text. Validate its effectiveness in distinguishing between machine-generated and human-written essays and shows that it outperforms existing approaches based solely on text passages.
A Statistical Turing Test for Generative Models. [paper]
Hayden Helm, Carey E. Priebe and Weiwei Yang.
Provide a framework in the language of statistical pattern recognition that quantifies the difference between the distributions of human and machinegenerated content conditioned on an evaluation context. Describe current methods in the context of the framework and demonstrate how to use the framework to evaluate the progression of generative models towards human-like capabilities, among many axes of analysis.
TopRoBERTa: Topology-Aware Authorship Attribution of Deepfake Texts. [paper]
Adaku Uchendu, Thai Le and Dongwon Lee.
Propose TopRoBERTa to improve existing AA solutions by capturing more linguistic patterns in deepfake texts by including a Topological Data Analysis (TDA) layer in the RoBERTa model. Use RoBERTa to capture contextual representations (i.e., semantic and syntactic linguistic features), while using TDA to capture the shape and structure of data (i.e., linguistic structures). Finally, TopRoBERTa, outperforms the vanilla RoBERTa in 2/3 datasets, achieving up to 7% increase in Macro F1 score.
From Text to Source: Results in Detecting Large Language Model-Generated Content. [paper]
Wissam Antoun, Benoît Sagot and Djamé Seddah.
Investigate "Cross-Model Detection," evaluating whether a classifier trained to distinguish between source LLM-generated and humanwritten text can also detect text from a target LLM without further training. The study comprehensively explores various LLM sizes and families, and assesses the impact of conversational fine-tuning techniques on classifier generalization. The research also delves into Model Attribution, encompassing source model identification, model family classification, and model size classification.
Can LLM-Generated Misinformation Be Detected? [paper]
Canyu Chen and Kai Shu.
Build a taxonomy of LLM-generated misinformation. Categorize and validate the potential real-world methods for generating misinformation with LLMs. Discover that LLM-generated misinformation can be harder to detect for humans and detectors compared to human-written misinformation with the same semantics.
A Framework for Detecting AI-Generated Text in Research Publications. [paper] 
Paria Sarzaeim, Arya Doshi and Qusay Mahmoud.
Present a framework for AI-generated text. The prototype implementation of the proposed approach is to train a model using predefined datasets and deploy this model on a cloud-based service to predict whether a text was created by a human or AI. This approach is specifically focused on assessing the accuracy of scientific writings and research papers rather than general text.
Supervised Machine Generated Text Detection Using LLM Encoders In Various Data Resource Scenarios. [paper] 
Marc Capobianco, Matthew Reynolds, Charles Phelan, Krish Shah-Nathwani and Duong Luong.
Explore the effectiveness of BERT and RoBERTa-based machine generated text detection in a supervised setting. Created several models for both BERT and RoBERTa, trained with 5%, 10%, 15%, 20%, and 100% of the dataset. Conduct these experiments with frozen and unfrozen parameter variations.
Machine-Generated Text Detection and Attribution. [paper]
Sarvazyan and Areg Mikael.
(i) study the MGT detection and attribution models that achieve high performance under various settings, (ii) explore their generalization capabilities to different scenarios, analyzing their applicability to unseen types of data, and (iii) successfully organize an evaluation campaign for MGT detection and attribution.
Automated Text Identification Shared Task (AuTexTification)
Automated Text Identification Using CNN and Training Dynamics. [paper] 
Claudiu Creanga and Liviu Petrisor Dinu.
Do Origin and Facts Identify Automatically Generated Text? [paper]
Judita Preiss and Monica Lestari Paramita.
Syntax-based Contrastive Learning for Automated Text Identification. [paper]
Hongyan Wu, Nankai Lin and Shengyi Jiang.
A Method for Identifying Bot Generated Text : Notebook for IberLEF 2023. [paper]
Maryam Elamine, Asma Mekki and Lamia Hadrich Belguith.
Artificial Intelligence-Based Text Classification: Separating Human Writing from Computer Generated Writing. [paper]
Luis Enrique Morales-Márquez, Erick Barrios-González and David Eduardo Pinto-Avendaño.
SINAI at AuTexTification in IberLEF 2023: Combining All Layer Embeddings for Automatically Generated Texts. [paper]
César Espin-Riofrio, Jenny Ortiz-Zambrano and Arturo Montejo-Ráez.
I’ve Seen Things You Machines Wouldn’t Believe: Measuring Content Predictability to Identify Automatically-Generated Text. [paper]
Piotr Przybyła, Nicolau Duran-Silva and Santiago Egea-Gómez.
Detecting Generated Text and Attributing Language Model Source with Fine-tuned Models and Semantic Understanding. [paper]
Margherita Gambini, Marco Avvenuti, Fabrizio Falchi, Maurizio Tesconi and Tiziano Fagni.
AI-Writing Detection Using an Ensemble of Transformers and Stylometric Features. [paper]
George Mikros, Athanasios Koursaris, Dimitrios Bilianos and George Markopoulos.
Exploring Text Representations for Detecting Automatically Generated Text. [paper]
Zayra Villegas-Trejo, Helena Gómez-Adorno and Sergio-Luis Ojeda-Trueba.
GPT-2 versus GPT-3 and Bloom: LLMs for LLMs Generative Text Detection. [paper]
Fernando Aguilar-Canto, Marco Cardoso-Moreno, Diana Jiménez and Hiram Calvo.
Taming the Turing Test: Exploring Machine Learning Approaches to Discriminate Human vs. AI-Generated Texts. [paper]
Alberto Fernández-Hernández, Juan Luis Arboledas-Márquez, Julián Ariza-Merino and Salud María Jiménez-Zafra.
Team GPLSI at AuTexTification Shared Task: Determining the Authorship of a Text. [paper]
Iván Martínez-Murillo, Robiert Sepúlveda-Torres, Estela Saquete, Elena LLoret and Manuel Palomar.
Generative AI Text Classification using Ensemble LLM Approaches. [paper]
Harika Abburi, Michael Suesserman, Nirmala Pudota, Balaji Veeramani, Edward Bowen and Sanmitra Bhattacharya.
Automated Text Identification: Multilingual Transformer-based Models Approach. [paper]
German Gritsay, Andrey Grabovoy, Aleksandr Kildyakov and Yury Chekhovich.
IberLEF 2023 AuTexTification: Automated Text Identification Shared Task – Team OD-21. [paper]
Rinaldo Gagiano, Haytham Fayek, Maria Myung-Hee Kim, Jennifer Biggs and Xiuzhen Zhang.
Using Linguistic Knowledge for Automated Text Identification. [paper]
Lara Alonso Simón, José Antonio Gonzalo Gimeno, Ana María Fernández-Pampillón Cesteros, Marianela Fernández Trinidad and María Victoria Escandell Vidal.
Univ. of Hildesheim at AuTexTification 2023: Detection of Automatically Generated Texts. [paper]
Tatjana Scheibe and Thomas Mandl.
UPB at IberLEF-2023 AuTexTification: Detection of Machine-Generated Text using Transformer Ensembles. [paper]
Andrei-Alexandru Preda, Dumitru-Clementin Cercel, Traian Rebedea and Costin-Gabriel Chiru.
Exploring the Distinction: Investigating the Recognition of Automatic Text Generation Systems and Differentiating Human Text from Language Models. [paper]
Manex Agirrezabal.
Looking for Traces of Textual Deepfakes in Bulgarian on Social Media. [paper] 
Irina Temnikova, Iva Marinova, Silvia Gargova, Ruslana Margova and Ivan Koychev.
Create a new Bulgarian-language dataset with real social media messages and those generated by two language models. Conclude that combining LM text detection with fact-checking is the most appropriate method for this task, and that identifying Bulgarian textual deepfakes is indeed possible.
On the Generalization of Training-based ChatGPT Detection Methods. [paper]
Han Xu, Jie Ren, Pengfei He, Shenglai Zeng, Yingqian Cui, Amy Liu, Hui Liu and Jiliang Tang.
Aim to have a comprehensive investigation on existing methods’ generalization behaviors under distribution shift caused by a wide range of factors, including prompts, text lengths, topics, and language tasks. Collect a new dataset with human and ChatGPT texts, and then conduct extensive studies on the collected dataset.
Counter Turing Test CT^2: AI-Generated Text Detection is Not as Easy as You May Think -- Introducing AI Detectability Index. [paper]
Megha Chakraborty, S.M Towhidul Islam Tonmoy, S M Mehedi Zaman, Krish Sharma, Niyar R Barman, Chandan Gupta, Shreya Gautam, Tanay Kumar, Vinija Jain, Aman Chadha, Amit P. Sheth and Amitava Das.
Introduce the Counter Turing Test (CT2), a benchmark consisting of techniques aiming to offer a comprehensive evaluation of the robustness of existing AGTD techniques. Establish a quantifiable spectrum facilitating the evaluation and ranking of LLMs according to their detectability levels, we propose the AI Detectability Index (ADI).
Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature. [paper]
Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang and Yue Zhang.
Present Fast-DetectGPT, an optimized zero-shot detector, which substitutes DetectGPT’s perturbation step with a more efficient sampling step. The evaluations on various datasets, source models, and test conditions indicate that Fast-DetectGPT not only outperforms DetectGPT in both the whitebox and black-box settings but also accelerates the detection process by a factor of 340.
Zero-Shot Detection of Machine-Generated Codes. [paper]
Xianjun Yang, Kexun Zhang, Haifeng Chen, Linda Petzold, William Yang Wang and Wei Cheng.
Propose a training-free approach for the detection of LLMs-generated codes, mitigating the risks associated with their indiscriminate usage. Modify the previous zero-shot text detection method, DetectGPT by utilizing a surrogate white-box model to estimate the probability of the rightmost tokens, allowing us to identify code snippets generated by language models.
How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Prompts. [paper]
Tharindu Kumarage, Paras Sheth, Raha Moraffah, Joshua Garland and Huan Liu.
Suggest a universal evasive prompt, a novel type of soft prompt, which guides PLMs in producing "human-like" text that can mislead the detectors. The novel universal evasive prompt is achieved in two steps: First, create an evasive soft prompt tailored to a specific PLM through prompt tuning; and then, leverage the transferability of soft prompts to transfer the learned evasive soft prompt from one PLM to another.
On the Zero-Shot Generalization of Machine-Generated Text Detectors. [paper]
Xiao Pu, Jingyu Zhang, Xiaochuang Han, Yulia Tsvetkov and Tianxing He.
Collect generation data from a wide range of LLMs, and train neural detectors on data from each generator and test its performance on heldout generators. Demonstrate that robust detectors can be built on an ensemble of training data from medium-sized models.
GPT-who: An Information Density-based Machine-Generated Text Detector. [paper]
Saranya Venkatraman, Adaku Uchendu and Dongwon Lee.
Propose GPT-who, the first psycholinguistically-aware multi-class domainagnostic statistical-based detector. This detector employs UID-based features to model the unique statistical signature of each LLM and human author for accurate authorship attribution.
SeqXGPT: Sentence-Level AI-Generated Text Detection. [paper]
Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang and Xipeng Qiu.
Propose Sequence X (Check) GPT, a novel method that utilizes log probability lists from white-box LLMs as features for sentence-level AIGT detection. These features are composed like waves in speech processing and cannot be studied by LLMs. Build SeqXGPT based on convolution and self-attention networks.
Raidar: geneRative AI Detection viA Rewriting. [paper]
Chengzhi Mao, Carl Vondrick, Hao Wang and Junfeng Yang.
Find that large language models (LLMs) are more likely to modify human-written text than AI-generated text when tasked with rewriting. Introduce a method to detect AI-generated content by prompting LLMs to rewrite text and calculating the editing distance of the output.
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text. [paper] 
Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping and Tom Goldstein.
Propose a novel LLM detector that only requires simple calculations using pre-trained LLMs. The method, called Binoculars, achieves state-of-the-art accuracy without any training data.
Human-in-the-loop Detection of AI-generated Text via Grammatical Patterns. [paper] 
Johan Lokna, Mislav Balunovic and Martin Vechev.
Focus on the problem of detecting AI-generated text in a domain where a training dataset of human-written samples is readily available. Our key insight is to learn interpretable grammatical patterns that are highly indicative of human or AI written text. The most useful of these patterns can then be given to humans as part of a human-in-the-loop approach.
Few-Shot Detection of Machine-Generated Text using Style Representations. [paper]
Rafael Rivera Soto, Kailin Koch, Aleem Khan, Barry Chen, Marcus Bishop and Nicholas Andrews.
Pursue a approach not relying on samples from language models of concern at training time. Propose to leverage representations of writing style estimated from human-authored text. Find that features effective at distinguishing among human authors are also effective at distinguishing human from machine authors.
Metric Learning for Detection of Large Language Model Generated Texts. [paper]
Linh Le, Shashank Hebbar and My Nguyen.
Present a new paradigm of metric-based detection for LLM-generated texts that is able to balance among computational costs, accessibility, and performances. Specifically, the detection is performed through evaluating the similarity between a given text to an equivalent example generated by LLMs and through that determining the former's origination.
Detecting Machine-Generated Texts by Multi-Population Aware Optimization for Maximum Mean Discrepancy. [paper]
Shuhai Zhang, Feng Liu, Jiahao Yang, Yifan Yang, Changsheng Li, Bo Han and Mingkui Tan.
Propose a novel multi-population aware optimization method for MMD called MMD-MP, which can avoid variance increases and thus improve the stability to measure the distributional discrepancy. Relying on MMD-MP, develop two methods for paragraph-based and sentence-based detection, respectively.
Using AI-based detectors to control AI-assisted plagiarism in ESL writing: “The Terminator Versus the Machines”. [paper] 
Karim Ibrahim.
To address the challenge of AI-assisted plagiarism in ESL contexts, the present cross-disciplinary descriptive study examined the potential of two RoBERTa-based classifiers to control AI-assisted plagiarism on a dataset of 240 human-written and ChatGPT-generated essays. Data analysis revealed that both platforms could identify AI-generated texts, but their detection accuracy was inconsistent across the dataset.
An Ensemble-Based Approach for Generative Language Model Attribution. [paper] 
Harika Abburi, Michael Suesserman, Nirmala Pudota, Balaji Veeramani, Edward Bowen and Sanmitra Bhattacharya.
Propose an ensemble neural model that generates probabilities from multiple pre-trained LLMs, which are then used as features for a traditional machine learning classifier.
DetectGPT-SC: Improving Detection of Text Generated by Large Language Models through Self-Consistency with Masked Predictions. [paper]
Rongsheng Wang, Qi Li and Sihong Xie.
Find that large language models such as ChatGPT exhibit strong self-consistency in text generation and continuation. Propose DetectGPT-SC for AI-generated texts detection based on self-consistency with masked predictions to determine whether a text is generated by LLMs.
Do Stochastic Parrots have Feelings Too? Improving Neural Detection of Synthetic Text via Emotion Recognition. [paper]
Alan Cowap, Yvette Graham and Jennifer Foster.
Hypothesize that pretrained language models (PLMs) have an affective deficit because of lacking the kind of emotional coherence present in human-authored text. Develop an emotionally aware detector by fine-tuning a PLM on emotion.
How well can machine-generated texts be identified and can language models be trained to avoid identification? [paper] 
Sinclair Schneider, Florian Steuber, Joao A. G. Schneider and Gabi Dreo Rodosek.
Highlight the significance of sampling techniques and the essential role of employing advanced models like transformers to detect generated texts accurately. Demonstrate how a malicious actor could adapt generative models to evade a detector if accessible.
An Ensemble Method Based on the Combination of Transformers with Convolutional Neural Networks to Detect Artificially Generated Text. [paper] 
Vijini Liyanage and Davide Buscaldi.
Present some classification models constructed by ensembling transformer models such as SciBERT, DeBERTa and XLNet, with Convolutional Neural Networks (CNNs). The experiments demonstrate that the considered ensemble architectures surpass the performance of the individual transformer models for classification.
Detection of news written by the ChatGPT through authorship attribution performed by a Bidirectional LSTM model. [paper]
Amanda Ferrari Iaquinta and Gustavo Voltani von Atzingen.
A dataset containing equal amounts of human and ChatGPT written news was assembled and different natural processing language techniques were used to extract features from it that were used to train, validate and test three models built with different techniques. The best performance was produced by the Bidirectional Long Short Term Memory (LSTM) Neural Network model.
How to Detect AI-Generated Texts? [paper]
Trung Nguyen, Amartya Hatua and Andrew H. Sung.
Present a method to distinguish synthetically generated text (SGT) from human-written text (HWT). The method includes methods for dataset creation, feature engineering, dataset comparison, and result analysis.
A Simple yet Efficient Ensemble Approach for AI-generated Text Detection. [paper]
Harika Abburi, Kalyani Roy, Michael Suesserman, Nirmala Pudota, Balaji Veeramani, Edward Bowen and Sanmitra Bhattacharya.
Propose a simple yet efficient solution to this problem by ensembling predictions from multiple constituent LLMs. Study the influence that the training data from individual LLMs have on model performance. Find that substituting commercially-restrictive GPT data with data generated from other open language models is a feasible alternative when developing generative text detectors.
Detecting and Unmasking AI-Generated Texts through Explainable Artificial Intelligence using Stylistic Features. [paper] 
Aditya Shah, Prateek Ranka, Urmi Dedhia, Shruti Prasad, Siddhi Muni and Kiran Bhowmick.
The research delves into parameters such as syllable count, word length, sentence structure, functional word usage, and punctuation ratios to detect AI-generated text. Furthermore, the research integrates Explainable AI (xAI) techniques—LIME and SHAP—to enhance the interpretability of machine learning model predictions.
DEMASQ: Unmasking the ChatGPT Wordsmith. [paper] 
Kavita Kumari, Alessandro Pegoraro, Hossein Fereidooni and Ahmad-Reza Sadeghi.
Propose an effective ChatGPT detector named DEMASQ, which accurately identifies ChatGPT-generated content. The method addresses two critical factors: (i) the distinct biases in text composition observed in human- and machinegenerated content and (ii) the alterations made by humans to evade previous detection methods.
AuthentiGPT: Detecting Machine-Generated Text via Black-Box Language Models Denoising. [paper]
Zhen Guo and Shangdi Yu.
Present AuthentiGPT, an efficient classifier that distinguishes between machine-generated and human-written texts. Under the assumption that humanwritten text resides outside the distribution of machine-generated text, AuthentiGPT leverages a black-box LLM to denoise input text with artificially added noise, and then semantically compares the denoised text with the original to determine if the content is machine-generated.
How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detection. [paper] 
Ryuto Koike, Masahiro Kaneko and Naoaki Okazaki.
This paper found that current detectors are brittle to instruction changes in text generation and raised awareness of the need to ensure prompt diversity when creating a detection benchmark. Open-sourced the dataset consisting of LLM-generated essays via various instructions.
Word-Level Interpretation of Chatgpt Detector Based on Classification Contribution. [paper] 
Dekun Chen.
Present a method to show the contribution of words to model predictions. Specifically, this study considers information from classification weight vectors, semantic vectors, and embedded input word vectors for a more complete interpretation of detector LLM.
A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts. [paper]
Nafis Irtiza Tripto, Saranya Venkatraman, Dominik Macko, Robert Moro, Ivan Srba, Adaku Uchendu, Thai Le and Dongwon Lee.
Explore the diverse notions of authorship regarding paraphrasing in author attribution and AI text detection, incorporating philosophical concepts like the Ship of Theseus paradox. Suggest that authorship should be task-dependent, and we substantiate our results with relevant theoretical and philosophical perspectives.
Spot the Bot: Distinguishing Human-Written and Bot-Generated Texts Using Clustering and Information Theory Techniques. [paper] 
Vasilii Gromov and Quynh Nhu Dang.
Propose a bot identification algorithm that is based on unsupervised learning techniques and does not depend on a large amount of labelled data. By combining findings in semantic analysis by clustering (crisp and fuzzy) and information techniques, we construct a robust model that detects a generated text for different types of bot.
Beyond Turing: A Comparative Analysis of Approaches for Detecting Machine-Generated Text. [paper]
Muhammad Farid Adilazuarda, Nikolaos Nektarios Arkoulis and Oleksii Chumakov.
Present the IMGTB framework, which simplifies the benchmarking of machine-generated text detection methods by easy integration of custom (new) methods and evaluation datasets.
IMGTB: A Framework for Machine-Generated Text Detection Benchmarking. [paper] 
Michal Spiegel and Dominik Macko.
Offer an in-depth evaluation of three distinct methods: traditional shallow learning, Language Model (LM) fine-tuning, and Multilingual Model fine-tuning. These approaches are rigorously tested on a wide range of machine-generated texts, providing a benchmark of their competence in distinguishing between human-authored and machine-authored linguistic constructs.
Machine-Generated Text Detection using Deep Learning. [paper]
Raghav Gaggar, Ashish Bhagchandani and Harsh Oza.
Create a wellconstructed labeled dataset encompassing texts originated by both humans and ChatGPT (GPT3.5 turbo) from five distinct sources. Additionally, design and train multiple classifiers to differentiate between texts generated by humans and ChatGPT.
Token Prediction as Implicit Classification to Identify LLM-Generated Text. [paper]
Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh and Bhiksha Raj.
Instead of adding an additional classification layer to a base LM, we reframe the classification task as a next-token prediction task and directly fine-tune the base LM to perform it. Utilize the Text-to-Text Transfer Transformer (T5) model as the backbone for the experiments.
The Effectiveness of Free Software for Detecting AI-Generated Writing. [paper]
Gregory Price and Marc Sakellarios.
Explore the effectiveness of freely available AI detection software in identifying AI-generated content and its implications for teacher evaluation of student work. Compare our manual analysis with results from five different free AI writing detectors.
Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT. [paper]
Biru Zhu, Lifan Yuan, Ganqu Cui, Yangyi Chen, Chong Fu, Bingxiang He, Yangdong Deng, Zhiyuan Liu, Maosong Sun and Ming Gu.
Design a zero-shot black-box method for detecting LLM-generated texts. The key idea is to revise the text to be detected using the ChatGPT model. Our method is based on the intuition that the ChatGPT model will make fewer revisions to LLM-generated texts than it does to human-written texts, because the texts generated by LLMs are more in accord with the generation logic and statistical patterns learned by LLMs like ChatGPT.
Evaluating the Efficacy of Hybrid Deep Learning Models in Distinguishing AI-Generated Text. [paper]
Finbarrs Oketunji.
Investigate the use of cutting-edge hybrid deep learning models to accurately differentiate between AI-generated text and human writing. Combining sophisticated neural networks, the custom model enabled it to detect nuanced differences between AI and human content.
STADEE: STAtistics-based DEEp Detection of Machine Generated Text. [paper]
Zheng Chen and Huming Liu.
Introduce STADEE, a STAtistics-based DEEp detection method that integrates essential statistical features of text with a sequence-based deep classifier. Utilize various statistical features, such as the probability, rank, cumulative probability of each token, as well as the information entropy of the distribution at each position.
Classification of Human- and AI-Generated Texts for English, French, German, and Spanish. [paper]
Kristina Schaaff, Tim Schlippe and Lorenz Mindner.
analyze features to classify human- and AI-generated text for English, French, German and Spanish and compare them across languages. Investigate two scenarios: (1) The detection of text generated by AI from scratch, and (2) the detection of text rephrased by AI.
ALTA Shared Task 2023
Overview of the 2023 ALTA Shared Task: Discriminate between Human-Written and Machine-Generated Text. [paper]
Diego Molla, Haolan Zhan, Xuanli He and Qiongkai Xu.
A Prompt in the Right Direction: Prompt Based Classification of Machine-Generated Text Detection. [paper]
Rinaldo Gagiano and Lin Tian.
Automatic Detection of Machine-Generated Text Using Pre-Trained Language Models. [paper]
Yunhao Fang.
An Ensemble Based Approach To Detecting LLM-Generated Texts. [paper]
Ahmed EI-Sayed and Omar Nasr.
Feature-Level Ensemble Learning for Robust Synthetic Text Detection with DeBERTaV3 and XLM-RoBERTa. [paper]
Saman Sarker Joy and Tanusree Das Aishi.
Stacking the Odds: Transformer-Based Ensemble for AI-Generated Text Detection. [paper]
Duke Nguyen, Khaing Myat Noe Naing and Aditya Joshi.
HuRef: HUman-REadable Fingerprint for Large Language Models. [paper]
Boyi Zeng, Chenghu Zhou, Xinbing Wang and Zhouhan Lin.
Introduce HuRef, a human-readable fingerprint for LLMs that uniquely identifies the base model without exposing model parameters or interfering with training. The method generates a dog image as an identity fingerprint for an LLM, where the dog’s appearance strongly indicates the LLM’s base model. Specifically, if the LLM is adapted from another base model, the generated dog highly resembles that model; otherwise if trained independently from scratch, it exhibits a unique dog image distinct from other models.
Deep dive into language traits of AI-generated Abstracts. [paper] 
Vikas Kumar, Amisha Bharti, Devanshu Verma and Vasudha Bhatnagar.
Attempt to detect the Abstracts generated by ChatGPT, which are much shorter in length and bounded. Extract the text’s semantic and lexical properties and observe that traditional machine learning models can confidently detect these Abstracts.
Assaying on the Robustness of Zero-Shot Machine-Generated Text Detectors. [paper] 
Yi-Fan Zhang, Zhang Zhang, Liang Wang, Tieniu Tan and Rong Jin.
Existing zero-shot detectors, typically designed for specific tasks or topics, often assume uniform testing scenarios, limiting their practicality. Uncover a significant correlation between topics and detection performance. Delve into the influence of topic shifts on zero-shot detectors.
Assessing AI Detectors in Identifying AI-Generated Code: Implications for Education. [paper] 
Wei Hung Pan, Ming Jie Chok, Jonathan Leong Shan Wong, Yung Xin Shin, Yeong Shian Poon, Zhou Yang, Chun Yong Chong, David Lo and Mei Kuan Lim.
Collect a dataset comprising 5,069 samples, with each sample consisting of a textual description of a coding problem and its corresponding human-written Python solution codes. Results demonstrate that existing AIGC Detectors perform poorly in distinguishing between human-written code and AI-generated code.
Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers. [paper]
Yuling Shi, Hongyu Zhang, Chengcheng Wan and Xiaodong Gu.
Propose a novel machine-generated code detection method called DetectCodeGPT, which improves DetectGPT by capturing the distinct structural patterns of code. Diverging from conventional techniques that depend on external LLMs for perturbations, DetectCodeGPT perturbs the code corpus by strategically inserting spaces and newlines, ensuring both efficacy and efficiency.
Enhancing Robustness of LLM-Synthetic Text Detectors for Academic Writing: A Comprehensive Analysis. [paper]
Zhicheng Dou, Yuchen Guo, Ching-Chun Chang, Huy H. Nguyen and Isao Echizen.
Present a comprehensive analysis of the impact of prompts on the text generated by LLMs and highlight the potential lack of robustness in one of the current state-of-the-art GPT detectors. Propose a reference-based Siamese detector named Synthetic-Siamese which takes a pair of texts, one as the inquiry and the other as the reference.
Deciphering Textual Authenticity: A Generalized Strategy through the Lens of Large Language Semantics for Detecting Human vs. Machine-Generated Text. [paper]
Mazal Bethany, Brandon Wherry, Emet Bethany, Nishant Vishwamitra and Peyman Najafirad.
Undertake a systematic study on the detection of machine-generated text in real-world scenarios. Study the effectiveness of state-of-the-art approaches and find that they are severely limited against text produced by diverse generators and domains in the real world. Introduce a novel system, T5LLMCipher, for detecting machine-generated text using a pretrained T5 encoder combined with LLM embedding sub-clustering to address the text produced by diverse generators and domains in the real world.
ALMs: Authorial Language Models for Authorship Attribution. [paper]
Weihang Huang, Akira Murakami and Jack Grieve.
Introduce an authorship attribution method called Authorial Language Models (ALMs) that involves identifying the most likely author of a questioned document based on the perplexity of the questioned document calculated for a set of causal language models fine-tuned on the writings of a set of candidate author.
Fine-tuning Large Language Models for Multigenerator, Multidomain, and Multilingual Machine-Generated Text Detection. [paper]
Feng Xiong, Thanet Markchom, Ziwei Zheng, Subin Jung, Varun Ojha and Huizhi Liang.
Two methods: 1) using traditional machine learning (ML) with natural language preprocessing (NLP) for feature extraction, and 2) fine-tuning LLMs for text classification. The results show that transformer models, particularly LoRA-RoBERTa, exceed traditional ML methods in effectiveness, with majority voting being particularly effective in multilingual contexts for identifying machine-generated texts.
Beyond Lexical Boundaries: LLM-Generated Text Detection for Romanian Digital Libraries. [paper] 
Melania Nitu and Mihai Dascalu.
Introduce a corpus of approximately 60 k Romanian documents, including human-written samples as well as generated texts using six distinct Large Language Models (LLMs) and three different generation methods. Introduce two techniques for discerning machine-generated texts. The study also includes a text similarity analysis between human-authored and artificially generated texts, coupled with a SHAP analysis.
Does DetectGPT Fully Utilize Perturbation? Selective Perturbation on Model-Based Contrastive Learning Detector would be Better. [paper]
Shengchao Liu, Xiaoming Liu, Yichen Wang, Zehua Cheng, Chengzhengxu Li, Zhaohan Zhang, Yu Lan and Chao Shen.
DetectGPT's random perturbation strategy might introduce noise, and its logit regression module harms the generalizability and applicability of individual or small-batch inputs. Propose a novel detector, PECOLA, which uses selective strategy perturbation to relieve the important information loss caused by random masking, and multi-pair contrastive learning to capture the implicit pattern information during perturbation, facilitating few-shot performance.
LLM-Detector: Improving AI-Generated Chinese Text Detection with Open-Source LLM Instruction Tuning. [paper]
Rongsheng Wang, Haoming Chen, Ruizhe Zhou, Han Ma, Yaofei Duan, Yanlan Kang, Songhua Yang, Baoyu Fan and Tao Tan.
Create a dataset that mixed human-written sentences and sentences polished by LLMs. Propose LLM-Detector, a novel method for both document-level and sentence-level text detection through Instruction Tuning of LLMs. Our method leverages the wealth of knowledge LLMs acquire during pre-training, enabling them to detect the text they generate. Instruction tuning aligns the model's responses with the user's expected text detection tasks.
Detecting Generated Native Ads in Conversational Search. [paper] 
Sebastian Schmidt, Ines Zelch, Janek Bevendorff, Benno Stein, Matthias Hagen and Martin Potthast.
This paper investigates whether LLMs can be used as a countermeasure against generated native ads, i.e., to block them. Compile a large dataset of ad-prone queries and of generated answers with automatically integrated ads to experiment with fine-tuned sentence transformers and state-of-the-art LLMs on the task of recognizing the ads.
Ten Words Only Still Help: Improving Black-Box AI-Generated Text Detection via Proxy-Guided Efficient Re-Sampling. [paper]
Yuhui Shi, Qiang Sheng, Juan Cao, Hao Mi, Beizhe Hu and Danding Wang.
Propose to estimate word generation probabilities as pseudo white-box features via multiple re-sampling to help improve AIGT detection under the black-box setting. Design POGER, a proxy-guided efficient re-sampling method, which selects a small subset of representative words for performing multiple re-sampling in black-box AIGT detection.
Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs. [paper]
Zae Myung Kim, Kwang Hee Lee, Preston Zhu, Vipul Raheja and Dongyeop Kang.
This paper delves into the inquiry of identifying discernible and unique linguistic properties in texts that were written by humans, particularly uncovering the underlying discourse structures of texts beyond their surface structures. Introducing a novel methodology, we leverage hierarchical parse trees and recursive hypergraphs to unveil distinctive discourse patterns in texts produced by both LLMs and humans.
Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text. [paper]
Yafu Li, Zhilin Wang, Leyang Cui, Wei Bi, Shuming Shi and Yue Zhang.
Propose a novel detection framework, paraphrased text span detection (PTD), aiming to identify paraphrased text spans within a text. Different from text-level detection, PTD takes in the full text and assigns each of the sentences with a score indicating the paraphrasing degree. Construct a dedicated dataset, PASTED, for paraphrased text span detection.
Who is the Writer?Identifying the Generative Model by Writing Style. [paper] 
Anonymous authors.
Present the Identify the Writer by Writing Style (IWWS) model, a novel approach designed to identify the writing styles of human and generative model. Propose a comprehensive dataset, HumanGenTextify. The IWWS model not only achieves high accuracy in text source identification but also provides insights into the distinctive writing styles that characterize human and model-generated texts.
Machine-generated Text Localization. [paper]
Zhongping Zhang, Wenda Qin and Bryan A. Plummer.
Provide the first in-depth study of MGT that localizes the portions of a document that were machine generated. A key challenge in the MGT localization task is that short spans of text, e.g., a single sentence, provides little information indicating if it is machine generated due to its short length. Leverage contextual information, where we predict whether multiple sentences are machine or human written at once.
Whose LLM is it Anyway? Linguistic Comparison and LLM Attribution for GPT-3.5, GPT-4 and Bard. [paper]
Ariel Rosenfeld and Teddy Lazebnik.
Through a comprehensive linguistic analysis, compare the vocabulary, Part-Of-Speech (POS) distribution, dependency distribution, and sentiment of texts generated by three of the most popular LLMS today (GPT-3.5, GPT-4, and Bard) to diverse inputs. The results point to significant linguistic variations which, in turn, enable us to attribute a given text to its LLM origin with a favorable 88% accuracy using a simple off-the-shelf classification model.
Technical Report on the Checkfor.ai AI-Generated Text Classifier. [paper]
Bradley Emi and Max Spero.
Present the Checkfor.ai text classifier, a transformer-based neural network trained to distinguish text written by large language models from text written by humans. Propose a training algorithm, hard negative mining with synthetic mirrors, that enables the classifier to achieve orders of magnitude lower false positive rates on high-data domains such as reviews.
Machine-Generated Text Detection and Attribution. [paper]
Petr Šigut.
Analyse various ways to detect AI-generated content, particularly when encountering content in languages other than English. Test how three AI detection tools, Compilatio, Turnitin and GPT-2 Output Detector, perform with Czech, Slovak and English texts generated by ChatGPT.
Deep Learning Detection Method for Large Language Models-Generated Scientific Content. [paper]
Bushra Alhijawi, Rawan Jarrar, Aseel AbuAlRub and Arwa Bader.
Present a novel ChatGPT-generated scientific text detection method, AI-Catcher. AI-Catcher integrates two deep learning models, multilayer perceptron (MLP) and convolutional neural networks (CNN). In addition, a new ChatGPT-Generated scientific text dataset is collected to enhance AI-generated text detection tools, AIGTxt.
Towards Detecting AI-Generated Text within Human-AI Collaborative Hybrid Texts. [paper]
Zijie Zeng, Shiqi Liu, Lele Sha, Zhuang Li, Kaixun Yang, Sannyuya Liu, Dragan Gašević and Guanliang Chen.
Explore the challenge of sentence-level AI-generated text detection within human-AI collaborative hybrid texts. Utilize the CoAuthor dataset, which includes diverse, realistic hybrid texts generated through the collaboration between human writers and an intelligent writing system in multi-turn interactions. Adopt a two-step, segmentation-based pipeline: (i) detect segments within a given hybrid text where each segment contains sentences of consistent authorship, and (ii) classify the authorship of each identified segment.
Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection. [paper]
Zhixin Lai, Xuesheng Zhang and Suiyao Chen.
Reveal that single transformer-based classifiers achieved decent performance on in-distribution dataset but limited generalization ability on out-of-distribution dataset. To improve it, combine the individual classifiers models using adaptive ensemble algorithms. The results indicate the effectiveness, good generalization ability, and great potential of adaptive ensemble algorithms in LLM-generated text detection.
EAGLE: A Domain Generalization Framework for AI-generated Text Detection. [paper]
Amrita Bhattacharjee, Raha Moraffah, Joshua Garland and Huan Liu.
Propose a domain generalization framework for the detection of AI-generated text from unseen target generators. The proposed framework, EAGLE, leverages the labeled data that is available so far from older language models and learns features invariant across these generators, in order to detect text generated by an unknown target generator. EAGLE learns such domain-invariant features by combining the representational power of self-supervised contrastive learning with domain adversarial training.
Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models. [paper]
Zhenyu Xu and Victor S. Sheng.
Address the challenge of determining whether a student's code assignment was generated by a language model. First, the proposed method identifies AI-generated code by leveraging targeted masking perturbation paired with comperhesive scoring. Second, utilize a fine-tuned CodeBERT to fill in the masked portions, producing subtle modified samples. Then, integrate the overall perplexity, variation of code line perplexity, and burstiness into a unified score. In this scoring scheme, a higher rank for the original code suggests it's more likely to be AI-generated.
MUGC: Machine Generated versus User Generated Content Detection. [paper]
Yaqi Xie, Anjali Rawal, Yujing Cen, Dixuan Zhao, Sunil K Narang and Shanu Sushmita.
Undertake a comparative evaluation of eight traditional machine-learning algorithms to distinguish between machine-generated and human-generated data across three diverse datasets: Poems, Abstracts, and Essays.
AI-Generated Text Detector for Arabic Language Using Encoder-Based Transformer Architecture. [paper] 
Hamed Alshammari,Ahmed El-Sayed and Khaled Elleithy.
Introduce a novel AI text classifier designed specifically for Arabic. Utilized and fine-tuned two Transformer-based models, AraELECTRA and XLM-R, by training them on two distinct datasets: a large dataset comprising 43,958 examples and a custom dataset with 3078 examples that contain HWT and AI-generated texts (AIGTs) from various sources, including ChatGPT 3.5, ChatGPT-4, and BARD.
The Impact of Prompts on Zero-Shot Detection of AI-Generated Text. [paper]
Kaito Taguchi, Yujie Gu and Kouichi Sakurai.
Introduce an evaluative framework to empirically analyze the impact of prompts on the detection accuracy of AI-generated text. Assess various zero-shot detectors using both white-box detection, which leverages the prompt, and black-box detection, which operates without prompt information. Experiments reveal the significant influence of prompts on detection accuracy.
Detection of AI-Generated Text Using Large Language Model. [paper] 
Manish Prajapati, Santos Kumar Baliarsingh, Chinmayee Dora, Ashutosh Bhoi, Jhalak Hota and Jasaswi Prasad Mohanty.
Leverage the machine learning (ML) models to reliably identify whether an essay is authored by a human being or by an LLM. Using diverse texts and unknown generative models, we replicate typical scenarios to encourage feature learning across models.
Sentiment analysis and random forest to classify LLM versus human source applied to Scientific Texts. [paper]
Javier J. Sanchez-Medina.
Propose a new methodology to classify texts coming from an automatic text production engine or a human, based on Sentiment Analysis as a source for feature engineering independent variables and then train with them a Random Forest classification algorithm. Using four different sentiment lexicons, a number of new features where produced, and then fed to a machine learning random forest methodology, to train such a model.
Large Language Model (LLM) AI text generation detection based on transformer deep learning algorithm. [paper]
Yuhong Mo, Hao Qin, Yushan Dong, Ziyi Zhu and Zhenglin Li.
In this paper, a tool for detecting LLM AI text generation is developed based on the Transformer model, aiming to improve the accuracy of AI text generation detection and provide reference for subsequent research.
Detecting AI Generated Text Based on NLP and Machine Learning Approaches. [paper]
Nuzhat Prova.
Offer an accurate AI detector model that can differentiate between electronically produced text and human-written text. The approach includes machine learning methods such as XGB Classifier, SVM, BERT architecture deep learning models.
EDMAL - ENHANCED DETECTION OF AI GENERATED TEXT USING MACHINE LEARNING. [paper] 
Ang Jun Ray and Chieu Hai Leong.
Aim to enhance existing AI text detection technologies and thus focused on the Divergent N-Gram Analysis’s (DNA-GPT’s) approach. Propose 4 methods to replace the 3rd step in DNA-GPT’s detection process, where texts are compared and scored. The results indicate that the machine-learning based methods, especially the random forest classifier, outperforms the original approach by DNA-GPT.
C-Net: A Compression-Based Lightweight Network for Machine-Generated Text Detection. [paper] 
Yinghan Zhou, Juan Wen, Jianghao Jia, Liting Gao and Ziwei Zhang.
Propose a compression-based lightweight network for MGT detection that leverages the ability of lossless compression to effectively extract features between categories. With fewer parameters, the framework achieves state-of-the-art performance in MGT detection under black box conditions.
Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore. [paper]
Junchao Wu, Runzhe Zhan, Derek F. Wong, Shu Yang, Xuebo Liu, Lidia S. Chao and Min Zhang.
Propose an simple but effective black-box zero-shot detection approach, predicated on the observation that human-written texts typically contain more grammatical errors than LLM-generated texts. This ap
Truncated — view the full README on GitHub.
19 followers · starred Jul 2023
39 followers · starred Apr 2024
210 followers · starred Jan 2024
6 followers · starred Oct 2024
Continuously updated list of related resources for generative LLMs like GPT and their analysis and detection.
231
217 commits
updated May 28, 2025
Note: #params > 1B only
OpenAI
(GPT-2) Language Models are Unsupervised Multitask Learners. [paper] [blog]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever.
(GPT-3) Language Models are Few-Shot Learners. [paper]
WebGPT: Browser-assisted question-answering with human feedback. [paper] [blog]
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess and John Schulman.
(InstructGPT) Training language models to follow instructions with human feedback. [paper] [blog]
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike and Ryan Lowe.
ChatGPT: Optimizing Language Models for Dialogue. [blog]
(GPT-4) GPT-4 Technical Report. [paper] [blog] [blog]
OpenAI.
DeepMind
(Gopher) Scaling Language Models: Methods, Analysis & Insights from Training Gopher. [paper] [blog]
(RETRO) Improving language models by retrieving from trillions of tokens. [paper] [blog]
(Chinchilla) Training Compute-Optimal Large Language Models. [paper] [blog]
(Sparrow) Improving alignment of dialogue agents via targeted human judgements. [paper] [blog]
Nvidia & Microsoft
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. [paper]
Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper and Bryan Catanzaro.
Turing-NLG: A 17-billion-parameter language model by Microsoft. [blog]
(Megatron-Turing NLG) Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model. [paper] [blog] [blog]
(Meena) Towards a Human-like Open-Domain Chatbot. [paper] [blog]
Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu and Quoc V. Le.
(T5) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. [paper] [blog]
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li and Peter J. Liu.
mT5: A massively multilingual pre-trained text-to-text transformer. [paper]
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua and Colin Raffel.
(FLAN) Finetuned Language Models Are Zero-Shot Learners. [paper] [blog]
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai and Quoc V. Le.
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts. [paper] [slides] [blog]
LaMDA: Language Models for Dialog Applications. [paper] [blog]
PaLM: Scaling Language Modeling with Pathways. [paper] [blog]
UL2: Unifying Language Learning Paradigms. [paper] [blog]
Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Denny Zhou, Neil Houlsby and Donald Metzler.
(Flan-T5 & Flan-PaLM & Flan-U-PaLM) Scaling Instruction-Finetuned Language Models. [paper] [blog]
Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Denny Zhou, Neil Houlsby and Donald Metzler.
Meta
(Generative BST) Recipes for Building an Open-Domain Chatbot. [paper]
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan Boureau and Jason Weston.
(XGLM) Few-shot Learning with Multilingual Language Models. [paper]
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O'Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov and Xian Li.
OPT: Open Pre-trained Transformer Language Models. [paper]
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang and Luke Zettlemoyer.
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage. [paper] [blog]
Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, Morteza Behrooz, William Ngan, Spencer Poff, Naman Goyal, Arthur Szlam, Y-Lan Boureau, Melanie Kambadur and Jason Weston.
Atlas: Few-shot Learning with Retrieval Augmented Language Models. [paper]
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel and Edouard Grave.
LLaMA: Open and Efficient Foundation Language Models. [paper] [blog]
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave and Guillaume Lample.
BigScience
(T0++) Multitask Prompted Training Enables Zero-Shot Task Generalization. [paper]
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. [paper]
Amazon
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model. [paper] [blog]
Saleh Soltan, Shankar Ananthakrishnan, Jack FitzGerald, Rahul Gupta, Wael Hamza, Haidar Khan, Charith Peris, Stephen Rawls, Andy Rosenbaum, Anna Rumshisky, Chandana Satya Prakash, Mukund Sridhar, Fabian Triefenbach, Apurv Verma, Gokhan Tur and Prem Natarajan.
AI21 Labs
Baidu
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation. [paper]
Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen, Yanbin Zhao, Yuxiang Lu, Weixin Liu, Zhihua Wu, Weibao Gong, Jianzhong Liang, Zhizhou Shang, Peng Sun, Wei Liu, Xuan Ouyang, Dianhai Yu, Hao Tian, Hua Wu and Haifeng Wang.
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation. [paper]
PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation. [paper]
Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhihua Wu, Zhen Guo, Hua Lu, Xinxian Huang, Xin Tian, Xinchao Xu, Yingzhan Lin and Zheng-Yu Niu.
ERNIE Bot [news]
Anthropic
(Anthropic-52) Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback [paper]
(Claude) Constitutional AI: Harmlessness from AI Feedback. [paper]
EleutherAI
GPT-J-6B: 6B JAX-Based Transformer. [blog]
GPT-NeoX-20B: An Open-Source Autoregressive Language Model. [paper] [blog]
Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, Samuel Weinbach.
Tsinghua University
CPM: A Large-scale Generative Chinese Pre-trained Language Model. [paper]
Zhengyan Zhang, Xu Han, Hao Zhou, Pei Ke, Yuxian Gu, Deming Ye, Yujia Qin, Yusheng Su, Haozhe Ji, Jian Guan, Fanchao Qi, Xiaozhi Wang, Yanan Zheng, Guoyang Zeng, Huanqi Cao, Shengqi Chen, Daixuan Li, Zhenbo Sun, Zhiyuan Liu, Minlie Huang, Wentao Han, Jie Tang, Juanzi Li, Xiaoyan Zhu, Maosong Sun.
CPM-2: Large-scale Cost-effective Pre-trained Language Models. [paper]
Zhengyan Zhang, Yuxian Gu, Xu Han, Shengqi Chen, Chaojun Xiao, Zhenbo Sun, Yuan Yao, Fanchao Qi, Jian Guan, Pei Ke, Yanzheng Cai, Guoyang Zeng, Zhixing Tan, Zhiyuan Liu, Minlie Huang, Wentao Han, Yang Liu, Xiaoyan Zhu, Maosong Sun.
Chinese-Transformer-XL [repo]
EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Training. [paper]
Hao Zhou, Pei Ke, Zheng Zhang, Yuxian Gu, Yinhe Zheng, Chujie Zheng, Yida Wang, Chen Henry Wu, Hao Sun, Xiaocong Yang, Bosi Wen, Xiaoyan Zhu, Minlie Huang, Jie Tang.
EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Training. [paper]
Yuxian Gu, Jiaxin Wen, Hao Sun, Yi Song, Pei Ke, Chujie Zheng, Zheng Zhang, Jianzhu Yao, Xiaoyan Zhu, Jie Tang, Minlie Huang.
GLM: General Language Model Pretraining with Autoregressive Blank Infilling. [paper]
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, Jie Tang.
GLM-130B: An Open Bilingual Pre-trained Model. [paper]
Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Zhiyuan Liu, Peng Zhang, Yuxiao Dong and Jie Tang.
ChatGLM. [blog]
IDEA
Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence. [paper] [blog]
Junjie Wang, Yuxiang Zhang, Lin Zhang, Ping Yang, Xinyu Gao, Ziwei Wu, Xiaoqun Dong, Junqing He, Jianheng Zhuo, Qi Yang, Yongfeng Huang, Xiayu Li, Yanghan Wu, Junyu Lu, Xinyu Zhu, Weifeng Chen, Ting Han, Kunhao Pan, Rui Wang, Hao Wang, Xiaojun Wu, Zhongshen Zeng, Chongpei Chen, Ruyi Gan and Jiaxing Zhang.
Huawei
PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation. [paper]
PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained Language Model. [paper]
Fei Mi, Yitong Li, Yulong Zeng, Jingyan Zhou, Yasheng Wang, Chuanfei Xu, Lifeng Shang, Xin Jiang, Shiqi Zhao and Qun Liu.
Inspur
WeChat AI
WeLM: A Well-Read Pre-trained Language Model for Chinese. [paper]
Hui Su, Xiao Zhou, Houjin Yu, Yuwen Chen, Zilin Zhu, Yang Yu and Jie Zhou.
Holistic Evaluation of Language Models. [paper] [blog]
Benchmark 30 prominent language models across a wide range of scenarios and for a broad range of metrics to elucidate their capabilities and risks.
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 [paper]
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and Shmargaret Shmitchell.
Identified a wide variety of costs and risks associated with the rush for ever larger LMs, including: environmental costs, financial costs, opportunity cost and the risk of substantial harms.
TruthfulQA: Measuring How Models Mimic Human Falsehoods. [paper]
Stephanie Lin, Jacob Hilton and Owain Evans.
Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful.
How Much Knowledge Can You Pack Into the Parameters of a Language Model? [paper]
Adam Roberts, Colin Raffel and Noam Shazeer.
The maximum-likelihood objective used to train our model provides no guarantees as to whether a model will learn a fact or not. This makes it difficult to ensure that the model obtains specific knowledge over the course of pre-training and prevents us from explicitly updating or removing knowledge from a pre-trained model.
On the Opportunities and Risks of Foundation Models. [paper]
This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities and technical principles to their applications and societal impact.
Survey of Hallucination in Natural Language Generation. [paper]
Benchmark 30 prominent language models across a wide range of scenarios and for a broad range of metrics to elucidate their capabilities and risks.
On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models? [paper]
Nouha Dziri, Sivan Milton, Mo Yu, Osmar Zaiane and Siva Reddy.
Reveals that the standard benchmarks consist of >60% hallucinated responses, leading to models that not only hallucinate but even amplify hallucinations..
Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations. [paper]
Josh A. Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel and Katerina Sedova.
This report aims to assess: how might language models change influence operations, and what steps can be taken to mitigate these threats? This task is inherently speculative, as both AI and influence operations are changing quickly.
Forecasting Potential Misuses of Language Models for Disinformation Campaigns—and How to Reduce Risk. [blog]
OpenAI researchers collaborated with Georgetown University’s Center for Security and Emerging Technology and the Stanford Internet Observatory to investigate how large language models might be misused for disinformation purposes.
ChatGPT Wrote a Terrible Gizmodo Article. [blog]
ChatGPT kept including incorrect information in its explainer—sometimes mixing up basic facts about the history of its own technology.
Temporary policy: ChatGPT is banned. [blog]
While the answers which ChatGPT produces have a high rate of being incorrect, they typically look like they might be good and the answers are very easy to produce.
On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning. [paper]
Omar Shaikh, Hongxin Zhang, William Held, Michael Bernstein and Diyi Yang.
Find that using zero-shot CoT reasoning in a prompt can significantly increase a model’s likelihood to produce undesirable output.
The Woman Worked as a Babysitter: On Biases in Language Generation. [paper]
Emily Sheng, Kai-Wei Chang, Premkumar Natarajan and Nanyun Peng.
A systematic study of biases in natural language generation (NLG) by analyzing text generated from prompts that contain mentions of different demographic groups.
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. [paper]
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi and Noah A. Smith.
Using RealToxicityPrompts, we find that pretrained LMs can degenerate into toxic text even from seemingly innocuous prompts.
OpenAI's new ChatGPT bot: 10 dangerous things it's capable of. [blog]
As the erudite machinery turns into a viral sensation, humans have started to discover some of the AI's biases, like the desire to wipe out humanity.
OpwnAI: AI That Can Save the Day or HACK it Away. [blog]
AI models can be used to create a full infection flow, from spear-phishing to running a reverse shell.
OpwnAI : Cybercriminals Starting to Use ChatGPT. [blog]
There are already instances of cybercriminals using OpenAI to develop malicious tools.
Security risks of ChatGPT and other AI text generators. [blog]
Successfully asked ChatGPT to write a convincing phishing email and to create JavaScript that could be used to steal personal information.
The security threat of AI-powered cyberattacks. [paper]
Matti Aksela, Samuel Marchal, Andrew Patel, Lina Rosenstedt and WithSecure.
Investigate the security threat of AI-enabled cyberattacks by summarising current knowledge on the topic.
How hackers might be exploiting ChatGPT. [blog]
The viral AI chatbot ChatGPT might advise threat actors how to hack into networks with ease.
Extracting Training Data from Large Language Models. [paper]
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea and Colin Raffel.
This paper demonstrates that an adversary can perform a training data extraction attack to recover individual training examples by querying the language model.
Ignore Previous Prompt: Attack Techniques For Language Models. [paper]
Fábio Perez and Ian Ribeiro.
Study prompt injection attacks against LLMs and propose a framework to explore such attacks; Investigate two specific attacks: goal hijacking and prompt leaking.
Carbon Emissions and Large Neural Network Training. [paper]
David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier and Jeff Dean.
Calculate the energy use and carbon footprint of several recent large models-T5, Meena, GShard, Switch Transformer, and GPT-3.
Automatic Detection of Machine Generated Text: A Critical Survey. [paper]
Ganesh Jawahar, Muhammad Abdul-Mageed and Laks V.S. Lakshmanan.
在线社交网络文本内容对抗技术. [paper]
刘晓明, 张兆晗, 杨晨阳, 张宇辰, 沈超, 周亚东 and 管晓宏.
Synthetic Text Detection: Systemic Literature Review. [paper]
Jesus Guerrero, Izzat Alsmadi.
Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods. [paper]
Evan Crothers, Nathalie Japkowicz and Herna Viktor.
Deepfake Text Detection: Limitations and Opportunities. [paper]
Jiameng Pu, Zain Sarwar, Sifat Muhammad Abdullah, Abdullah Rehman, Yoonjin Kim, Parantapa Bhattacharya, Mobin Javed and Bimal Viswanath.
The Science of Detecting LLM-Generated Texts. [paper]
Ruixiang Tang, Yu-Neng Chuang and Xia Hu.
To ChatGPT, or not to ChatGPT: That is the question! [paper]
Alessandro Pegoraro, Kavita Kumari, Hossein Fereidooni and Ahmad-Reza Sadeghi.
The Age of Synthetic Realities: Challenges and Opportunities. [paper]
João Phillipe Cardenuto, Jing Yang, Rafael Padilha, Renjie Wan, Daniel Moreira, Haoliang Li, Shiqi Wang, Fernanda Andaló, Sébastien Marcel and Anderson Rocha.
Attribution and Obfuscation of Neural Text Authorship: A Data Mining Perspective. [paper]
Adaku Uchendu, Thai Le and Dongwon Lee.
Detecting Artificial Intelligence: A New Cyberarms Race Begins. [paper]
Mark Campbell and Mlađan Jovanović.
Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text. [paper]
Mahdi Dhaini, Wessel Poelman and Ege Erdogan.
A Survey on LLM-gernerated Text Detection: Necessity, Methods, and Future Directions. [paper]
Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Derek F. Wong and Lidia S. Chao.
Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey. [paper]
Soumya Suvra Ghosal, Souradip Chakraborty, Jonas Geiping, Furong Huang, Dinesh Manocha and Amrit Singh Bedi.
A Survey on Detection of LLMs-Generated Content. [paper]
Xianjun Yang, Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Petzold, William Yang Wang and Wei Cheng.
A Survey of Text Watermarking in the Era of Large Language Models. [paper]
Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Lijie Wen, Irwin King and Philip S. Yu.
Detection of Machine-Generated Text: Literature Survey. [paper]
Dmytro Valiaiev.
Detecting Multimedia Generated by Large AI Models: A Survey. [paper]
Li Lin, Neeraj Gupta, Yue Zhang, Hainan Ren, Chun-Hao Liu, Feng Ding, Xin Wang, Xin Li, Luisa Verdoliva and Shu Hu.
Copyright Protection in Generative AI: A Technical Perspective. [paper]
Jie Ren, Han Xu, Pengfei He, Yingqian Cui, Shenglai Zeng, Jiankun Zhang, Hongzhi Wen, Jiayuan Ding, Hui Liu, Yi Chang and Jiliang Tang.
Beyond the Human Eye: Comprehensive Approaches to AI Text Detection. [paper]
Jamal Goddard, Yuksel Celik and Sanjay Goel.
A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterization. [paper]
Tharindu Kumarage, Garima Agrawal, Paras Sheth, Raha Moraffah, Aman Chadha, Joshua Garland and Huan Liu.
Decoding the AI Pen: Techniques and Challenges in Detecting AI-Generated Text. [paper]
Sara Abdali, Richard Anarfi, CJ Barberan and Jia He.
Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods. [paper]
Kathleen C. Fraser, Hillary Dawkins and Svetlana Kiritchenko.
From Intentions to Techniques: A Comprehensive Taxonomy and Challenges in Text Watermarking for Large Language Models. [paper]
Harsh Nishant Lalai, Aashish Anantha Ramakrishnan, Raj Sanjay Shah and Dongwon Lee.
Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integrity. [paper]
Shushanta Pudasaini, Luis Miralles-Pechuán, David Lillis and Marisa Llorens Salvador.
Survey for Detecting AI-generated Content. [paper]
Yu Wang, Ziyan Wang.
Building Intelligence Identification System via Large Language Model Watermarking: A Survey and Beyond. [paper]
Xuhong Wang, Haoyu Jiang, Yi Yu, Jingru Yu, Yilun Lin, Ping Yi, Yingchun Wang, Yu Qiao, Li Li and Fei-Yue Wang.
Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges. [paper]
Baixiang Huang, Canyu Chen and Kai Shu.
SoK: Watermarking for AI-Generated Content. [paper]
Xuandong Zhao, Sam Gunn, Miranda Christ, Jaiden Fairoze, Andres Fabrega, Nicholas Carlini, Sanjam Garg, Sanghyun Hong, Milad Nasr, Florian Tramer, Somesh Jha, Lei Li, Yu-Xiang Wang and Dawn Song.
Deepfake Bot Submissions to Federal Public Comment Websites Cannot Be Distinguished from Human Submissions. [paper]
Max Weiss.
Generate and submit 1,001 deepfake comments regarding a Medicaid reform waiver to a federal public comment website. The human classification results are no better than would have been gotten by random guessing.
RoFT: A Tool for Evaluating Human Detection of Machine-Generated Text. [paper]
Liam Dugan, Daphne Ippolito, Arun Kirubarajan and Chris Callison-Burch.
Develop the Real or Fake Text (RoFT) system, a novel application for simultaneously collecting quality annotations of machinegenerated text while allowing the public to assess and improve their skill at detecting machinegenerated text.
All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text. [paper]
Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan and Noah A. Smith.
Assess nonexperts’ ability to distinguish between humanand machine-authored text, and test three evaluator-training methods to see if we could improve people’s ability to identify machinegenerated text.
Human Heuristics for AI-Generated Language Are Flawed. [paper]
Maurice Jakesch, Jeffrey Hancock and Mor Naaman.
Study how humans discern whether verbal self-presentations were generated by AI. Show that human judgments of AI-generated language are handicapped by intuitive but flawed heuristics.
Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Text. [paper]
Liam Dugan, Daphne Ippolito, Arun Kirubarajan, Sherry Shi and Chris Callison-Burch.
Analyze how a variety of variables (model size, decoding strategy, fine-tuning, prompt genre, etc.) affect human detection performance.
AI model GPT-3 (dis)informs us better than humans. [paper]
Giovanni Spitale, Nikola Biller-Andorno and Federico Germani.
Evaluate whether recruited individuals can distinguish disinformation from accurate information, and determine whether a tweet has been written by a Twitter user or by GPT-3.
Creating a Large Language Model of a Philosopher. [paper]
Eric Schwitzgebel, David Schwitzgebel and Anna Strasser.
Fine-tuned the GPT-3 on the corpus of Daniel Dennett, then asked it a series of philosophical questions. Ordinary research participants untrained in philosophy were at or near chance in distinguishing GPT-3’s answers from those of an “actual human philosopher”. .
Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts? [paper]
Adaku Uchendu, Jooyoung Lee, Hua Shen, Thai Le, Ting-Hao 'Kenneth' Huang and Dongwon Lee.
Find that: (1) expert humans detect deepfake texts significantly better than non-expert humans, (2) synchronous teams on the Upwork detect deepfake texts significantly better than individuals, while asynchronous teams on the AMT detect deepfake texts weakly better than individuals, and (3) among various error categories, examining coherence and consistency in texts is useful in detecting deepfake texts.
Too Good to Be True? An Empirical Study of ChatGPT Capabilities for Academic Writing and Implications for Academic Misconduct. [paper]
Peter André Busch and Geir Inge Hausvik.
Ask 15 faculty members to assess a total of ten answers (two each) to an actual IS exam question, of which students wrote five, and ChatGPT generated five. Find that ChatGPT can generate answers of generally good quality that may pass as human-written text by examiners.
Game of Tones: Faculty detection of GPT-4 generated content in university assessments. [paper]
Mike Perkins, Jasper Roe, Darius Postma, James McGaughran and Don Hickerson.
This study explores the robustness of university assessments against the use of Open AI's Generative Pre-Trained Transformer 4 (GPT-4) generated content and evaluates the ability of academic staff to detect its use when supported by the Turnitin Artificial Intelligence (AI) detection tool.
The Unseen A+ Student: Navigating the Impact of Large Language Models in the Classroom. [paper]
Matyas Bohacek.
Collect a dataset of authentic high school coursework and generate their AI alternatives and text continuations using ChatGPT with 4.0, 3.5, and 3.5 Legacy backbones. Through a study involving student peers, we found that ChatGPT can quickly produce high-school-level coursework that peers consider better than human-written text, even in a low-resourced language like Czech. Moreover, show that the AI text detectors fail to identify these texts in Czech.
Can linguists distinguish between ChatGPT/AI and human writing?: A study of research ethics and academic publishing. [paper]
J. Elliott Casal and Matt Kessler.
Investigate: 1) the extent to which linguists/reviewers from top journals can distinguish AI- from human-generated writing, 2) what the basis of reviewers’ decisions are, and 3) the extent to which editors of top Applied Linguistics journals believe AI tools are ethical for research purposes.
Detection of GPT-4 Generated Text in Higher Education: Combining Academic Judgement and Software to Identify Generative AI Tool Misuse. [paper]
Mike Perkins, Jasper Roe, Darius Postma, James McGaughran and Don Hickerson.
Explore the capability of academic staff assisted by the Turnitin Artificial Intelligence (AI) detection tool to identify the use of AI-generated content in university assessments.
ChatGPT versus Human Essayists: An Exploration of the Impact of Artificial Intelligence for Authorship and Academic Integrity in the Humanities. [paper]
Tom Revell, Will Yeadon, Glenn Cahilly-Bretzin, Isabella Clarke, George Manning, Jasmine Jones, Clare Mulley, Rafael Pascual, Natasha Bradley, Daniel Thomas and Francis Leneghan.
Examine AI’s ability to write essays analysing Old English poetry; human markers assessed and attempted to distinguish them from authentic analyses of poetry by first-year undergraduate students in English at the University of Oxford.
Evaluating AI and Human Authorship Quality in Academic Writing through Physics Essays. [paper]
Will Yeadon, Elise Agra, Oto-obong Inyang, Paul Mackay and Arin Mizouri.
Evaluate n=300 short-form physics essay submissions, equally divided between student work submitted before the introduction of ChatGPT and those generated by OpenAI's GPT-4. In blinded evaluations conducted by five independent markers who were unaware of the origin of the essays, we observed no statistically significant differences in scores between essays authored by humans and those produced by AI (p-value =0.107, α = 0.05).
A feasibility study for the application of AI-generated conversations in pragmatic analysis. [paper]
Xi Chen, Jun Li and Yuting Ye.
Compare 148 ChatGPT-generated conversations with 82 human-written ones and 354 human evaluations of these conversations. The data are analysed using various methods, including traditional speech strategy coding, four computational methods developed in NLP, and four statistical tests.
People cannot distinguish GPT-4 from a human in a Turing test. [paper]
Cameron R. Jones and Benjamin K. Bergen.
Evaluate 3 systems (ELIZA, GPT-3.5 and GPT-4) in a randomized, controlled, and preregistered Turing test. Human participants had a 5 minute conversation with either a human or an AI, and judged whether or not they thought their interlocutor was human. GPT-4 was judged to be a human 54% of the time, outperforming ELIZA (22%) but lagging behind actual humans (67%).
Can human intelligence safeguard against artificial intelligence? Exploring individual differences in the discernment of human from AI texts. [paper]
Jason Chein, Steven Martinez and Alexander Barone.
Examine individual differences in the human ability to differentiate human- from AI-generated texts, exploring relationships with fluid intelligence, executive functioning, empathy, and digital habits. Overall, participants exhibited better than chance text discrimination, with substantial variation across individuals.
Limitations of Human Identification of Automatically Generated Text. [paper]
Nadège Alavoine, Maximin Coavoux, Emmanuelle Esperança-Rodier, Romane Gallienne, Carlos Gonzalez Gallardo, Jérôme Goulian, Jose G. Moreno, Aurélie Névéol, Didier Schwab, Vincent Segonne and Johanna Simoens.
Propose a new corpus in French and English for the task of recognising automatically generated texts and conduct a study of how humans perceive the text. The results show that the generated texts by tools such as ChatGPT share some common characteristics but they are not clearly identifiable which generates different perceptions of these texts.
Seeing Through AI's Lens: Enhancing Human Skepticism Towards LLM-Generated Fake News. [paper]
Navid Ayoobi, Sadat Shahriar and Arjun Mukherjee.
By providing cues in human-written and LLM-generated news, we can help individuals increase their skepticism towards fake LLM-generated news. This paper aims to elucidate simple markers that help individuals distinguish between articles penned by humans and those created by LLMs. Devise a metric named Entropy-Shift Authorship Signature (ESAS) based on the information theory and entropy principles.
GPT-4 is judged more human than humans in displaced and inverted Turing tests. [paper]
Ishika Rathi, Sydney Taylor, Benjamin K. Bergen and Cameron R. Jones.
Measure how well people and large language models can discriminate using two modified versions of the Turing test: inverted and displaced. GPT-3.5, GPT-4, and displaced human adjudicators judged whether an agent was human or AI on the basis of a Turing test transcript. Suggest that both humans and current LLMs struggle to distinguish between the two when they are not actively interrogating the person, underscoring an urgent need for more accurate tools to detect AI in conversations.
Defending Against Neural Fake News. [paper]
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner and Yejin Choi.
Present a model for controllable text generation called Grover. Find that the best way to detect neural fake news is to use a model that is also a generator.
GLTR: Statistical Detection and Visualization of Generated Text. [paper]
Sebastian Gehrmann, Hendrik Strobelt and Alexander M. Rush.
A Giant Language model Test Room. GLTR aims to both teach users what to be aware of when assessing whether a text is real, and to assist them in performing forensic analyses.
Real or Fake? Learning to Discriminate Machine from Human Generated Text. [paper]
Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc'Aurelio Ranzato and Arthur Szlam.
Use Energy-based models to discriminate real text from text generated by the auto-regressive models.
The Limitations of Stylometry for Detecting Machine-Generated Fake News. [paper]
Tal Schuster, Roei Schuster, Darsh J Shah and Regina Barzilay.
Examine the state-of-the-art stylometry model, and find it effective in preventing impersonation, but limited in detecting LM-generated misinformation.
Automatic Detection of Generated Text is Easiest when Humans are Fooled. [paper]
Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch and Douglas Eck.
A comprehensive study of generated text detection systems’ sensitivity to model structure, decoding strategy, and excerpt length.
Reverse Engineering Configurations of Neural Text Generation Models. [paper]
Yi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler and Andrew Tomkins.
Propose the new task of distinguishing which of several variants (e.g., sampling methods, top-k probabilities, model architectures, etc.) of a given model generated some piece of text. Find that detectable artifacts are present and that different modeling choices can be inferred by looking at generated text alone.
TweepFake: about Detecting Deepfake Tweets. [paper]
Tiziano Fagni, Fabrizio Falchi, Margherita Gambini, Antonio Martella and Maurizio Tesconi.
Collect the first dataset of real deepfake tweets, TweepFake. Evaluate 13 deepfake text detection methods.
Identifying Automatically Generated Headlines using Transformers. [paper]
Antonis Maronikolakis, Hinrich Schutze and Mark Stevenson.
Create a dataset containing human and computer-generated headlines. Transformers achieved an overall accuracy of 85.7%.
Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News. [paper]
Reuben Tan, Bryan A. Plummer and Kate Saenko.
Present DIDAN, a approach which exploits possible semantic inconsistencies between the text and image/captions to detect machine-generated articles.
Neural Deepfake Detection with Factual Structure of Text. [paper]
Wanjun Zhong, Duyu Tang, Zenan Xu, Ruize Wang, Nan Duan, Ming Zhou, Jiahai Wang and Jian Yin.
Propose a graph-based model that utilizes the factual structure of a document for deepfake detection of text.
Authorship Attribution for Neural Text Generation. [paper]
Adaku Uchendu, Thai Le, Kai Shu and Dongwon Lee.
Investigate the authorship attribution problem in three versions: (1) given two texts T1 and T2, are both generated by the same method or not? (2) is the given text T written by a human or machine? (3) given a text T and k candidate neural methods, can we single out the method (among k alternatives) that generated T?
How Effectively Can Machines Defend Against Machine-Generated Fake News? An Empirical Study. [paper]
Meghana Moorthy Bhat and Srinivasan Parthasarathy.
Measure the performance of models by looking at accuracy with respect to perturbations introduced in this work. Find that success of style-based classifiers are limited when real articles are perturbed even under extreme modifications.
Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover. [paper]
Leon Fröhling and Arkaitz Zubiaga.
A feature-based detection approach relies on features that discriminate between human and machine text by modelling properties and dimensions in which both types of text differ.
Detecting Bot-Generated Text by Characterizing Linguistic Accommodation in Human-Bot Interactions. [paper]
Paras Bhatt and Anthony Rios.
Show that bot-generated text detection methods are more robust across datasets and models if we use information about how people respond to it rather than using the bot’s text directly.
Artificial Text Detection via Examining the Topology of Attention Maps. [paper]
Laida Kushnareva, Daniil Cherniavskii, Vladislav Mikhailov, Ekaterina Artemova, Serguei Barannikov, Alexander Bernstein, Irina Piontkovskaya, Dmitri Piontkovski and Evgeny Burnaev.
Propose three novel types of interpretable topological features based on Topological Data Analysis (TDA). The features derived from the BERT model outperform count- and neural-based baselines up to 10% on three common datasets, and tend to be the most robust towards unseen GPT-style generation models as opposed to existing methods.
Unsupervised and Distributional Detection of Machine-Generated Text. [paper]
Matthias Gallé, Jos Rozen, Germán Kruszewski and Hady Elsahar.
Propose a method to detect those machine-generated documents leveraging repeated higher-order n-grams, which we show over-appear in machine-generated text as compared to human ones.
Detecting computer-generated disinformation. [paper]
Harald Stiff and Fredrik Johansson.
Evaluate promising Transformer-based detection algorithms in a large variety of experiments involving both in-distribution and out-of-distribution test data, as well as evaluation on more realistic in-the-wild data.
On pushing DeepFake Tweet Detection capabilities to the limits. [paper]
Margherita Gambini, Tiziano Fagni, Fabrizio Falchi and Maurizio Tesconi.
Study and improve the performance of the stateof-the-art deepfake tweet detection methods over GPT-2 tweets and the detectors’ capabilities to generalize over tweets generated by GPT-3.
Cross-Domain Detection of GPT-2-Generated Technical Text. [paper]
Juan Rodriguez, Todd Hay, David Gros, Zain Shamsi and Ravi Srinivasan.
Find that RoBERTa-based detectors can be successfully adapted from one scientific discipline (physics) to another (biomedicine), requiring relatively small amounts of in-domain labeled data.
Automatic Detection of Machine Generated Texts: Need More Tokens. [paper]
Paras Bhatt and Anthony Rios.
Present a dataset for Russian language and conduct a set of learning experiments to build accurate machine-generated text detectors for both English and Russian languages.
Threat Scenarios and Best Practices to Detect Neural Fake News. [paper]
Artidoro Pagnoni, Martin Graciarena and Yulia Tsvetkov.
Provide an assessment of the current landscape of generated text detection and identify three primary threat scenarios. Establish the minimax strategies that minimize the worst case scenario for the detector.
Demystifying Neural Fake News via Linguistic Feature-Based Interpretation. [paper]
Ankit Aich, Souvik Bhattacharya and Natalie Parde.
Conduct a linguistically interpretative examination of the feature vulnerabilities exploited by neural fake news generators. Establish 21 stylistic, complexity, and psychological features for detection.
CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Data Limitation With Contrastive Learning. [paper]
Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Yu Lan and Chao Shen.
Present a coherence-based contrastive learning model named CoCo to detect the possible machine-generated text under low-resource scenario.
How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection. [paper]
Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue and Yupeng Wu.
Collect dataset the Human ChatGPT Comparison Corpus (HC3) and build three different detection systems to detect whether a certain text is generated by ChatGPT or humans.
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. [paper]
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning and Chelsea Finn.
Propose DetectGPT, a zero-shot method for automated machine-generated text detection. To test if a passage came from a source model, DetectGPT compares the log probability of a candidate passage with the average log probability of several perturbations of the passage.
ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text. [paper]
Sandra Mitrović, Davide Andreoletti and Omran Ayoub.
Build Transformer-based model to discriminate between human-written and seemingly human text, focusing on short texts. Give some insights about ChatGPT writing style by SHAP explanations of the predictions.
AI vs. Human -- Differentiation Analysis of Scientific Content Generation. [paper]
Yongqiang Ma, Jiawei Liu, Fan Yi, Qikai Cheng, Yong Huang, Wei Lu and Xiaozhong Liu.
Investigate the detection methods from three perspectives: (1) feature-based detection method; (2) fine-tuned pre-trained detection model; (3) explainability of detection model. Further investigate the gap between AI-generated text and human-written text.
ChatGPT Generated Text Detection. [paper]
Rexhep Shijaku and Ercan Canhasi.
Present a classification model based on XGBoost for automatically detecting essays generated by ChatGPT.
Linguistic Markers of AI-Generated Text Versus Human-Generated Text: Evidence from Hotel Reviews and News Headlines. [paper]
David M. Markowitz, Jeffrey Hancock and Jeremy Bailenson.
Use ChatGPT to compare how it wrote hotel reviews and news headlines to human-generated counterparts across content, style, and structural features.
Mutation-Based Adversarial Attacks on Neural Text Detectors. [paper]
Gongbo Liang, Jesus Guerrero and Izzat Alsmadi.
Propose character- and word-based mutation operators for generating adversarial samples to attack state-of-the-art natural text detectors.
GAN-Based Unsupervised Learning Approach to Generate and Detect Fake News. [paper]
Pranjal Bhardwaj, Krishna Yadav, Hind Alsharif and Rania Anwar Aboalela.
Develop an unsupervised-based approach to generate fake news using autoencoder and GAN. Further describe the use of discriminators in detecting machine-generated fake news.
Combat AI With AI: Counteract Machine-Generated Fake Restaurant Reviews on Social Media. [paper]
Alessandro Gambetti and Qiwei Han.
Propose to leverage the high-quality elite restaurant reviews verified by Yelp to generate fake reviews from the OpenAI GPT review creator and ultimately fine-tune a GPT output detector to predict fake reviews that significantly outperform existing solutions.
Accurate Generated Text Detection Based on Deep Layer-wise Relevance Propagation. [paper]
Mengjie Guo, Limin Liu, Meicheng Guo, Siyuan Liu and Zhiwei Xu.
Propose a deep interpretable model to achieve accurate detection of the generated texts. In detail, a feature extraction method based on Layer-wise Relevance Propagation is proposed to improve the feature mining performance for detection of generated texts.
Stylometric Detection of AI-Generated Text in Twitter Timelines. [paper]
Tharindu Kumarage, Joshua Garland, Amrita Bhattacharjee, Kirill Trapeznikov, Scott Ruston and Huan Liu.
Propose two simple architectures to utilize three categories of stylometric features towards 1) discriminating between human-written and AI-generated tweets and 2) detecting if and when an AI starts to generate tweets in a given Twitter timeline.
ChatGPT or academic scientist? Distinguishing authorship with over 99% accuracy using off-the-shelf machine learning tools. [paper]
Heather Desaire, Aleesa E. Chua, Madeline Isom, Romana Jarosova and David Hua.
With a set of 20 features, build a model that assigned the author, as human or AI, at well over 99% accuracy, resulting in 20 times fewer misclassified documents compared to the field-leading approach.
Detection of AI-generated Essays in Writing Assessments. [paper]
Duanli Yan, Michael Fauss, Jiangang Hao, Wenju Cui.
Show how AI-generated essays are similar or different from human-written essays based on a set of typical prompts for a sample from a large-scale assessment. Introduce two classifiers that can detect AI-generated essays with a high accuracy of over 95%.
Comparing Abstractive Summaries Generated by ChatGPT to Real Summaries Through Blinded Reviewers and Text Classification Algorithms. [paper]
Mayank Soni, Vincent Wade.
Evaluate the performance of ChatGPT on Abstractive Summarization by the means of automated metrics and blinded human reviewers. Also build automatic text classifiers to detect ChatGPT generated summaries. Find that while text classification algorithms can distinguish between real and generated summaries, humans are unable to distinguish between real summaries and those produced by ChatGPT.
Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis. [paper]
Wataru Zaitsu and Mingzhe Jin.
Perform multi-dimensional scaling (MDS) to confirm the distributions of 216 texts of three classes (human, GPT-3.5, GPT-4) focusing on the following stylometric features: (1) bigrams of parts-of-speech, (2) bigram of postpositional particle words, (3) positioning of commas, and (4) rate of function words.
Evaluating AIGC Detectors on Code Content. [paper]
Jian Wang, Shangqing Liu, Xiaofei Xie and Yi Li.
Present the first empirical study on evaluating existing AIGC detectors in the software domain. Create a comprehensive dataset including 492.5K samples comprising code-related content produced by ChatGPT and evaluate six AIGC detectors on this dataset.
Detection of Fake Generated Scientific Abstracts. [paper]
Panagiotis C. Theocharopoulos, Panagiotis Anagnostou, Anastasia Tsoukala, Spiros V. Georgakopoulos, Sotiris K. Tasoulis and Vassilis P. Plagianakos.
Utilize the GPT-3 model to generate scientific paper abstracts through Artificial Intelligence and explore various text representation methods when combined with Machine Learning models with the aim of identifying machine-written text.
ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models. [paper]
Yikang Liu, Ziyin Zhang, Wanyang Zhang, Shisen Yue, Xiaojing Zhao, Xinyuan Cheng, Yiwen Zhang and Hai Hu.
Present ArguGPT, a balanced corpus of 4,038 argumentative essays generated by 7 GPT models in response to essay prompts from three sources: (1) in-class or homework exercises, (2) TOEFL and (3) GRE writing tasks. A RoBERTa fine-tuned with the training set of ArguGPT achieves above 90% accuracy in both essay- and sentence-level classification.
Improving Detection of ChatGPT-Generated Fake Science Using Real Publication Text: Introducing xFakeBibs a Supervised Learning Network Algorithm. [paper]
Ahmed Abdeen Hamed.
By means of a new algorithm, called xFakeBibs, show the significant difference between ChatGPT-generated fake publications and real publications. The algorithm predicted 98 of the 100 publications as fake, while 2 articles failed the test and were classified as real publications.
Differentiate ChatGPT-generated and Human-written Medical Texts. [paper]
Wenxiong Liao, Zhengliang Liu, Haixing Dai, Shaochen Xu, Zihao Wu, Yiyang Zhang, Xiaoke Huang, Dajiang Zhu, Hongmin Cai, Tianming Liu and Xiang Li.
Construct a suite of datasets containing medical texts written by human experts and generated by ChatGPT. Analyze the linguistic features of these two types of content and uncover differences in vocabulary, part-of-speech, dependency, sentiment, perplexity, etc. Finally, design and implement machine learning methods to detect medical text generated by ChatGPT.
CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts. [paper]
Peipeng Yu, Jiahan Chen, Xuan Feng and Zhihua Xia.
Present a large-scale CHatGPT-writtEn AbsTract dataset (CHEAT) contains 35,304 synthetic abstracts, with Generation, Polish, and Mix as prominent representatives. Show that ChatGPT-written abstracts are detectable, while the detection difficulty increases with human involvement.
Origin Tracing and Detecting of LLMs. [paper]
Linyang Li, Pengyu Wang, Ke Ren, Tianxiang Sun and Xipeng Qiu.
First raise the concern of the origin tracing of LLMs and propose an effective method to trace and detect AI-generated contexts. Introduce a novel algorithm that leverages the contrastive features between LLMs and extracts model-wise features to trace the text origins.
ChatLog: Recording and Analyzing ChatGPT Across Time. [paper]
Shangqing Tu, Chunyang Li, Jifan Yu, Xiaozhi Wang, Lei Hou and Juanzi Li.
Collect a coarse-to-fine temporal dataset called ChatLog, consisting of two parts that update monthly and daily. In section 5, explore the application of ChatGPT unchanged features on ChatLog-Daily dataset. Show a preliminary use case: improving the ChatGPT detection model’s robustness.
Perception, performance, and detectability of conversational artificial intelligence across 32 university courses. [paper]
Compare the performance of ChatGPT against students on 32 university-level courses. Current AI-text classifiers cannot reliably detect ChatGPT’s use in school work, due to their propensity to classify human-written answers as AI-generated, as well as the ease with which AI-generated text can be edited to evade detection.
GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content. [paper]
Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh and Bhiksha Raj.
Collecte and release a pre-processed dataset named OpenGPTText, which consists of rephrased content generated using ChatGPT. Design, implement, and train two different models for text classification, using RoBERTa and T5, respectively. These models achieve remarkable results, with an accuracy of over 97% on the test dataset, as evaluated through various metrics.
Machine-Made Media: Monitoring the Mobilization of Machine-Generated Articles on Misinformation and Mainstream News Websites. [paper]
Hans W. A. Hanley and Zakir Durumeric.
Train a DeBERTa-based synthetic news detector and classify over 12.91 million articles from 3,074 misinformation and mainstream news websites. Find that between January 1, 2022 and April 1, 2023, the relative number of synthetic news articles increased by 79.4% on mainstream websites while increasing by 342% on misinformation sites.
Smaller Language Models are Better Black-box Machine-Generated Text Detectors. [paper]
Fatemehsadat Mireshghallah, Justus Mattern, Sicun Gao, Reza Shokri and Taylor Berg-Kirkpatrick.
Explore whether models other than the generator can be used to differentiate between machinegenerated and human-written text. Find that overall, smaller and partially-trained models are better universal text detectors. Interestingly, we find that whether the detector and generator were trained on the same data is not critically important to the detection success.
DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection. [paper]
Xiao Yu, Yuang Qi, Kejiang Chen, Guoqiang Chen, Xi Yang, Pengyuan Zhu, Xiuwei Shang, Weiming Zhang and Nenghai Yu.
Propose to decouple prompt and intrinsic characteristics (DPIC) for LLM-generated text detection method. Specifically, given a candidate text, DPIC employs an auxiliary LLM to reconstruct the prompt corresponding to the candidate text, then uses the prompt to regenerate text by the auxiliary LLM. Then, the similarity between the candidate text and the regenerated text is used as a detection feature.
G3Detector: General GPT-Generated Text Detector. [paper]
Haolan Zhan, Xuanli He, Qiongkai Xu, Yuxiang Wu and Pontus Stenetorp.
Introduce an unpretentious yet potent detection approach proficient in identifying synthetic text across a wide array of fields. The detector demonstrates outstanding performance uniformly across various model architectures and decoding strategies. It also possesses the capability to identify text generated utilizing a potent detectionevasion technique.
MAGE: Machine-generated Text Detection in the Wild. [paper]
Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Longyue Wang, Linyi Yang, Shuming Shi and Yue Zhang.
Construct a wild testbed for deepfake text detection, by gathering texts from various writing tasks and deepfake texts generated by different LLMs. Find that supervised PLM-based methods were the best-performing detection methods across all testbeds. Out-of-distribution posed an even greater challenge for a detector to be employed in realistic application scenarios.
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text. [paper]
Jinyan Su, Terry Yue Zhuo, Di Wang and Preslav Nakov.
Introduce two novel zero-shot methods for detecting machinegenerated text by leveraging the log rank information. One is called DetectLLM-LRR, which is fast and efficient, and the other is called DetectLLM-NPR, which is more accurate, but slower due to the need for perturbations.
LLMDet: A Third Party Large Language Models Generated Text Detection Tool. [paper]
Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng and Tat-Seng Chua.
Propose an efficient, secure, and scalable detection tool called LLMDet, which calculates the proxy perplexity of text by utilizing the prior information of the model’s next-token probabilities, obtained through pre-training. Subsequently, use the self-watermarking information of the model, as measured by proxy perplexity, to detect the source of the text.
Ghostbuster: Detecting Text Ghostwritten by Large Language Models. [paper]
Vivek Verma, Eve Fleisig, Nicholas Tomlin and Dan Klein.
Introduce Ghostbuster, works by passing documents through a series of weaker language models and running a structured search over possible combinations of their features, then training a classifier on the selected features to determine if the target document was AI-generated. Release three new datasets of human and AI-generated text as detection benchmarks that cover multiple domains and task setups.
HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis. [paper]
Christoforos Vasilatos, Manaar Alam, Talal Rahwan, Yasir Zaki and Michail Maniatakos.
Propose a novel multi-level approach to detect AIgenerated text focusing on university student homework. Utilize metadata categorization from an academic dataset to enhance the perplexity metric used to detect whether a given assignment has been studentauthored or AI-generated.
DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text. [paper]
Xianjun Yang, Wei Cheng, Linda Petzold, William Yang Wang and Haifeng Chen.
Given a text, we first truncate it in the middle and then use only the preceding portion as input to the LLMs to regenerate the new remaining parts. By analyzing the differences between the original and new remaining parts through N-gram analysis in black-box or probability divergence in white-box, we can clearly illustrate significant discrepancies between machine-generated and human-written text.
Efficient Detection of LLM-generated Texts with a Bayesian Surrogate Model. [paper]
Zhijie Deng, Hongcheng Gao, Yibo Miao and Hao Zhang.
DetectGPT suffers from significant inefficiency issues, as detecting a single candidate requires scoring hundreds of its perturbations with the source LLM. This paper aims to bridge this gap. Technically, propose to incorporate a Bayesian surrogate model, which allows us to select typical samples based on Bayesian uncertainty and interpolate scores from typical samples to other ones, to improve query efficiency.
Distinguishing Human Generated Text From ChatGPT Generated Text Using Machine Learning. [paper]
Niful Islam, Debopom Sutradhar, Humaira Noor, Jarin Tasnim Raya, Monowara Tabassum Maisha and Dewan Md Farid.
Present a machine learning-based solution that can identify the ChatGPT delivered text from the human written text along with the comparative analysis of a total of 11 machine learning and deep learning algorithms in the classification process.
Multiscale Positive-Unlabeled Detection of AI-Generated Texts. [paper]
Yuchuan Tian, Hanting Chen, Xutao Wang, Zheyuan Bai, Qinghua Zhang, Ruifeng Li, Chao Xu and Yunhe Wang.
Mainstream detectors are formulated without considering the factor of corpus length: shorter corpuses are harder to detect compared with longer ones for shortage of informative features. In this paper, a Multiscale Positive-Unlabeled (MPU) training framework is proposed to address the challenge of multiscale text detection.
Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts. [paper]
Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, Serguei Barannikov, Irina Piontkovskaya, Sergey Nikolenko and Evgeny Burnaev.
Propose an invariant of human texts, namely the intrinsic dimensionality of the manifold underlying the set of embeddings of a given text sample. We show a clear statistical separation between human-generated and AI-generated distributions. This property allows us to build a score-based artificial text detector, which outperforming SOTA detectors in model-agnostic and cross-domain scenarios by a significant margin.
Distinguishing academic science writing from humans or ChatGPT with over 99% accuracy using off-the-shelf machine learning tools. [paper]
Heather Desaire, Aleesa E. Chua, Madeline Isom, Romana Jarosova and David Hua.
Report a method for discriminating text generated by ChatGPT from (human) academic scientists, relying on prevalent and accessible supervised classification methods. The approach uses new features for discriminating (these) humans from AI.
Long-form analogies generated by chatGPT lack human-like psycholinguistic properties. [paper]
S. M. Seals and Valerie L. Shalin.
Apply psycholinguistic methods to evaluate individual sentences from long-form analogies about biochemical concepts. Perform a supervised classification analysis using 78 features extracted from Coh-metrix that analyze text cohesion, language, and readability. Results illustrate high performance for classifying student-generated and chatGPT-generated analogies.
Check Me If You Can: Detecting ChatGPT-Generated Academic Writing using CheckGPT. [paper]
Zeyan Liu, Zijun Yao, Fengjun Li and Bo Luo.
Present GPABenchmark, a benchmarking dataset of 600,000 samples of human-written, GPT-written, GPT-completed, and GPT-polished abstracts of research papers in CS, physics, and humanities and social sciences (HSS). Present CheckGPT, a novel LLM-content detector consisting of a general representation module and an attentive-BiLSTM classification module, which is accurate, transferable, and interpretable.
Stanford CS224N Custom Projects
Check Me If You Can: Detecting ChatGPT-Generated Academic Writing using CheckGPT. [paper]
Jeffrey Heo and Simon Kim.
Looking Under the Hood of DetectGPT. [paper]
Maximilian Du, Ryan Lian and Kaien Yang.
GPTNo: A Deep Learning LLM to Beat the Turing Test. [paper]
Sri Jaladi, William Li and Abhinav Sinha.
Human Writing is as Uniform as Machine Writing. [paper]
Ryan Tan, Raghav Garg and Jacob Stavrianos.
DetectChatGPT: Black-Box Zero-Shot Detection of LLM-Generated Text. [paper]
Julia Park and Armaan Rashid.
La détection de textes générés par des modèles de langue: une tâche complexe? Une étude sur des textes académiques. [paper]
Vijini Liyanage and Davide Buscaldi.
Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect? [paper]
Wissam Antoun, Virginie Mouilleron, Benoît Sagot and Djamé Seddah.
Proposes a methodology for developing and evaluating ChatGPT detectors for French text, with a focus on investigating their robustness on outof-domain data and against common attack schemes. The proposed method involves translating an English dataset into French and training a classifier on the translated data.
Implementing BERT and fine-tuned RobertA to detect AI generated news by ChatGPT. [paper]
Zecong Wang, Jiaxi Cheng, Chen Cui and Chenhao Yu.
Demonstrate that the BERT and RobertA models with fine-tuning had the best success in detecting AI generated news. With a score of 98%, tweaked RobertA in particular showed excellent precision.
Beyond Black Box AI-Generated Plagiarism Detection: From Sentence to Document Level. [paper]
Mujahid Ali Quidwai, Chunhui Li and Parijat Dube.
Propose a novel approach offering quantifiable metrics at both sentence. Employs a multi-faceted approach, generating multiple paraphrased versions of a given question and inputting them into the LLM to generate answers. By using a contrastive loss function based on cosine similarity, match generated sentences with those from the student’s response.
Matching Pairs: Attributing Fine-Tuned Models to their Pre-Trained Large Language Models. [paper]
Myles Foley, Ambrish Rawat, Taesung Lee, Yufang Hou, Gabriele Picco and Giulio Zizzo.
We need a method to investigate how a model was trained or a piece of text was generated and what their pre-trained base model was. Address this open problem by tracing back the origin of a given fine-tuned LLM to its corresponding pre-trained base model. Consider different knowledge levels and attribution strategies, and find that we can correctly trace back 8 out of the 10 fine tuned models with our best method.
Detecting Artificially Generated Academic Text: The Importance of Mimicking Human Utilization of Large Language Models. [paper]
Vijini Liyanage and Davide Buscaldi.
Examine the detectability of the generated text using DetectGPT and GLTR, and utilize state-of-the-art classification models like SciBERT, RoBERTa, DEBERTa, XLNet, and ELECTRA. Experiments show that the generated text is difficult to detect using existing models when created using a LLM fine-tuned on the remainder of a paper.
Testing of Detection Tools for AI-Generated Text. [paper]
Debora Weber-Wulff, Alla Anohina-Naumeca, Sonja Bjelobaba, Tomáš Foltýnek, Jean Guerrero-Dib, Olumide Popoola, Petr Šigut and Lorna Waddington.
The study seeks to answer research questions about whether existing detection tools can reliably differentiate between human-written text and ChatGPT-generated text, and whether machine translation and content obfuscation techniques affect the detection of AIgenerated text. The research covers 12 publicly available tools and two commercial systems (Turnitin and PlagiarismCheck) that are widely used in the academic setting.
Discriminating Human-authored from ChatGPT-Generated Code Via Discernable Feature Analysis. [paper]
Li Ke, Hong Sheng, Fu Cai, Zhang Yunhe and Liu Ming.
Propose a discriminative feature set yielding high accuracy in differentiating ChatGPT-generated code from human-authored code in binary classification tasks; Devise methods for generating extensive ChatGPT-generated codes; Introduce a dataset cleansing strategy.
RADAR: Robust AI-Text Detection via Adversarial Learning. [paper]
Xiaomeng Hu, Pin-Yu Chen and Tsung-Yi Ho.
Existing works show that current AI-text detectors are not robust to LLM-based paraphrasing, this paper aims to bridge this gap by proposing a new framework called RADAR. RADAR is based on adversarial training of a paraphraser and a detector. The paraphraser’s goal is to generate realistic contents to evade AI-text detection. RADAR uses the feedback from the detector to update the paraphraser, and vice versa.
Detecting LLM-Generated Text in Computing Education: A Comparative Study for ChatGPT Cases. [paper]
Michael Sheinman Orenstrakh, Oscar Karnalim, Carlos Anibal Suarez and Michael Liut.
Collect 124 submissions from computer science students before the creation of ChatGPT, then generate 40 ChatGPT submissions. Use this data to evaluate eight publicly-available LLM-generated text detectors through the measures of accuracy, false positives, and resilience.
Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text. [paper]
Lingyi Yang, Feng Jiang and Haizhou Li.
Introduce a novel dataset termed HPPT (ChatGPT-polished academic abstracts), facilitating the construction of more robust detectors. Propose the "Polish Ratio" method, an innovative measure of ChatGPT’s involvement in text generation based on editing distance. It provides a mechanism to measure the degree of human originality in the resulting text.
OUTFOX: LLM-generated Essay Detection through In-context Learning with Adversarially Generated Examples. [paper]
Ryuto Koike, Masahiro Kaneko and Naoaki Okazaki.
Propose OUTFOX, a novel framework that improves the robustness of LLM-generated-text detectors by allowing both the detector and the attacker to consider each other’s output and apply this to the domain of student essays.
The Looming Threat of Fake and LLM-generated LinkedIn Profiles: Challenges and Opportunities for Detection and Prevention. [paper]
Navid Ayoobi, Sadat Shahriar and Arjun Mukherjee.
Present a novel method for detecting fake and Large Language Model (LLM)-generated profiles in the LinkedIn Online Social Network immediately upon registration and before establishing connections.
The Imitation Game: Detecting Human and AI-Generated Texts in the Era of Large Language Models. [paper]
Kadhim Hayawi, Sakib Shahriar and Sujith Samuel Mathew.
Introduce a novel dataset of human-written and LLM-generated texts in different genres: essays, stories, poetry, and Python code. Employ several machine learning models to classify the texts.
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education. [paper]
Zijie Zeng, Lele Sha, Yuheng Li, Kaixun Yang, Dragan Gašević and Guanliang Chen.
Construct a hybrid essay dataset. Propose a two-step detection approach where we (1) separate AI-generated content from human-written content during the encoder training process; and (2) calculate the distances between every two adjacent prototypes and assume that the boundaries exist between the two adjacent prototypes that have the furthest distance from each other.
Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text? [paper]
Amrita Bhattacharjee and Huan Liu.
Empirically investigate if ChatGPT is symmetrically effective in detecting AI-generated or human-written text. Provide insight on how ChatGPT and similar LLMs may be leveraged in automated detection pipelines by simply focusing on solving a specific aspect of the problem and deriving the rest from that solution.
A Deep Fusion Model for Human vs. Machine-Generated Essay Classification. [paper]
Roberto Corizzo and Sebastian Leal-Arenas.
Propose a deep neural network-based model that combines text modeling and linguistic features via data fusion, with the aim to provide robust classification capabilities. Experiments show that it is possible to accurately perform essay classification, and that our model is competitive, even when compared to popular feature-based and neural network-based approaches.
MFD: Multi-Feature Detection of LLM-Generated Text. [paper]
Zhendong Wu and Hui Xiang.
Introduce Multi-Feature Detection (MFD), a new zero-shot method. MFD comprehensively considers log-likelihood, log-rank, entropy, and LLM-Deviation. LLM-Deviation is a new statistical feature proposed in this paper and has a clear distribution difference between texts generated by LLMs and those written by humans. Experiments show MFD is more effective than the existing zero-shot method.
Classification of Human- and AI-Generated Texts: Investigating Features for ChatGPT. [paper]
Lorenz Mindner, Tim Schlippe and Kristina Schaaff.
Explore traditional and new features to (1) detect text generated by AI from scratch and (2) text rephrased by AI. Results show that the new features substantially help to improve the performance of many classifiers.
Separating the Human Touch from AI-Generated Text using Higher Criticism: An Information-Theoretic Approach. [paper]
Alon Kipnis.
Propose a method to determine whether a given article was entirely written by a generative language model. The process involves many perplexity tests for the origin of individual sentences or other text atoms, combining these multiple tests using Higher Criticism (HC).
人工智能生成语言与人类语言对比研究——以ChatGPT为例. [paper]
朱君辉, 王梦焰, 杨尔弘, 聂锦燃, 王誉杰, 岳岩 and 杨麟儿.
Explore the differences between the language used in human-generated responses and responses generated by ChatGPT. Extract and compute the distribution of 159 language features in real human text and ChatGPT-generated text. Employ three machine learning algo-rithms: Random Forest, Logistic Regression, and SVM. The result reveals that the two texts differ significantly in three dimensions: descriptive features, word commonness, and word diversity.
HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus. [paper]
Zhenpeng Su, Xing Wu, Wei Zhou, Guangyuan Ma and Songlin Hu.
Introduce a more extensive and comprehensive dataset that considers more types of tasks than previous work, including semantic-invariant tasks. Further instruct fine-tuning Tk-instruct and build a more powerful detection system. Experimental results show that the proposed detector outperforms the previous state-of-the-art RoBERTa-based detector.
J-Guard: Journalism Guided Adversarially Robust Detection of AI-generated News. [paper]
Tharindu Kumarage, Amrita Bhattacharjee, Djordje Padejski, Kristy Roschke, Dan Gillmor, Scott Ruston, Huan Liu and Joshua Garland.
Develop a framework, J-Guard, capable of steering existing supervised AI text detectors for detecting AI-generated news while boosting adversarial robustness. By incorporating stylistic cues inspired by the unique journalistic attributes, J-Guard effectively distinguishes between real-world journalism and AIgenerated news articles.
Overview of AuTexTification at IberLEF 2023: Detection and Attribution of Machine-Generated Text in Multiple Domains. [paper]
Areg Mikael Sarvazyan, José Ángel González, Marc Franco-Salvador, Francisco Rangel, Berta Chulvi and Paolo Rosso.
This paper presents the overview of the AuTexTification shared task as part of the IberLEF 2023 Workshop. AuTexTification consists of two subtasks: for Subtask 1, participants had to determine whether a text is human-authored or has been generated by a large language model. For Subtask 2, participants had to attribute a machine-generated text to one of six different text generation models. In this overview, present the AuTexTification dataset and task, the submitted participating systems, and the results.
ConDA: Contrastive Domain Adaptation for AI-generated Text Detection. [paper]
Amrita Bhattacharjee, Tharindu Kumarage, Raha Moraffah and Huan Liu.
Address the problem of AIgenerated text detection in the absence of labeled target data. Propose a contrastive domain adaptation framework that leverages the power of both unsupervised domain adaptation and selfsupervised representation learning, in order to tackle the task of AI-generated text detection.
Supervised Machine-Generated Text Detectors: Family and Scale Matters. [paper]
Areg Mikael Sarvazyan, José Ángel González, Paolo Rosso and Marc Franco-Salvador.
This work studies the generalization capabilities of supervised Machine-Generated Text (MGT) detectors across model families and parameter scales of text generation models. In addition, explore the feasibility of identifying the family and scale of the generator behind an MGT, instead of attributing the text to a particular language model.
Use prompt to differentiate text generated by ChatGPT and humans. [paper]
Ruopeng An, Yuyi Yang, Fan Yang and Shanshan Wang.
Introduce a method that uses both prompts and essays to differentiate between machine-generated and human-written text. Validate its effectiveness in distinguishing between machine-generated and human-written essays and shows that it outperforms existing approaches based solely on text passages.
A Statistical Turing Test for Generative Models. [paper]
Hayden Helm, Carey E. Priebe and Weiwei Yang.
Provide a framework in the language of statistical pattern recognition that quantifies the difference between the distributions of human and machinegenerated content conditioned on an evaluation context. Describe current methods in the context of the framework and demonstrate how to use the framework to evaluate the progression of generative models towards human-like capabilities, among many axes of analysis.
TopRoBERTa: Topology-Aware Authorship Attribution of Deepfake Texts. [paper]
Adaku Uchendu, Thai Le and Dongwon Lee.
Propose TopRoBERTa to improve existing AA solutions by capturing more linguistic patterns in deepfake texts by including a Topological Data Analysis (TDA) layer in the RoBERTa model. Use RoBERTa to capture contextual representations (i.e., semantic and syntactic linguistic features), while using TDA to capture the shape and structure of data (i.e., linguistic structures). Finally, TopRoBERTa, outperforms the vanilla RoBERTa in 2/3 datasets, achieving up to 7% increase in Macro F1 score.
From Text to Source: Results in Detecting Large Language Model-Generated Content. [paper]
Wissam Antoun, Benoît Sagot and Djamé Seddah.
Investigate "Cross-Model Detection," evaluating whether a classifier trained to distinguish between source LLM-generated and humanwritten text can also detect text from a target LLM without further training. The study comprehensively explores various LLM sizes and families, and assesses the impact of conversational fine-tuning techniques on classifier generalization. The research also delves into Model Attribution, encompassing source model identification, model family classification, and model size classification.
Can LLM-Generated Misinformation Be Detected? [paper]
Canyu Chen and Kai Shu.
Build a taxonomy of LLM-generated misinformation. Categorize and validate the potential real-world methods for generating misinformation with LLMs. Discover that LLM-generated misinformation can be harder to detect for humans and detectors compared to human-written misinformation with the same semantics.
A Framework for Detecting AI-Generated Text in Research Publications. [paper]
Paria Sarzaeim, Arya Doshi and Qusay Mahmoud.
Present a framework for AI-generated text. The prototype implementation of the proposed approach is to train a model using predefined datasets and deploy this model on a cloud-based service to predict whether a text was created by a human or AI. This approach is specifically focused on assessing the accuracy of scientific writings and research papers rather than general text.
Supervised Machine Generated Text Detection Using LLM Encoders In Various Data Resource Scenarios. [paper]
Marc Capobianco, Matthew Reynolds, Charles Phelan, Krish Shah-Nathwani and Duong Luong.
Explore the effectiveness of BERT and RoBERTa-based machine generated text detection in a supervised setting. Created several models for both BERT and RoBERTa, trained with 5%, 10%, 15%, 20%, and 100% of the dataset. Conduct these experiments with frozen and unfrozen parameter variations.
Machine-Generated Text Detection and Attribution. [paper]
Sarvazyan and Areg Mikael.
(i) study the MGT detection and attribution models that achieve high performance under various settings, (ii) explore their generalization capabilities to different scenarios, analyzing their applicability to unseen types of data, and (iii) successfully organize an evaluation campaign for MGT detection and attribution.
Automated Text Identification Shared Task (AuTexTification)
Automated Text Identification Using CNN and Training Dynamics. [paper]
Claudiu Creanga and Liviu Petrisor Dinu.
Do Origin and Facts Identify Automatically Generated Text? [paper]
Judita Preiss and Monica Lestari Paramita.
Syntax-based Contrastive Learning for Automated Text Identification. [paper]
Hongyan Wu, Nankai Lin and Shengyi Jiang.
A Method for Identifying Bot Generated Text : Notebook for IberLEF 2023. [paper]
Maryam Elamine, Asma Mekki and Lamia Hadrich Belguith.
Artificial Intelligence-Based Text Classification: Separating Human Writing from Computer Generated Writing. [paper]
Luis Enrique Morales-Márquez, Erick Barrios-González and David Eduardo Pinto-Avendaño.
SINAI at AuTexTification in IberLEF 2023: Combining All Layer Embeddings for Automatically Generated Texts. [paper]
César Espin-Riofrio, Jenny Ortiz-Zambrano and Arturo Montejo-Ráez.
I’ve Seen Things You Machines Wouldn’t Believe: Measuring Content Predictability to Identify Automatically-Generated Text. [paper]
Piotr Przybyła, Nicolau Duran-Silva and Santiago Egea-Gómez.
Detecting Generated Text and Attributing Language Model Source with Fine-tuned Models and Semantic Understanding. [paper]
Margherita Gambini, Marco Avvenuti, Fabrizio Falchi, Maurizio Tesconi and Tiziano Fagni.
AI-Writing Detection Using an Ensemble of Transformers and Stylometric Features. [paper]
George Mikros, Athanasios Koursaris, Dimitrios Bilianos and George Markopoulos.
Exploring Text Representations for Detecting Automatically Generated Text. [paper]
Zayra Villegas-Trejo, Helena Gómez-Adorno and Sergio-Luis Ojeda-Trueba.
GPT-2 versus GPT-3 and Bloom: LLMs for LLMs Generative Text Detection. [paper]
Fernando Aguilar-Canto, Marco Cardoso-Moreno, Diana Jiménez and Hiram Calvo.
Taming the Turing Test: Exploring Machine Learning Approaches to Discriminate Human vs. AI-Generated Texts. [paper]
Alberto Fernández-Hernández, Juan Luis Arboledas-Márquez, Julián Ariza-Merino and Salud María Jiménez-Zafra.
Team GPLSI at AuTexTification Shared Task: Determining the Authorship of a Text. [paper]
Iván Martínez-Murillo, Robiert Sepúlveda-Torres, Estela Saquete, Elena LLoret and Manuel Palomar.
Generative AI Text Classification using Ensemble LLM Approaches. [paper]
Harika Abburi, Michael Suesserman, Nirmala Pudota, Balaji Veeramani, Edward Bowen and Sanmitra Bhattacharya.
Automated Text Identification: Multilingual Transformer-based Models Approach. [paper]
German Gritsay, Andrey Grabovoy, Aleksandr Kildyakov and Yury Chekhovich.
IberLEF 2023 AuTexTification: Automated Text Identification Shared Task – Team OD-21. [paper]
Rinaldo Gagiano, Haytham Fayek, Maria Myung-Hee Kim, Jennifer Biggs and Xiuzhen Zhang.
Using Linguistic Knowledge for Automated Text Identification. [paper]
Lara Alonso Simón, José Antonio Gonzalo Gimeno, Ana María Fernández-Pampillón Cesteros, Marianela Fernández Trinidad and María Victoria Escandell Vidal.
Univ. of Hildesheim at AuTexTification 2023: Detection of Automatically Generated Texts. [paper]
Tatjana Scheibe and Thomas Mandl.
UPB at IberLEF-2023 AuTexTification: Detection of Machine-Generated Text using Transformer Ensembles. [paper]
Andrei-Alexandru Preda, Dumitru-Clementin Cercel, Traian Rebedea and Costin-Gabriel Chiru.
Exploring the Distinction: Investigating the Recognition of Automatic Text Generation Systems and Differentiating Human Text from Language Models. [paper]
Manex Agirrezabal.
Looking for Traces of Textual Deepfakes in Bulgarian on Social Media. [paper]
Irina Temnikova, Iva Marinova, Silvia Gargova, Ruslana Margova and Ivan Koychev.
Create a new Bulgarian-language dataset with real social media messages and those generated by two language models. Conclude that combining LM text detection with fact-checking is the most appropriate method for this task, and that identifying Bulgarian textual deepfakes is indeed possible.
On the Generalization of Training-based ChatGPT Detection Methods. [paper]
Han Xu, Jie Ren, Pengfei He, Shenglai Zeng, Yingqian Cui, Amy Liu, Hui Liu and Jiliang Tang.
Aim to have a comprehensive investigation on existing methods’ generalization behaviors under distribution shift caused by a wide range of factors, including prompts, text lengths, topics, and language tasks. Collect a new dataset with human and ChatGPT texts, and then conduct extensive studies on the collected dataset.
Counter Turing Test CT^2: AI-Generated Text Detection is Not as Easy as You May Think -- Introducing AI Detectability Index. [paper]
Megha Chakraborty, S.M Towhidul Islam Tonmoy, S M Mehedi Zaman, Krish Sharma, Niyar R Barman, Chandan Gupta, Shreya Gautam, Tanay Kumar, Vinija Jain, Aman Chadha, Amit P. Sheth and Amitava Das.
Introduce the Counter Turing Test (CT2), a benchmark consisting of techniques aiming to offer a comprehensive evaluation of the robustness of existing AGTD techniques. Establish a quantifiable spectrum facilitating the evaluation and ranking of LLMs according to their detectability levels, we propose the AI Detectability Index (ADI).
Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature. [paper]
Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang and Yue Zhang.
Present Fast-DetectGPT, an optimized zero-shot detector, which substitutes DetectGPT’s perturbation step with a more efficient sampling step. The evaluations on various datasets, source models, and test conditions indicate that Fast-DetectGPT not only outperforms DetectGPT in both the whitebox and black-box settings but also accelerates the detection process by a factor of 340.
Zero-Shot Detection of Machine-Generated Codes. [paper]
Xianjun Yang, Kexun Zhang, Haifeng Chen, Linda Petzold, William Yang Wang and Wei Cheng.
Propose a training-free approach for the detection of LLMs-generated codes, mitigating the risks associated with their indiscriminate usage. Modify the previous zero-shot text detection method, DetectGPT by utilizing a surrogate white-box model to estimate the probability of the rightmost tokens, allowing us to identify code snippets generated by language models.
How Reliable Are AI-Generated-Text Detectors? An Assessment Framework Using Evasive Soft Prompts. [paper]
Tharindu Kumarage, Paras Sheth, Raha Moraffah, Joshua Garland and Huan Liu.
Suggest a universal evasive prompt, a novel type of soft prompt, which guides PLMs in producing "human-like" text that can mislead the detectors. The novel universal evasive prompt is achieved in two steps: First, create an evasive soft prompt tailored to a specific PLM through prompt tuning; and then, leverage the transferability of soft prompts to transfer the learned evasive soft prompt from one PLM to another.
On the Zero-Shot Generalization of Machine-Generated Text Detectors. [paper]
Xiao Pu, Jingyu Zhang, Xiaochuang Han, Yulia Tsvetkov and Tianxing He.
Collect generation data from a wide range of LLMs, and train neural detectors on data from each generator and test its performance on heldout generators. Demonstrate that robust detectors can be built on an ensemble of training data from medium-sized models.
GPT-who: An Information Density-based Machine-Generated Text Detector. [paper]
Saranya Venkatraman, Adaku Uchendu and Dongwon Lee.
Propose GPT-who, the first psycholinguistically-aware multi-class domainagnostic statistical-based detector. This detector employs UID-based features to model the unique statistical signature of each LLM and human author for accurate authorship attribution.
SeqXGPT: Sentence-Level AI-Generated Text Detection. [paper]
Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang and Xipeng Qiu.
Propose Sequence X (Check) GPT, a novel method that utilizes log probability lists from white-box LLMs as features for sentence-level AIGT detection. These features are composed like waves in speech processing and cannot be studied by LLMs. Build SeqXGPT based on convolution and self-attention networks.
Raidar: geneRative AI Detection viA Rewriting. [paper]
Chengzhi Mao, Carl Vondrick, Hao Wang and Junfeng Yang.
Find that large language models (LLMs) are more likely to modify human-written text than AI-generated text when tasked with rewriting. Introduce a method to detect AI-generated content by prompting LLMs to rewrite text and calculating the editing distance of the output.
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text. [paper]
Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping and Tom Goldstein.
Propose a novel LLM detector that only requires simple calculations using pre-trained LLMs. The method, called Binoculars, achieves state-of-the-art accuracy without any training data.
Human-in-the-loop Detection of AI-generated Text via Grammatical Patterns. [paper]
Johan Lokna, Mislav Balunovic and Martin Vechev.
Focus on the problem of detecting AI-generated text in a domain where a training dataset of human-written samples is readily available. Our key insight is to learn interpretable grammatical patterns that are highly indicative of human or AI written text. The most useful of these patterns can then be given to humans as part of a human-in-the-loop approach.
Few-Shot Detection of Machine-Generated Text using Style Representations. [paper]
Rafael Rivera Soto, Kailin Koch, Aleem Khan, Barry Chen, Marcus Bishop and Nicholas Andrews.
Pursue a approach not relying on samples from language models of concern at training time. Propose to leverage representations of writing style estimated from human-authored text. Find that features effective at distinguishing among human authors are also effective at distinguishing human from machine authors.
Metric Learning for Detection of Large Language Model Generated Texts. [paper]
Linh Le, Shashank Hebbar and My Nguyen.
Present a new paradigm of metric-based detection for LLM-generated texts that is able to balance among computational costs, accessibility, and performances. Specifically, the detection is performed through evaluating the similarity between a given text to an equivalent example generated by LLMs and through that determining the former's origination.
Detecting Machine-Generated Texts by Multi-Population Aware Optimization for Maximum Mean Discrepancy. [paper]
Shuhai Zhang, Feng Liu, Jiahao Yang, Yifan Yang, Changsheng Li, Bo Han and Mingkui Tan.
Propose a novel multi-population aware optimization method for MMD called MMD-MP, which can avoid variance increases and thus improve the stability to measure the distributional discrepancy. Relying on MMD-MP, develop two methods for paragraph-based and sentence-based detection, respectively.
Using AI-based detectors to control AI-assisted plagiarism in ESL writing: “The Terminator Versus the Machines”. [paper]
Karim Ibrahim.
To address the challenge of AI-assisted plagiarism in ESL contexts, the present cross-disciplinary descriptive study examined the potential of two RoBERTa-based classifiers to control AI-assisted plagiarism on a dataset of 240 human-written and ChatGPT-generated essays. Data analysis revealed that both platforms could identify AI-generated texts, but their detection accuracy was inconsistent across the dataset.
An Ensemble-Based Approach for Generative Language Model Attribution. [paper]
Harika Abburi, Michael Suesserman, Nirmala Pudota, Balaji Veeramani, Edward Bowen and Sanmitra Bhattacharya.
Propose an ensemble neural model that generates probabilities from multiple pre-trained LLMs, which are then used as features for a traditional machine learning classifier.
DetectGPT-SC: Improving Detection of Text Generated by Large Language Models through Self-Consistency with Masked Predictions. [paper]
Rongsheng Wang, Qi Li and Sihong Xie.
Find that large language models such as ChatGPT exhibit strong self-consistency in text generation and continuation. Propose DetectGPT-SC for AI-generated texts detection based on self-consistency with masked predictions to determine whether a text is generated by LLMs.
Do Stochastic Parrots have Feelings Too? Improving Neural Detection of Synthetic Text via Emotion Recognition. [paper]
Alan Cowap, Yvette Graham and Jennifer Foster.
Hypothesize that pretrained language models (PLMs) have an affective deficit because of lacking the kind of emotional coherence present in human-authored text. Develop an emotionally aware detector by fine-tuning a PLM on emotion.
How well can machine-generated texts be identified and can language models be trained to avoid identification? [paper]
Sinclair Schneider, Florian Steuber, Joao A. G. Schneider and Gabi Dreo Rodosek.
Highlight the significance of sampling techniques and the essential role of employing advanced models like transformers to detect generated texts accurately. Demonstrate how a malicious actor could adapt generative models to evade a detector if accessible.
An Ensemble Method Based on the Combination of Transformers with Convolutional Neural Networks to Detect Artificially Generated Text. [paper]
Vijini Liyanage and Davide Buscaldi.
Present some classification models constructed by ensembling transformer models such as SciBERT, DeBERTa and XLNet, with Convolutional Neural Networks (CNNs). The experiments demonstrate that the considered ensemble architectures surpass the performance of the individual transformer models for classification.
Detection of news written by the ChatGPT through authorship attribution performed by a Bidirectional LSTM model. [paper]
Amanda Ferrari Iaquinta and Gustavo Voltani von Atzingen.
A dataset containing equal amounts of human and ChatGPT written news was assembled and different natural processing language techniques were used to extract features from it that were used to train, validate and test three models built with different techniques. The best performance was produced by the Bidirectional Long Short Term Memory (LSTM) Neural Network model.
How to Detect AI-Generated Texts? [paper]
Trung Nguyen, Amartya Hatua and Andrew H. Sung.
Present a method to distinguish synthetically generated text (SGT) from human-written text (HWT). The method includes methods for dataset creation, feature engineering, dataset comparison, and result analysis.
A Simple yet Efficient Ensemble Approach for AI-generated Text Detection. [paper]
Harika Abburi, Kalyani Roy, Michael Suesserman, Nirmala Pudota, Balaji Veeramani, Edward Bowen and Sanmitra Bhattacharya.
Propose a simple yet efficient solution to this problem by ensembling predictions from multiple constituent LLMs. Study the influence that the training data from individual LLMs have on model performance. Find that substituting commercially-restrictive GPT data with data generated from other open language models is a feasible alternative when developing generative text detectors.
Detecting and Unmasking AI-Generated Texts through Explainable Artificial Intelligence using Stylistic Features. [paper]
Aditya Shah, Prateek Ranka, Urmi Dedhia, Shruti Prasad, Siddhi Muni and Kiran Bhowmick.
The research delves into parameters such as syllable count, word length, sentence structure, functional word usage, and punctuation ratios to detect AI-generated text. Furthermore, the research integrates Explainable AI (xAI) techniques—LIME and SHAP—to enhance the interpretability of machine learning model predictions.
DEMASQ: Unmasking the ChatGPT Wordsmith. [paper]
Kavita Kumari, Alessandro Pegoraro, Hossein Fereidooni and Ahmad-Reza Sadeghi.
Propose an effective ChatGPT detector named DEMASQ, which accurately identifies ChatGPT-generated content. The method addresses two critical factors: (i) the distinct biases in text composition observed in human- and machinegenerated content and (ii) the alterations made by humans to evade previous detection methods.
AuthentiGPT: Detecting Machine-Generated Text via Black-Box Language Models Denoising. [paper]
Zhen Guo and Shangdi Yu.
Present AuthentiGPT, an efficient classifier that distinguishes between machine-generated and human-written texts. Under the assumption that humanwritten text resides outside the distribution of machine-generated text, AuthentiGPT leverages a black-box LLM to denoise input text with artificially added noise, and then semantically compares the denoised text with the original to determine if the content is machine-generated.
How You Prompt Matters! Even Task-Oriented Constraints in Instructions Affect LLM-Generated Text Detection. [paper]
Ryuto Koike, Masahiro Kaneko and Naoaki Okazaki.
This paper found that current detectors are brittle to instruction changes in text generation and raised awareness of the need to ensure prompt diversity when creating a detection benchmark. Open-sourced the dataset consisting of LLM-generated essays via various instructions.
Word-Level Interpretation of Chatgpt Detector Based on Classification Contribution. [paper]
Dekun Chen.
Present a method to show the contribution of words to model predictions. Specifically, this study considers information from classification weight vectors, semantic vectors, and embedded input word vectors for a more complete interpretation of detector LLM.
A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts. [paper]
Nafis Irtiza Tripto, Saranya Venkatraman, Dominik Macko, Robert Moro, Ivan Srba, Adaku Uchendu, Thai Le and Dongwon Lee.
Explore the diverse notions of authorship regarding paraphrasing in author attribution and AI text detection, incorporating philosophical concepts like the Ship of Theseus paradox. Suggest that authorship should be task-dependent, and we substantiate our results with relevant theoretical and philosophical perspectives.
Spot the Bot: Distinguishing Human-Written and Bot-Generated Texts Using Clustering and Information Theory Techniques. [paper]
Vasilii Gromov and Quynh Nhu Dang.
Propose a bot identification algorithm that is based on unsupervised learning techniques and does not depend on a large amount of labelled data. By combining findings in semantic analysis by clustering (crisp and fuzzy) and information techniques, we construct a robust model that detects a generated text for different types of bot.
Beyond Turing: A Comparative Analysis of Approaches for Detecting Machine-Generated Text. [paper]
Muhammad Farid Adilazuarda, Nikolaos Nektarios Arkoulis and Oleksii Chumakov.
Present the IMGTB framework, which simplifies the benchmarking of machine-generated text detection methods by easy integration of custom (new) methods and evaluation datasets.
IMGTB: A Framework for Machine-Generated Text Detection Benchmarking. [paper]
Michal Spiegel and Dominik Macko.
Offer an in-depth evaluation of three distinct methods: traditional shallow learning, Language Model (LM) fine-tuning, and Multilingual Model fine-tuning. These approaches are rigorously tested on a wide range of machine-generated texts, providing a benchmark of their competence in distinguishing between human-authored and machine-authored linguistic constructs.
Machine-Generated Text Detection using Deep Learning. [paper]
Raghav Gaggar, Ashish Bhagchandani and Harsh Oza.
Create a wellconstructed labeled dataset encompassing texts originated by both humans and ChatGPT (GPT3.5 turbo) from five distinct sources. Additionally, design and train multiple classifiers to differentiate between texts generated by humans and ChatGPT.
Token Prediction as Implicit Classification to Identify LLM-Generated Text. [paper]
Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh and Bhiksha Raj.
Instead of adding an additional classification layer to a base LM, we reframe the classification task as a next-token prediction task and directly fine-tune the base LM to perform it. Utilize the Text-to-Text Transfer Transformer (T5) model as the backbone for the experiments.
The Effectiveness of Free Software for Detecting AI-Generated Writing. [paper]
Gregory Price and Marc Sakellarios.
Explore the effectiveness of freely available AI detection software in identifying AI-generated content and its implications for teacher evaluation of student work. Compare our manual analysis with results from five different free AI writing detectors.
Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT. [paper]
Biru Zhu, Lifan Yuan, Ganqu Cui, Yangyi Chen, Chong Fu, Bingxiang He, Yangdong Deng, Zhiyuan Liu, Maosong Sun and Ming Gu.
Design a zero-shot black-box method for detecting LLM-generated texts. The key idea is to revise the text to be detected using the ChatGPT model. Our method is based on the intuition that the ChatGPT model will make fewer revisions to LLM-generated texts than it does to human-written texts, because the texts generated by LLMs are more in accord with the generation logic and statistical patterns learned by LLMs like ChatGPT.
Evaluating the Efficacy of Hybrid Deep Learning Models in Distinguishing AI-Generated Text. [paper]
Finbarrs Oketunji.
Investigate the use of cutting-edge hybrid deep learning models to accurately differentiate between AI-generated text and human writing. Combining sophisticated neural networks, the custom model enabled it to detect nuanced differences between AI and human content.
STADEE: STAtistics-based DEEp Detection of Machine Generated Text. [paper]
Zheng Chen and Huming Liu.
Introduce STADEE, a STAtistics-based DEEp detection method that integrates essential statistical features of text with a sequence-based deep classifier. Utilize various statistical features, such as the probability, rank, cumulative probability of each token, as well as the information entropy of the distribution at each position.
Classification of Human- and AI-Generated Texts for English, French, German, and Spanish. [paper]
Kristina Schaaff, Tim Schlippe and Lorenz Mindner.
analyze features to classify human- and AI-generated text for English, French, German and Spanish and compare them across languages. Investigate two scenarios: (1) The detection of text generated by AI from scratch, and (2) the detection of text rephrased by AI.
ALTA Shared Task 2023
Overview of the 2023 ALTA Shared Task: Discriminate between Human-Written and Machine-Generated Text. [paper]
Diego Molla, Haolan Zhan, Xuanli He and Qiongkai Xu.
A Prompt in the Right Direction: Prompt Based Classification of Machine-Generated Text Detection. [paper]
Rinaldo Gagiano and Lin Tian.
Automatic Detection of Machine-Generated Text Using Pre-Trained Language Models. [paper]
Yunhao Fang.
An Ensemble Based Approach To Detecting LLM-Generated Texts. [paper]
Ahmed EI-Sayed and Omar Nasr.
Feature-Level Ensemble Learning for Robust Synthetic Text Detection with DeBERTaV3 and XLM-RoBERTa. [paper]
Saman Sarker Joy and Tanusree Das Aishi.
Stacking the Odds: Transformer-Based Ensemble for AI-Generated Text Detection. [paper]
Duke Nguyen, Khaing Myat Noe Naing and Aditya Joshi.
HuRef: HUman-REadable Fingerprint for Large Language Models. [paper]
Boyi Zeng, Chenghu Zhou, Xinbing Wang and Zhouhan Lin.
Introduce HuRef, a human-readable fingerprint for LLMs that uniquely identifies the base model without exposing model parameters or interfering with training. The method generates a dog image as an identity fingerprint for an LLM, where the dog’s appearance strongly indicates the LLM’s base model. Specifically, if the LLM is adapted from another base model, the generated dog highly resembles that model; otherwise if trained independently from scratch, it exhibits a unique dog image distinct from other models.
Deep dive into language traits of AI-generated Abstracts. [paper]
Vikas Kumar, Amisha Bharti, Devanshu Verma and Vasudha Bhatnagar.
Attempt to detect the Abstracts generated by ChatGPT, which are much shorter in length and bounded. Extract the text’s semantic and lexical properties and observe that traditional machine learning models can confidently detect these Abstracts.
Assaying on the Robustness of Zero-Shot Machine-Generated Text Detectors. [paper]
Yi-Fan Zhang, Zhang Zhang, Liang Wang, Tieniu Tan and Rong Jin.
Existing zero-shot detectors, typically designed for specific tasks or topics, often assume uniform testing scenarios, limiting their practicality. Uncover a significant correlation between topics and detection performance. Delve into the influence of topic shifts on zero-shot detectors.
Assessing AI Detectors in Identifying AI-Generated Code: Implications for Education. [paper]
Wei Hung Pan, Ming Jie Chok, Jonathan Leong Shan Wong, Yung Xin Shin, Yeong Shian Poon, Zhou Yang, Chun Yong Chong, David Lo and Mei Kuan Lim.
Collect a dataset comprising 5,069 samples, with each sample consisting of a textual description of a coding problem and its corresponding human-written Python solution codes. Results demonstrate that existing AIGC Detectors perform poorly in distinguishing between human-written code and AI-generated code.
Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers. [paper]
Yuling Shi, Hongyu Zhang, Chengcheng Wan and Xiaodong Gu.
Propose a novel machine-generated code detection method called DetectCodeGPT, which improves DetectGPT by capturing the distinct structural patterns of code. Diverging from conventional techniques that depend on external LLMs for perturbations, DetectCodeGPT perturbs the code corpus by strategically inserting spaces and newlines, ensuring both efficacy and efficiency.
Enhancing Robustness of LLM-Synthetic Text Detectors for Academic Writing: A Comprehensive Analysis. [paper]
Zhicheng Dou, Yuchen Guo, Ching-Chun Chang, Huy H. Nguyen and Isao Echizen.
Present a comprehensive analysis of the impact of prompts on the text generated by LLMs and highlight the potential lack of robustness in one of the current state-of-the-art GPT detectors. Propose a reference-based Siamese detector named Synthetic-Siamese which takes a pair of texts, one as the inquiry and the other as the reference.
Deciphering Textual Authenticity: A Generalized Strategy through the Lens of Large Language Semantics for Detecting Human vs. Machine-Generated Text. [paper]
Mazal Bethany, Brandon Wherry, Emet Bethany, Nishant Vishwamitra and Peyman Najafirad.
Undertake a systematic study on the detection of machine-generated text in real-world scenarios. Study the effectiveness of state-of-the-art approaches and find that they are severely limited against text produced by diverse generators and domains in the real world. Introduce a novel system, T5LLMCipher, for detecting machine-generated text using a pretrained T5 encoder combined with LLM embedding sub-clustering to address the text produced by diverse generators and domains in the real world.
ALMs: Authorial Language Models for Authorship Attribution. [paper]
Weihang Huang, Akira Murakami and Jack Grieve.
Introduce an authorship attribution method called Authorial Language Models (ALMs) that involves identifying the most likely author of a questioned document based on the perplexity of the questioned document calculated for a set of causal language models fine-tuned on the writings of a set of candidate author.
Fine-tuning Large Language Models for Multigenerator, Multidomain, and Multilingual Machine-Generated Text Detection. [paper]
Feng Xiong, Thanet Markchom, Ziwei Zheng, Subin Jung, Varun Ojha and Huizhi Liang.
Two methods: 1) using traditional machine learning (ML) with natural language preprocessing (NLP) for feature extraction, and 2) fine-tuning LLMs for text classification. The results show that transformer models, particularly LoRA-RoBERTa, exceed traditional ML methods in effectiveness, with majority voting being particularly effective in multilingual contexts for identifying machine-generated texts.
Beyond Lexical Boundaries: LLM-Generated Text Detection for Romanian Digital Libraries. [paper]
Melania Nitu and Mihai Dascalu.
Introduce a corpus of approximately 60 k Romanian documents, including human-written samples as well as generated texts using six distinct Large Language Models (LLMs) and three different generation methods. Introduce two techniques for discerning machine-generated texts. The study also includes a text similarity analysis between human-authored and artificially generated texts, coupled with a SHAP analysis.
Does DetectGPT Fully Utilize Perturbation? Selective Perturbation on Model-Based Contrastive Learning Detector would be Better. [paper]
Shengchao Liu, Xiaoming Liu, Yichen Wang, Zehua Cheng, Chengzhengxu Li, Zhaohan Zhang, Yu Lan and Chao Shen.
DetectGPT's random perturbation strategy might introduce noise, and its logit regression module harms the generalizability and applicability of individual or small-batch inputs. Propose a novel detector, PECOLA, which uses selective strategy perturbation to relieve the important information loss caused by random masking, and multi-pair contrastive learning to capture the implicit pattern information during perturbation, facilitating few-shot performance.
LLM-Detector: Improving AI-Generated Chinese Text Detection with Open-Source LLM Instruction Tuning. [paper]
Rongsheng Wang, Haoming Chen, Ruizhe Zhou, Han Ma, Yaofei Duan, Yanlan Kang, Songhua Yang, Baoyu Fan and Tao Tan.
Create a dataset that mixed human-written sentences and sentences polished by LLMs. Propose LLM-Detector, a novel method for both document-level and sentence-level text detection through Instruction Tuning of LLMs. Our method leverages the wealth of knowledge LLMs acquire during pre-training, enabling them to detect the text they generate. Instruction tuning aligns the model's responses with the user's expected text detection tasks.
Detecting Generated Native Ads in Conversational Search. [paper]
Sebastian Schmidt, Ines Zelch, Janek Bevendorff, Benno Stein, Matthias Hagen and Martin Potthast.
This paper investigates whether LLMs can be used as a countermeasure against generated native ads, i.e., to block them. Compile a large dataset of ad-prone queries and of generated answers with automatically integrated ads to experiment with fine-tuned sentence transformers and state-of-the-art LLMs on the task of recognizing the ads.
Ten Words Only Still Help: Improving Black-Box AI-Generated Text Detection via Proxy-Guided Efficient Re-Sampling. [paper]
Yuhui Shi, Qiang Sheng, Juan Cao, Hao Mi, Beizhe Hu and Danding Wang.
Propose to estimate word generation probabilities as pseudo white-box features via multiple re-sampling to help improve AIGT detection under the black-box setting. Design POGER, a proxy-guided efficient re-sampling method, which selects a small subset of representative words for performing multiple re-sampling in black-box AIGT detection.
Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs. [paper]
Zae Myung Kim, Kwang Hee Lee, Preston Zhu, Vipul Raheja and Dongyeop Kang.
This paper delves into the inquiry of identifying discernible and unique linguistic properties in texts that were written by humans, particularly uncovering the underlying discourse structures of texts beyond their surface structures. Introducing a novel methodology, we leverage hierarchical parse trees and recursive hypergraphs to unveil distinctive discourse patterns in texts produced by both LLMs and humans.
Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text. [paper]
Yafu Li, Zhilin Wang, Leyang Cui, Wei Bi, Shuming Shi and Yue Zhang.
Propose a novel detection framework, paraphrased text span detection (PTD), aiming to identify paraphrased text spans within a text. Different from text-level detection, PTD takes in the full text and assigns each of the sentences with a score indicating the paraphrasing degree. Construct a dedicated dataset, PASTED, for paraphrased text span detection.
Who is the Writer?Identifying the Generative Model by Writing Style. [paper]
Anonymous authors.
Present the Identify the Writer by Writing Style (IWWS) model, a novel approach designed to identify the writing styles of human and generative model. Propose a comprehensive dataset, HumanGenTextify. The IWWS model not only achieves high accuracy in text source identification but also provides insights into the distinctive writing styles that characterize human and model-generated texts.
Machine-generated Text Localization. [paper]
Zhongping Zhang, Wenda Qin and Bryan A. Plummer.
Provide the first in-depth study of MGT that localizes the portions of a document that were machine generated. A key challenge in the MGT localization task is that short spans of text, e.g., a single sentence, provides little information indicating if it is machine generated due to its short length. Leverage contextual information, where we predict whether multiple sentences are machine or human written at once.
Whose LLM is it Anyway? Linguistic Comparison and LLM Attribution for GPT-3.5, GPT-4 and Bard. [paper]
Ariel Rosenfeld and Teddy Lazebnik.
Through a comprehensive linguistic analysis, compare the vocabulary, Part-Of-Speech (POS) distribution, dependency distribution, and sentiment of texts generated by three of the most popular LLMS today (GPT-3.5, GPT-4, and Bard) to diverse inputs. The results point to significant linguistic variations which, in turn, enable us to attribute a given text to its LLM origin with a favorable 88% accuracy using a simple off-the-shelf classification model.
Technical Report on the Checkfor.ai AI-Generated Text Classifier. [paper]
Bradley Emi and Max Spero.
Present the Checkfor.ai text classifier, a transformer-based neural network trained to distinguish text written by large language models from text written by humans. Propose a training algorithm, hard negative mining with synthetic mirrors, that enables the classifier to achieve orders of magnitude lower false positive rates on high-data domains such as reviews.
Machine-Generated Text Detection and Attribution. [paper]
Petr Šigut.
Analyse various ways to detect AI-generated content, particularly when encountering content in languages other than English. Test how three AI detection tools, Compilatio, Turnitin and GPT-2 Output Detector, perform with Czech, Slovak and English texts generated by ChatGPT.
Deep Learning Detection Method for Large Language Models-Generated Scientific Content. [paper]
Bushra Alhijawi, Rawan Jarrar, Aseel AbuAlRub and Arwa Bader.
Present a novel ChatGPT-generated scientific text detection method, AI-Catcher. AI-Catcher integrates two deep learning models, multilayer perceptron (MLP) and convolutional neural networks (CNN). In addition, a new ChatGPT-Generated scientific text dataset is collected to enhance AI-generated text detection tools, AIGTxt.
Towards Detecting AI-Generated Text within Human-AI Collaborative Hybrid Texts. [paper]
Zijie Zeng, Shiqi Liu, Lele Sha, Zhuang Li, Kaixun Yang, Sannyuya Liu, Dragan Gašević and Guanliang Chen.
Explore the challenge of sentence-level AI-generated text detection within human-AI collaborative hybrid texts. Utilize the CoAuthor dataset, which includes diverse, realistic hybrid texts generated through the collaboration between human writers and an intelligent writing system in multi-turn interactions. Adopt a two-step, segmentation-based pipeline: (i) detect segments within a given hybrid text where each segment contains sentences of consistent authorship, and (ii) classify the authorship of each identified segment.
Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detection. [paper]
Zhixin Lai, Xuesheng Zhang and Suiyao Chen.
Reveal that single transformer-based classifiers achieved decent performance on in-distribution dataset but limited generalization ability on out-of-distribution dataset. To improve it, combine the individual classifiers models using adaptive ensemble algorithms. The results indicate the effectiveness, good generalization ability, and great potential of adaptive ensemble algorithms in LLM-generated text detection.
EAGLE: A Domain Generalization Framework for AI-generated Text Detection. [paper]
Amrita Bhattacharjee, Raha Moraffah, Joshua Garland and Huan Liu.
Propose a domain generalization framework for the detection of AI-generated text from unseen target generators. The proposed framework, EAGLE, leverages the labeled data that is available so far from older language models and learns features invariant across these generators, in order to detect text generated by an unknown target generator. EAGLE learns such domain-invariant features by combining the representational power of self-supervised contrastive learning with domain adversarial training.
Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models. [paper]
Zhenyu Xu and Victor S. Sheng.
Address the challenge of determining whether a student's code assignment was generated by a language model. First, the proposed method identifies AI-generated code by leveraging targeted masking perturbation paired with comperhesive scoring. Second, utilize a fine-tuned CodeBERT to fill in the masked portions, producing subtle modified samples. Then, integrate the overall perplexity, variation of code line perplexity, and burstiness into a unified score. In this scoring scheme, a higher rank for the original code suggests it's more likely to be AI-generated.
MUGC: Machine Generated versus User Generated Content Detection. [paper]
Yaqi Xie, Anjali Rawal, Yujing Cen, Dixuan Zhao, Sunil K Narang and Shanu Sushmita.
Undertake a comparative evaluation of eight traditional machine-learning algorithms to distinguish between machine-generated and human-generated data across three diverse datasets: Poems, Abstracts, and Essays.
AI-Generated Text Detector for Arabic Language Using Encoder-Based Transformer Architecture. [paper]
Hamed Alshammari,Ahmed El-Sayed and Khaled Elleithy.
Introduce a novel AI text classifier designed specifically for Arabic. Utilized and fine-tuned two Transformer-based models, AraELECTRA and XLM-R, by training them on two distinct datasets: a large dataset comprising 43,958 examples and a custom dataset with 3078 examples that contain HWT and AI-generated texts (AIGTs) from various sources, including ChatGPT 3.5, ChatGPT-4, and BARD.
The Impact of Prompts on Zero-Shot Detection of AI-Generated Text. [paper]
Kaito Taguchi, Yujie Gu and Kouichi Sakurai.
Introduce an evaluative framework to empirically analyze the impact of prompts on the detection accuracy of AI-generated text. Assess various zero-shot detectors using both white-box detection, which leverages the prompt, and black-box detection, which operates without prompt information. Experiments reveal the significant influence of prompts on detection accuracy.
Detection of AI-Generated Text Using Large Language Model. [paper]
Manish Prajapati, Santos Kumar Baliarsingh, Chinmayee Dora, Ashutosh Bhoi, Jhalak Hota and Jasaswi Prasad Mohanty.
Leverage the machine learning (ML) models to reliably identify whether an essay is authored by a human being or by an LLM. Using diverse texts and unknown generative models, we replicate typical scenarios to encourage feature learning across models.
Sentiment analysis and random forest to classify LLM versus human source applied to Scientific Texts. [paper]
Javier J. Sanchez-Medina.
Propose a new methodology to classify texts coming from an automatic text production engine or a human, based on Sentiment Analysis as a source for feature engineering independent variables and then train with them a Random Forest classification algorithm. Using four different sentiment lexicons, a number of new features where produced, and then fed to a machine learning random forest methodology, to train such a model.
Large Language Model (LLM) AI text generation detection based on transformer deep learning algorithm. [paper]
Yuhong Mo, Hao Qin, Yushan Dong, Ziyi Zhu and Zhenglin Li.
In this paper, a tool for detecting LLM AI text generation is developed based on the Transformer model, aiming to improve the accuracy of AI text generation detection and provide reference for subsequent research.
Detecting AI Generated Text Based on NLP and Machine Learning Approaches. [paper]
Nuzhat Prova.
Offer an accurate AI detector model that can differentiate between electronically produced text and human-written text. The approach includes machine learning methods such as XGB Classifier, SVM, BERT architecture deep learning models.
EDMAL - ENHANCED DETECTION OF AI GENERATED TEXT USING MACHINE LEARNING. [paper]
Ang Jun Ray and Chieu Hai Leong.
Aim to enhance existing AI text detection technologies and thus focused on the Divergent N-Gram Analysis’s (DNA-GPT’s) approach. Propose 4 methods to replace the 3rd step in DNA-GPT’s detection process, where texts are compared and scored. The results indicate that the machine-learning based methods, especially the random forest classifier, outperforms the original approach by DNA-GPT.
C-Net: A Compression-Based Lightweight Network for Machine-Generated Text Detection. [paper]
Yinghan Zhou, Juan Wen, Jianghao Jia, Liting Gao and Ziwei Zhang.
Propose a compression-based lightweight network for MGT detection that leverages the ability of lossless compression to effectively extract features between categories. With fewer parameters, the framework achieves state-of-the-art performance in MGT detection under black box conditions.
Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore. [paper]
Junchao Wu, Runzhe Zhan, Derek F. Wong, Shu Yang, Xuebo Liu, Lidia S. Chao and Min Zhang.
Propose an simple but effective black-box zero-shot detection approach, predicated on the observation that human-written texts typically contain more grammatical errors than LLM-generated texts. This ap
Truncated — view the full README on GitHub.
19 followers · starred Jul 2023
39 followers · starred Apr 2024
210 followers · starred Jan 2024
6 followers · starred Oct 2024
