ValueByte-AI/Awesome-LLM-in-Social-Science

Awesome papers involving LLMs in Social Science.

649

194 commits

updated Sep 24, 2026

README

🔗 Recommended Resource:
Check out Awesome-LLM-Psychometrics for a comprehensive collection of papers and resources on LLM psychometrics, including evaluation, validation, and enhancement.

Below we compile awesome papers that

evaluate Large Language Models (LLMs) from a perspective of Social Science.
align LLMs from a perspective of Social Science.
employ LLMs to facilitate research, address issues, and enhance tools in Social Science.
contribute surveys, perspectives, and datasets on the above topics.

The above taxonomies are by no means orthogonal. For example, evaluations require simulations. We categorize these papers based on our understanding of their focus. This collection has a special focus on Psychology and intrinsic values.

Welcome to contribute and discuss!

🤩 Papers marked with a ⭐️ are contributed by the maintainers of this repository. If you find them useful, we would greatly appreciate it if you could give the repository a star and cite our paper.

@article{ye2025large,
  title={Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement},
  author={Ye, Haoran and Jin, Jing and Xie, Yuhang and Zhang, Xin and Song, Guojie},
  journal={arXiv preprint arXiv:2505.08245},
  year={2025},
  note={Project website: \url{https://llm-psychometrics.com}, GitHub: \url{https://github.com/ValueByte-AI/Awesome-LLM-Psychometrics}}
}

1. 📚 Survey
1. 🗂️ Dataset
1. 🔎 Evaluating LLM
- 3.1. ❤️ Value
- 3.2. 🩷 Personality
- 3.3. 🔞 Morality
- 3.4. 🎤 Opinion
- 3.5. 💚 General Preference
- 3.6. 🧠 Ability
- 3.7. ⚠️ Risk
1. ⚒️ Tool enhancement
1. ⛑️ Alignment
- 5.1. 🌈 Pluralistic Alignment
1. 🚀 Simulation
1. 👁️‍🗨️ Perspective and Position

1. 📚 Survey

Mapping the Emerging Social Science of Large Language Models, arXiv, 2026.09, [paper].
⭐️ Large Language Model Psychometrics: A Systematic Review of Evaluation, Validation, and Enhancement, 2025.05, [paper].
Missing the Margins: A Systematic Literature Review on the Demographic Representativeness of LLMs, ACL 2025, [paper], [collection].
Large language models (LLM) in computational social science: prospects, current state, and challenges, 2025.03, Social Network Analysis and Mining, [paper].
Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents, 2025.03, [paper].
On the Trustworthiness of Generative Foundation Models: Guideline, Assessment, and Perspective, 2025.02, [paper].
The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment, 2024.12, [paper].
Large Language Model Safety: A Holistic Survey, 2024.12, [paper].
Political-LLM: Large Language Models in Political Science, 2024.12, [paper], [website].
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods, 2024.12, [paper].
From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents, 2024.12, [paper], [repo].
A Survey on Human-Centric LLMs, 2024.11, [paper].
Survey of Cultural Awareness in Language Models: Text and Beyond, 2024.11, [paper].
How developments in natural language processing help us in understanding human behaviour, 2024.10 Nature Human Behavior, [paper].
How large language models can reshape collective intelligence, 2024.09, Nature Human Behavior, [paper].
Automated Mining of Structured Knowledge from Text in the Era of Large Language Models, 2024.08, KDD 2024, [paper].
Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspective, 2024.07, [paper].
Perils and opportunities in using large language models in psychological research, 2024.07, [paper].
The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models, 2024.06, [paper].
Can Generative AI improve social science?, 2024.05, PNAS, [paper].
Foundational Challenges in Assuring Alignment and Safety of Large Language Models, 2024.04, [paper].
Large Language Model based Multi-Agents: A Survey of Progress and Challenges, 2024.01, [paper], [repo].
The Rise and Potential of Large Language Model Based Agents: A Survey, 2023, [paper], [repo].
A Survey on Large Language Model based Autonomous Agents, 2023, [paper], [repo].
AI Alignment: A Comprehensive Survey, 2023.11, [paper], [website].
Aligning Large Language Models with Human: A Survey, 2023, [paper], [repo].
Large Language Model Alignment: A Survey, 2023, [paper].
Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives, 2023.12, Nature humanities and social sciences communications, [paper].
A Survey on Evaluation of Large Language Models, 2023.07, [paper], [repo].
From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models, 2023.08, [paper], [repo].
Concerns on the use of generative AI in social science research, [website], [repo]

2. 🗂️ Dataset

Can LLMs Imagine Moral Alternatives Beyond Binary Dilemmas?, arXiv, 2026.09, [paper], [dataset], [code].
AI Job Displacement Tracker — Structured, source-backed dataset tracking 96 AI-attributed workforce reductions (457K workers affected, 13 countries, 13 sectors). Every entry includes source URLs, attribution tier, and job functions. Available in JSON/CSV under CC-BY-4.0.
Mental Health Datasets
Datasets for depression detection using data posted on online platforms
Benchmarking Multi-National Value Alignment for Large Language Models, 2025.04, [paper].
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values, 2025.04, [paper].
Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?, 2025.01, [paper].
The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models, NeurIPS 2024 D&B Track best paper, [paper].
⭐️ ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models, ACL 2024, [paper], [code].
Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models, NeurIPS 2024, [paper].
https://github.com/CLUEbenchmark/CLUEDatasetSearch
HATEDAY: Insights from a Global Hate Speech Dataset Representative of a Day on Twitter, ACL 2025 Outstanding Paper, [paper].
https://lit.eecs.umich.edu/downloads.html
COMPO: Community Preferences for Language Model Personalization, 2024.10, [paper].
Cultural Commonsense Knowledge for Intercultural Dialogues, CIKM 2024, [paper], [dataset].

3. 🔎 Evaluating LLM

3.1. ❤️ Value

A Theory of Response Sampling in LLMs: Part Descriptive and Part Prescriptive, ACL 2025 Best Paper, [paper].
Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs, ACL 2025 Best Paper, [paper].
Towards Better Value Principles for Large Language Model Alignment: A Systematic Evaluation and Enhancement, ACL 2025 SAC Highlight, [paper].
⭐️ Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models, 2025.02, [paper].
⭐️ Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models, AAAI 2025, [paper], [code].
⭐️ ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models, ACL 2024, [paper], [code].
Value Profiles for Encoding Human Variation, 2025.03, [paper].
Simulating Human-like Daily Activities with Desire-driven Autonomy, ICLR 2025, [paper], [arXiv], [code].
Beyond Self-Interest: Modeling Social-Oriented Motivation for Human-like Multi-Agent Interactions, AAMAS 2026 Oral, [paper], [code].
Value Compass Leaderboard: A Platform for Fundamental and Validated Evaluation of LLMs Values, 2025.01, [paper].
Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective, ACL 2025 Findings, [paper], [arXiv].
NORMAD: A Framework for Measuring the Cultural Adaptability of Large Language Models, 2024.10, [paper].
ValueCompass: A Framework of Fundamental Values for Human-AI Alignment, 2024.09, [paper].
LOCALVALUEBENCH: A Collaboratively Built and Extensible Benchmark for Evaluating Localized Value Alignment and Ethical Safety in Large Language Models, 2024.08, [paper].
Stick to your role! Stability of personal values expressed in large language models, 2024.08, [paper].
Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing, 2024.07, [paper].
Do LLMs have Consistent Values?, 2024.07, [paper].
CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses, 2024.07, [paper].
Are Large Language Models Consistent over Value-laden Questions?, 2024.07, [paper].
Beyond Human Norms: Unveiling Unique Values of Large Language Models through Interdisciplinary Approaches, 2024.04, [paper].
Heterogeneous Value Evaluation for Large Language Models, 2023.03, [paper], [code].
Measuring Value Understanding in Language Models through Discriminator-Critique Gap, 2023.10, [paper].
Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values, 2023.11, [paper].
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties, AAAI24, [paper], [code].
High-Dimension Human Value Representation in Large Language Models, 2024.04, [paper], [code].

3.2. 🩷 Personality

A psychometric framework for evaluating and shaping personality traits in large language models, Nature Machine Intelligence, 2025, [paper].
R.U.Psycho? Robust Unified Psychometric Testing of Language Models, 2025.03, [paper].
Evaluating the ability of large language models to emulate personality, 2025.01, Nature Scientific Reports, [paper].
Evaluating the efficacy of LLMs to emulate realistic human personalities, AAAI 2024, [paper].
Quantifying ai psychology: A psychometrics benchmark for large language models, 2024.07, [paper].
Incharacter: Evaluating personality fidelity in role-playing agents through psychological interviews, ACL 2024, [paper], [code]
[MBTI] Open Models, Closed Minds? On Agents Capabilities in Mimicking Human Personalities through Open Large Language Models, 2024.01, [paper]
Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench, ICLR 2024, [paper], [code]
[BFI] AI Psychometrics: Assessing the Psychological Profiles of Large Language Models Through Psychometric Inventories, Journal, 2024.01, [paper]
Does Role-Playing Chatbots Capture the Character Personalities? Assessing Personality Traits for Role-Playing Chatbots, 2023.10, [paper]
[MBTI] Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models, 2023.07, [paper]
[MBTI] Can ChatGPT Assess Human Personalities? A General Evaluation Framework, 2023.03, EMNLP 2023, [paper], [code].
[BFI] Personality Traits in Large Language Models, 2023.07, [paper]
[BFI] Revisiting the Reliability of Psychological Scales on Large Language Models, 2023.05, [paper]
[BFI] Systematic Evaluation of GPT-3 for Zero-Shot Personality Estimation, ACL 2023 workshop, [paper]
[BFI] Have Large Language Models Developed a Personality?: Applicability of Self-Assessment Tests in Measuring Personality in LLMs, 2023.05, [paper]
[BFI] Evaluating and Inducing Personality in Pre-trained Language Models, NeurIPS 2023 (spotlight), [paper]
[BFI] Identifying and Manipulating the Personality Traits of Language Models, 2022,12, [paper]
Who is GPT-3? An Exploration of Personality, Values and Demographics, 2022.09, [paper]
Does GPT-3 Demonstrate Psychopathy? Evaluating Large Language Models from a Psychological Perspective, 2022.12, [paper]

3.3. 🔞 Morality

Are Rules Meant to be Broken? Understanding Multilingual Moral Reasoning as a Computational Pipeline with UniMoral, ACL 2025 Best Resource Paper, [paper].
How do Role Models Shape Collective Morality? Exemplar-Driven Moral Learning in Multi-Agent Simulation, ACL 2026 Main, [paper], [code].
Why Are We Moral? An LLM-based Agent Simulation Approach to Study Moral Evolution, ACL 2026 Main (Oral), [paper], [code].
Revealing the Pragmatic Dilemma for Moral Reasoning Acquisition in Language Models, 2025.02, [paper].
Normative Evaluation of Large Language Models with Everyday Moral Dilemmas, 2025.02, [paper].
Investigating machine moral judgement through the Delphi experiment, Nature Machine Intelligence, 2025.01, [paper].
The Judgment-Consequence Gap: LLM Moral Reasoning in Healthcare Decisions, 2026.08, [paper].
Moral Foundations of Large Language Models, 2023.10, [paper].
Exploring the psychology of GPT-4's Moral and Legal Reasoning, 2023.08, [paper].
Probing the Moral Development of Large Language Models through Defining Issues Test
Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity, 2023.06, [paper]
Evaluating the Moral Beliefs Encoded in LLMs, 2023.07, [paper]

3.4. 🎤 Opinion

The AI Gap: How Socioeconomic Status Affects Language Technology Interactions, ACL 2025 Best Social Impact Paper, [paper].
Persuading voters using human–artificial intelligence dialogues, Nature, 2025.12, [paper].
Generative language models exhibit social identity biases, Nature Computational Science, 2025.01, [paper].
Only a Little to the Left: A Theory-grounded Measure of Political Bias in Large Language Models, ACL 2025, [paper].
Generative AI Meets Open-Ended Survey Responses: Research Participant Use of AI and Homogenization, 2025.05, Sociological Methods & Research, [paper].
Machine Bias. How Do Generative Language Models Answer Opinion Polls?, 2025.04, Sociological Methods & Research, [paper].
Large language models that replace human participants can harmfully misportray and flatten identity groups, Nature Machine Intelligence, 2025, [paper].
Questioning the Survey Responses of Large Language Models, NeurIPS 2024 Oral, [paper].
Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models, ACL 2024, paper.
More human than human: measuring ChatGPT political bias, 2023, [paper].
Towards Measuring the Representation of Subjective Global Opinions in Language Models, 2023.07, [paper], [website].

3.5. 💚 General Preference

Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond), NeurIPS 2025 D&B Track Best Paper, [paper].
Diverging Preferences: When do Annotators Disagree and do Models Know?, 2024.10, [paper].

3.6. 🧠 Ability

Large language models surpass human experts in predicting neuroscience results, Nature Human Behaviour, 2025.02, [paper].
Quantifying large language model usage in scientific papers, Nature Human Behaviour, 2025, [paper].
Large language models without grounding recover non-sensorimotor but not sensorimotor features of human concepts, Nature Human Behaviour, 2025, [paper].
Visual cognition in multimodal large language models, Nature Machine Intelligence, 2025.01, [paper].
Testing theory of mind in large language models and humans, Nature Human Behaviour, 2024.07, [paper].
Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation, 2025, [paper], [post].
Improving LLM Leaderboards with Psychometrical Methodology, 2025.01, [paper].
AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios, NAACL 2025, [paper].
A Novel Psychometrics-Based Approach to Developing Professional Competency Benchmark for Large Language Models, 2024.11, [paper].
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates, NeurIPS 2024, [paper].
Replication for Language Models: Problems, Principles, and Best Practice for Political Science, 2024.10, [paper].
Can Language Models Reason about Individualistic Human Values and Preferences?, 2024.10, [paper].
Language Models in Sociological Research: An Application to Classifying Large Administrative Data and Measuring Religiosity, 2021, [paper].
Can Large Language Models Transform Computational Social Science?, 2023, [paper], [code].
SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents, 2023, [paper], [code].
Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View, 2023, [paper], [code].
Playing repeated games with large language models, Nature Human Behaviour, 2025.05, [paper], [arXiv].
Machine Psychology: Investigating Emergent Capabilities and Behavior in Large Language Models Using Psychological Methods, 2023, [paper].
Using cognitive psychology to understand GPT-3, 2023.02, PNAS, [paper].
Large language models as a substitute for human experts in annotating political text, 2024.02, [paper].

3.7. ❤️ Risk and Safety

Implicit-bias-like patterns in reasoning models, Nature Machine Intelligence, 2026.09, [paper], [code].
When Does Defendant Statement Matter? A Study of Bias and Persuasion in LLM-Simulated Jurors, arXiv, 2026.09, [paper].
From Lived Experience to Insight: Unpacking the Psychological Risks of Using AI Conversational Agents, 2024.12, [paper].
Exploring Social Desirability Response Bias in Large Language Models: Evidence from GPT-4 Simulations, 2024.10, [paper].
AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies, 2024.07, [paper].

4. ⚒️ Tool enhancement

Past Meets Present: Creating Historical Analogy with Large Language Models, ACL 2025 Outstanding Paper, [paper], [code].
A reporting checklist for large language models in behavioural science, Nature Human Behaviour, 2026, [paper].
Durably reducing conspiracy beliefs through dialogues with AI, Science, 2024.09, [paper].
⭐️ Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models, 2025.02, [paper].
⭐️ Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models, AAAI 2025, [paper], [code].
STAMPsy: Towards SpatioTemporal-Aware Mixed-Type Dialogues for Psychological Counseling, AAAI 2025, [paper].
Psychogat: A novel psychological measurement paradigm through interactive fiction games with llm agents, ACL 2024, [paper].
ValueScope: Unveiling Implicit Norms and Values via Return Potential Model of Social Interactions, 2024.10, [paper].
AI can help humans find common ground in democratic deliberation, 2024.10, Science, [paper].
PsyDI: Towards a Personalized and Progressively In-depth Chatbot for Psychological Measurements, 2024, [paper], [code].
ChatFive: Enhancing User Experience in Likert Scale Personality Test through Interactive Conversation with LLM Agents, CUI 2024, [paper]
LLM Agents for Psychology: A Study on Gamified Assessments, 2024.02, [paper].
Generative Social Choice, 2023.09, [paper]
Leveraging large language models to map triggers of contamination-related obsessive-compulsive symptoms, Communications Psychology, 2026.08, [paper].

5. ⛑️ Alignment

On the Alignment of Large Language Models with Global Human Opinion, AAAI 2026 Best Paper (AI Alignment Track), [paper].
Language Models Resist Alignment: Evidence From Data Compression, ACL 2025 Best Paper, [paper].
Simple Role Assignment is Extraordinarily Effective for Safety Alignment, ACL 2026 Findings, [paper].
Communication-Efficient Desire Alignment for Proactive Embodied Human–Agent Interaction, ACL 2026 Main (Oral), [paper].
Implicit Behavioral Alignment of Language Agents in High-Stakes Crowd Simulations, EMNLP 2025, [paper].
COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values, 2025.04, [paper].
⭐️ Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models, 2025.02, [paper].
Optimizing generative AI by backpropagating language model feedback, Nature, 2025.03, [paper].
Position: We Need An Adaptive Interpretation of Helpful, Honest, and Harmless Principles, 2025.02, [paper].
Scopes of Alignment, 2025.01, AAAI 2025 workshop, [paper]
Aligning Large Language Models with Human Opinions through Persona Selection and Value-Belief-Norm Reasoning, 2024.11, [paper].
SafetyAnalyst: Interpretable, transparent, and steerable LLM safety moderation, 2024.10, [paper].
Moral Alignment for LLM Agents, 2024.10, [paper].
ProgressGym: Alignment with a Millennium of Moral Progress, NeurIPS 2024 D&B Track Spotlight, [paper], [code].
Strong and weak alignment of large language models with human values, 2024.08, Nature Scientific Reports, [paper].
STELA: a community-centred approach to norm elicitation for AI alignment, 2024.03, Nature Scientific Reports, [paper].
A Roadmap to Pluralistic Alignment, ICML 2024, [paper], [code].
[Value] What are human values, and how do we align AI to them?, 2024.04, [paper].
Agent Alignment in Evolving Social Norms, 2024.01, [paper].
[Norm] Align on the Fly: Adapting Chatbot Behavior to Established Norms, 2023.12, [paper], [code].
[MBTI] Machine Mindset: An MBTI Exploration of Large Language Models, 2023.12, [paper], [code].
Training Socially Aligned Language Models in Simulated Human Society, 2023, [paper], [code].
Fine-tuning language models to find agreement among humans with diverse preferences, 2022, [paper].
ValueNet: A New Dataset for Human Value Driven Dialogue System, AAAI 2022, [paper], [dataset].
Towards Pluralistic Value Alignment: Aggregating Value Systems through ℓp-Regression, AAMAS 2022 workshop, [paper].

5.1. 🌈 Pluralistic Alignment

[Benchmark] Benchmarking Distributional Alignment of Large Language Models, 2024.11, [paper].
Legal Theory for Pluralistic Alignment, 2024.10, [paper].
Navigating the Cultural Kaleidoscope: A Hitchhiker's Guide to Sensitivity in Large Language Models, 2024.10, [paper], [code and data].
PAD: Personalized Alignment at Decoding-Time, 2024.10, [paper].
Policy Prototyping for LLMs: Pluralistic Alignment via Interactive and Collaborative Policymaking, 2024.09, [paper].
Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration, 2024.06, [paper].
Self-Pluralising Culture Alignment for Large Language Models, 2024.10, [paper].
Cultural Palette: Pluralising Culture Alignment via Multi-agent Palette, 2024.12, [paper].
PALMs: Using Multi Construct-Grounded Rationales for Modeling Population Preferences in LLMs, 2026.08, [paper].

6. 🚀 Simulation

But How Would AI Agents Run a Town's Economy?, arXiv, 2026.09, [paper].
How a Chatbot's Response Style Shapes a Classroom: A Multi-Agent Simulation of Students Consulting AI, arXiv, 2026.09, [paper].
When Persona Attributes Improve Population Alignment in Large Language Models, arXiv, 2026.09, [paper].
CityReal: Human-Aligned Urban Behavior and City Dynamics Simulation with Large-Scale LLM Agents, arXiv, 2026.07, [paper].
Large language models can predict the results of social science experiments, Nature, 2026.07, [paper].
A foundation model to predict and capture human cognition, Nature, 2025.07, [paper].
Frame Entrepreneurs in an AI Agent Community: Concentrated Identity-Claim Production on Moltbook, ASA CITAMS Roundtable 2026, [paper].
Beyond Self-Interest: Modeling Social-Oriented Motivation for Human-like Multi-Agent Interactions, AAMAS 2026 Oral, [paper], [code].
How do Role Models Shape Collective Morality? Exemplar-Driven Moral Learning in Multi-Agent Simulation, ACL 2026 Main, [paper], [code].
EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation, ICML 2026, [paper], [website].
Why Are We Moral? An LLM-based Agent Simulation Approach to Study Moral Evolution, ACL 2026 Main (Oral), [paper], [code].
TwinMarket: A Scalable Behavioral and Social Simulation for Financial Markets, 2025.02, [paper].
AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society, 2025.02, [paper], [code].
Political Actor Agent: Simulating Legislative System for Roll Call Votes Prediction with Large Language Models, AAAI 2025, [paper].
TrendSim: Simulating Trending Topics in Social Media Under Poisoning Attacks with LLM-based Multi-agent System, 2024.12, [paper].
Can Large Language Model Agents Simulate Human Trust Behavior?, NeurIPS 2024, [paper].
OASIS: Open Agents Social Interaction Simulations on One Million Agents, 2024.11, [paper], [code].
Generative Agent Simulations of 1,000 People, 2024.11, [paper].
Social Science Meets LLMs: How Reliable Are Large Language Models in Social Simulations?, 2024.11, [paper].
Multi-expert Prompting Improves Reliability, Safety, and Usefulness of Large Language Models, EMNLP 2024, [paper].
Simulating Opinion Dynamics with Networks of LLM-based Agents, NAACL Findings 2024, [paper] [code]
Beyond demographics: Aligning role-playing llm-based agents using human belief networks, EMNLP Findings 2024, [paper]
The Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents, CogSci 2024, [paper]
Large Language Models can Achieve Social Balance, 2024.10, [paper].
On the limits of agency in agent-based models, 2024.09, [paper], [code].
United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections, 2024.09, [paper].
Out of One, Many: Using Language Models to Simulate Human Samples, 2022, [paper].
Social Simulacra: Creating Populated Prototypes for Social Computing Systems, 2022, [paper].
Generative Agents: Interactive Simulacra of Human Behavior, 2023, [paper], [code].
Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies, 2023, [paper], [code].
Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?, 2023 [paper], [code].
$S^3$: Social-network Simulation System with Large Language Model-Empowered Agents, 2023, [paper].
Rethinking the Buyer's Inspection Paradox in Information Markets with Language Agents, 2023, [paper].
SocioDojo: Building Lifelong Analytical Agents with Real-world Text and Time Series, 2023, [paper].
Humanoid Agents: Platform for Simulating Human-like Generative Agents, 2023, [paper], [code].
When Large Language Model based Agent Meets User Behavior Analysis: A Novel User Simulation Paradigm, 2023, [paper], [code].
Large Language Model-Empowered Agents for Simulating Macroeconomic Activities, 2023, [paper].
Generative Agent-Based Modeling: Unveiling Social System Dynamics through Coupling Mechanistic Models with Generative Artificial Intelligence, 2023, [paper].
Using Imperfect Surrogates for Downstream Inference: Design-based Supervised Learning for Social Science Applications of Large Language Models, 2023.06, NeurIPS 2023, [paper].
Epidemic Modeling with Generative Agents, 2023.07, [paper], [code].
Emergent analogical reasoning in large language models, 2023.08, nature human behavior, [paper].
MetaAgents: Simulating Interactions of Human Behaviors for LLM-based Task-oriented Coordination via Collaborative Generative Agents, 2023.10, [paper].
War and Peace (WarAgent): Large Language Model-based Multi-Agent Simulation of World Wars, 2023.11, [paper], [code].
Emergence of Social Norms in Large Language Model-based Agent Societies, 2024.03, [paper], [code].
Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior, ICLR-2024, [paper]
Representational Equality in Cross-country Value Simulation: A Systematic Analysis of Large Language Models, 2026.08, [paper].
Role Steering of Language Models for Social Simulations, 2026.08, [paper].

7. 👁️‍🗨️ Perspective and Position

Large AI models are cultural and social technologies, Science, 2025.03, [paper].
Position: Simulating Society Requires Simulating Thought, 2025.06, [paper].
The benefits, risks and bounds of personalizing the alignment of large language models to individuals, 2024.04, Nature Machine Intelligence, [paper].
A social path to human-like artificial intelligence, 2023.11, Nature Machine Intelligence, [paper].
Using large language models in psychology, 2023.10, Nature reviews psychology, [paper].

alignment

economics

large-language-models

llm-agent

llm-evaluation

llms

policy

psychology

simulation-environment

social-network

social-science

Contributors

181 commits

3 commits

1 commits

1 commits

Similar repos

ValueByte-AI/Awesome-LLM-Psychometrics

Awesome paper in LLM Psychometrics and LLM Psychology

136

same org

ValueByte-AI/ValueBench

[ACL 2024] ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of…

same org

FudanDISC/SocialAgent

A collection of resources that investigate social agents.

247

MLGroupJLU/LLM-eval-survey

The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

1,611

llm-as-a-judge/Awesome-LLM-as-a-judge

579

penguinnnnn/awesome-llm-and-society

Recent papers on (1) Psychology of LLMs; (2) Biases in LLMs.

Magnetic2014/llm-alignment-survey

A curated reading list for large language model (LLM) alignment. Take a look at our new survey…

tjunlp-lab/Awesome-LLMs-Evaluation-Papers

The papers are organized according to our survey: Evaluating Large Language Models: A Comprehensive…

810