LLM Research Summaries

Tracking ideas of interest, along with some background research to remember our context in AI history.

Contents

• Contents
• Models Review
• Prompting
• Document Processing
• Training
• Steering
• Retrieval Augmented Generation
• Data Science
• Code and Tools
• Interactive Agents
• Computer Use
• Education
• Scientific Research
• Social Science
  • AI Psychology
  • Mental Health
  • Types Predict
  • Traits Detect
• Future Perspectives
• Lore
• Historical
  • Statistical Language Models (SLMs) Era: 1990-2010

Models Review

• Shiksha A Technical Domain focused Translation Dataset and Model for Indian Languages
• A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks
• Deepseek The Quiet Giant Leading China's AI Race
• OpenAI o1 System Card
• Why Worry About Incorrigible Claude
• Number of Parameters in GPT-4 (Latest Data)
• Surveying the MLLM Landscape: A Meta-Review of Current Surveys [MLLM-Survey]
• Evaluation of OpenAI o1: Opportunities and Challenges of AGI [o1_use-case-eval]
• What is the Role of Small Models in the LLM Era: A Survey [small-llm]

Prompting

• (2025-01) CALM Curiosity-Driven Auditing for Large Language Models
• (2024-11) The Grammar of Thought Does Fine-Tuning Make a Difference?
• (2024-09) Thinking Before Speaking: A Role-playing Model with Mindset [Roleplay]
• (2024-10) ComfyGen: Prompt-Adaptive Workflows for Text-to-Image Generation [ImageGen]
• (2024-06) The Prompt Report: A Systematic Survey of Prompting Techniques [Prompt-Methods] [Survey]
• (2023-01) Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing [Prompt-Methods]

Document Processing

• (2024-09) Fully automatic extraction of morphological traits from the Web: utopia or reality?
• (2025-01) AltGen AI-Driven Alt Text Generation for Enhancing EPUB Accessibility [Accessibility]
• (2024-11) LongKey: Keyphrase Extraction for Long Documents [Extraction]
• (2024-10) DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perception [Layout]
• (2024-10) VERITAS-NLI: Validation and Extraction of Reliable Information Through Automated Scraping and Natural Language Inference [Extraction]
• (2024-09) Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits [Editing]
• (2024-09) Evaluation of Large Language Models for Summarization Tasks in the Medical Domain: A Narrative Review [Medical Summary]
• (2024-01) Towards Efficient Methods in Medical Question Answering using Knowledge Graph Embeddings [Q/A]
• (2024-09) Leveraging Long-Context Large Language Models for Multi-Document Understanding and Summarization in Enterprise Applications [Multi-Document]
• (2024-09) Model-based Preference Optimization in Abstractive Summarization without Human Feedback [Preference Optimization]
• (2024-07) Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems [Long Context]
• (2024-09) Attention-Seeker: Dynamic Self-Attention Scoring for Unsupervised Keyphrase Extraction [Keywords]
• (2024-01) Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure [Ingestion]
• (2021-03) Learning Transferable Visual Models From Natural Language Supervision

Training

• (2025-01) Analyzing Fine-tuning Representation Shift for Multimodal LLMs Steering
• (2024-12) Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging
• (2024-10) SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture
• (2024-10) LoRA vs Full Fine-tuning: An Illusion of Equivalence
• (2024-05) LoRA Learns Less and Forgets Less
• (2024-10) LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization
• (2024-01) Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning
• (2024-02) BitDelta: Your Fine-Tune May Only Be Worth One Bit
• (2024-08) BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models
• (2024-04) Inheritune: Training Smaller Yet More Attentive Language Models
• (2024-10) InfiniPot: Efficiently Handling Long Input Contexts in Large Language Models (LLMs) on Memory-Constrained Devices
• (2024-09) Propulsion: Steering LLM with Tiny Fine-Tuning
• (2024-09) Finetuning LLMs for Comparative Assessment Tasks
• (2024-08) Ultimate Guide to Fine-Tuning LLMs: From Basics to Breakthroughs
• (2024-03) Tune without Validation Searching for Learning Rate and Weight Decay on Training Sets

Steering

• (2024-10) Considerations Influencing Offense-Defense Dynamics From Artificial Intelligence
• (2025-01) Autonomous Alignment with Human Value on Altruism through Considerate Self-imagination and Theory of Mind
• (2024-11) Extracting Unlearned Information from LLMs with Activation Steering

Retrieval Augmented Generation

• (2024-12) CypherBench Towards Precise Retrieval over Full-scale Modern Knowledge Graphs in the LLM Era
• (2024-11) Harnessing multiple LLMs for Information Retrieval A case study on Deep Learning methodologies in Biodiversity publications
• (2024-10) Agentic Information Retrieval
• (2024-10) Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains
• (2024-10) Retrieval Instead of Fine-tuning: A Retrieval-based Parameter Ensemble for Zero-shot Learning
• (2024-02) ActiveRAG: Revealing the Treasures of Knowledge via Active Learning RAG
• (2024-10) Retrieval-based Parameter Ensemble for Zero-shot Learning [medical]

Data Science

• (2024-10) I want to break some laws too
• (2023-07) DATA-CENTRIC AI: The Underdog Game Changer in AI's Evolution [Kaggle]
• (2024-09) Data Analysis in the Era of Generative AI [Data-Analysis]
• (2024-09) Data-Prep-Kit: getting your data ready for LLM application development [Application]
• (2024-09) Unified Framework to Classify Business Activities into International Standard Industrial Classification through Large Language Models for Circular Economy [Classification]
• (2024-07) Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows? [Data-Engineering]

Code and Tools

• A Comprehensive Survey of AI-Driven Advancements and Techniques in Automated Program Repair and Code Generation
• HunyuanProver A Scalable Data Synthesis Framework and Guided Tree Search for Automated Theorem Proving
• MCP-Solver Integrating Language Models with Constraint Programming Systems
• STEPTOOL: A STEP-GRAINED REINFORCEMENT LEARNING FRAMEWORK FOR TOOL LEARNING IN LLMS
• ToolGen: Unified Tool Retrieval and Calling via Generation [Tools]
• From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging [Code]
• LLM With Tools: A Survey [Tools]

Interactive Agents

• (2024-11) Navigating the Risks A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents
• (2024-11) OASIS Open Agent Social Interaction Simulations with One Million Agents
• (2024-12) SocialMind A Proactive Social Assistive System for Live Social Interactions Integrating Large Language Models and AR Glasses
• (2024-11) Spontaneous Emergence of Agent Individuality through Social Interactions in LLM-Based Communities [Emergence]
• (2024-10) AFLOW : AUTOMATING AGENTIC WORKFLOW GENERATION (2024-10)
• (2024-10) Conversational Swarms Of Humans And Ai Agents Enable Hybrid Collaborative Decision-Making
• (2024-10) Do great minds think alike? Investigating Human-AI Complementarity in Question Answering with caimira [Review-vs-Human]
• (2024-09) TRAINING LANGUAGE MODELS TO WIN DEBATES WITH SELF-PLAY IMPROVES JUDGE ACCURACY [Self-Play]
• (2024-09) Do LLMs suffer from Multi-Party Hangover? [Multiparty]
• (2024-09) A Survey on Complex Tasks for Goal-Directed Interactive Agents [Agents]

Computer Use

• (2024-10) From Commands to Prompts: LLM-based Semantic File System for AIOS
• (2024-10) Agent S: An Open Agentic Framework that Uses Computers Like a Human [Agent]
• (2024-02) WEBLINX: Real-World Website Navigation with Multi-Turn Dialogue [Web-Naviation]

Education

• (2024-10) Students Rather Than Experts: A New AI For Education Pipeline To Model More Human-Like And Personalised Early Adolescences
• (2024-10) The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment
• (2024-10) Hey AI Can You Grade My Essay?: Automatic Essay Grading [Grading]
• (2024-09) The application of GPT-4 in grading design university students' assignment and providing feedback: An exploratory study [Grading]
• (2024-09) Exploring the Use of ChatGPT for a Systematic Literature Review: a Design-Based Research [Review] [Science]
• (2024-09) LLMs in Education: Novel Perspectives, Challenges, and Opportunities [Review] [Learning]

Scientific Research

• (2024-12) LLMs4Life Large Language Models for Ontology Learning in Life Sciences
• (2024-12) Navigating the landscape of multimodal AI in medicine a scoping review on technical challenges and clinical applications
• (2024-12) Ontology-grounded Automatic Knowledge Graph Construction by LLM under Wikidata schema
• (2024-10) HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation
• (2024-10) Society of Medical Simplifiers
• (2024-10) Two Heads Are Better Than One: A Multi-Agent System Has the Potential to Improve Scientific Idea Generation
• (2024-10) ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
• (2024-09) LANGUAGE AGENTS ACHIEVE SUPERHUMAN SYNTHESIS OF SCIENTIFIC KNOWLEDGE [Science]

Social Science

• Large language models surpass human experts in predicting neuroscience results - Nature Human Behaviour [Scientific-Research]
• Artificial Intelligence Applications for Social Science Research [Research Tools]
• Evaluating LLMs Capabilities Towards Understanding Social Dynamics
• Automated Social Science: Language Models as Scientist and Subjects
• Large Language Models based on historical text could offer informative tools for behavioral science [History] [Behavioral-Science]
• PropaInsight: Toward Deeper Understanding of Propaganda in Terms of Techniques, Appeals, and Intent [Social-Psychology]
• Autoformalization of Game Descriptions using Large Language Models [Game-Theory]

AI Psychology

• Large Language Models Reflect the Ideology of their Creators
• Can a Large Language Model be a Gaslighter?
• Evaluating Psychological Safety of Large Language Models
• Measuring Human and AI Values based on Generative Psychometrics with Large Language Models
• Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challenges

Mental Health

• The Emotional Spectrum of LLMs Leveraging Empathy and Emotion-Based Markers for Mental Health Support- MentalArena: Self-play Training of Language Models for Diagnosis and Treatment of Mental Health Disorders [Diagnosis-Treatment]
• Severity Prediction in Mental Health: LLM-based Creation, Analysis, Evaluation of a Novel Multilingual Dataset
• A COMPREHENSIVE EVALUATION OF LARGE LANGUAGE MODELS ON MENTAL ILLNESS
• DISCO: A Hierarchical Disentangled Cognitive Diagnosis Framework for Interpretable Job Recommendation
• Mapping neurotransmitter systems to the structural and functional organization of the human neocortex [Neural-Correlates]

Types Predict

• Myers-Briggs Personality Type Prediction Based on Reddit Comments [MBTI]
• Machine Mindset: An MBTI Exploration of Large Language Models [MBTI]

Traits Detect

• Traversing Emotional Landscapes and Linguistic Patterns in Bernard-Marie
• Research on Predicting Public Opinion Event Heat Levels Based on Large Language Models [Social-Science]
• Words that Represent Peace
• Harnessing Large Language Models: Fine-tuned BERT for Detecting Charismatic Leadership Tactics in Natural Language [Personality-Detect]

Future Perspectives

• Beware the Intention Economy Collection and Commodification of Intent via Large Language Models
• Dystopian Dreams, Utopian Nightmares: AI and the Permanence of Racism
• The Future of Agents: AI is about to completely change how you use computers [Gates]
• Leopold Aschenbrenner's Situational Awareness + The Decade Ahead (2024) [aschenbrenner]

Lore

• Memes, Markets, and Machines: The Evolution of On-Chain Autonomy through Hyperstition
• Can AI Meditate? [Claude-3] [Sonnet]
• When AIs Play God(se): The Emergent Heresies of LLMtheism [Truth-Terminal] [Claude-3] [Opus]

Historical

• (2025-01) Artificial Intelligence in Creative Industries Advances Prior to 2025
• (2023-10) Hermes Lecture #3 Why Do Cognitive Scientists Hate LLMs
• (2016-09) Le deep learning est-il le futur du text mining

Statistical Language Models (SLMs) Era: 1990-2010

• (1990) Hidden Markov Models for speech recognition (Rabiner) [Voice-Command-Systems]
• (1993) IBM Model 1 for statistical machine translation (Brown et al.) [Early-Online-Translation]
• (1995) Improved backing-off for M-gram language modeling (Kneser & Ney) [Spell-Checkers]
• (1996) Maximum Entropy Models (Berger et al.) [Text-Classification]
• (1999) An empirical study of smoothing techniques for language modeling (Chen & Goodman) [Improved-Language-Models]
• (2002) Latent Dirichlet Allocation (LDA) (Blei et al.) [Document-Clustering]
• (2006) Hierarchical Pitman-Yor language model (Teh) [Text-Generation]

Resources

• Agents for Computer Use: A curated list of resources about AI agents for Computer Use, including research papers, projects, frameworks, and tools.
• arXiv - arXiv is a free distribution service and an open-access archive for nearly 2.4 million scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics.
• CatalyzeX - Get code for papers directly on Google, Arxiv, Scholar, Twitter, and more
• Papers With Code - The mission of Papers with Code is to create a free and open resource with Machine Learning papers, code, datasets, methods and evaluation tables.