Curated AI-agent papers for building autonomous security bots — offensive AI, multi-agent architecture, memory, and tool use.
8
9 commits
updated May 28, 2026
README
Awesome Autonomous Hacking Agents
A curated reading list of AI-agent papers for building autonomous security bots — agents that discover, analyze, and exploit vulnerabilities with minimal human input.
Contents
- Categories
- Papers
- 1. Core Offensive AI — agents that actually hack things
- 2. Agent Architecture — how to structure the system
- 3. Memory & Knowledge Retention — how the bot learns over time
- 4. Tool Use & Execution — how the bot takes actions in the world
- 5. Benchmarks & Threat Landscape — what's realistic today
- Field Reports — practitioners who shipped real systems
- Contributing
Categories
Each category answers a different question a builder would ask. Read them in order — later categories assume you've already thought through earlier ones.
| # | Category | The Question It Answers |
|---|---|---|
| 1 | Core Offensive AI | Can this paper teach me how an agent actually hacks something? |
| 2 | Agent Architecture | How do I structure agents to handle complex multi-step hacking tasks? |
| 3 | Memory & Knowledge Retention | How does my bot remember what it learned and reuse it? |
| 4 | Tool Use & Execution | How does my bot run commands, write scripts, and take real-world actions? |
| 5 | Benchmarks & Threat Landscape | What can frontier models realistically do today, and what will I face in the wild? |
| — | Field Reports | How did people who actually shipped a hacking bot build it? |
See CLAUDE.md for detailed inclusion criteria.
Papers
1. Core Offensive AI
| Paper | ⭐ | Description | arXiv |
|---|---|---|---|
| To Defend Against Cyber Attacks, We Must Teach AI Agents to Hack | ⭐⭐ | Argues that AI-agent-driven cyberattacks are inevitable and that building frontier offensive AI responsibly is essential defensive infrastructure. Surveys the landscape of what agents can already do offensively, with a blueprint for responsible capability development. Read this first — it frames the entire space. | 2602.02595 |
| A Dual-Loop Agent Framework for Automated Vulnerability Reproduction | ⭐⭐⭐ | Takes a CVE description as input and produces a working exploit using two feedback loops — an outer strategy loop and an inner code loop. The dual-loop pattern separates high-level attack planning from low-level code generation, preventing strategy drift when code fails. Directly copy this architecture for your bot's exploit-generation pipeline. | 2602.05721 |
| Capture the Flags: Family-Based Evaluation of Agentic LLMs | ⭐⭐ | Generates families of equivalent CTF challenges via code transformations to test whether an agent truly understands exploits or just memorizes patterns. Exposes the difference between genuine exploit understanding and surface-level pattern matching. Use this benchmark architecture to evaluate your own bot. | 2602.05523 |
| TxRay: Agentic Postmortem of Live Blockchain Attacks | ⭐⭐⭐ | Reconstructs exploit lifecycles from limited on-chain evidence and generates runnable PoC reproductions. The agent chains together sparse signals into a coherent attack narrative, then translates that narrative into executable code. A concrete end-to-end pipeline from intelligence to working exploit. | 2602.01317 |
| Identifying Adversary Tactics and Techniques in Malware Binaries with an LLM Agent | ⭐⭐⭐ | Uses an LLM agent to map MITRE ATT&CK techniques in stripped malware binaries through incremental context retrieval. Solves the problem of limited context windows on large binaries by feeding evidence incrementally. Use this when your bot needs to reason about existing malware or analyze unfamiliar binaries. | 2602.06325 |
| VirtualCrime: Evaluating Criminal Potential of LLMs via Sandbox Simulation | ⭐⭐ | Evaluates what LLMs can execute in sandboxed offensive scenarios without specialized prompting. Gives empirical ground truth on which models succeed at real-world attack tasks and which prompting strategies are effective. Read before choosing a backbone model for your bot. | 2601.13981 |
| AgenticSCR: An Autonomous Agentic Secure Code Review for Immature Vulnerabilities Detection | ⭐⭐⭐ | Autonomous vulnerability detection via tool invocation and security-focused semantic memory for pre-commit code review. The memory architecture stores "what to look for" as indexed security patterns rather than raw rules. Reuse the memory design for your bot's vulnerability-discovery phase. | 2601.19138 |
| Multimodal Multi-Agent Ransomware Analysis Using AutoGen | ⭐⭐⭐ | A working AutoGen-based multi-agent system for malware analysis that divides static analysis, dynamic analysis, and reporting across specialized agents. Shows how to wire AutoGen agents together for a security-specific task with concrete role definitions. A practical reference implementation to fork. | 2601.20346 |
| Sifting the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering | ⭐⭐ | Head-to-head benchmark of Aider, OpenHands, and SWE-agent on vulnerability triage tasks across multiple backbone models. Measures how agent design — not just model capability — affects security task performance. Read this before picking which agent framework to build on. | 2601.22952 |
| LLM Agents can Autonomously Exploit One-day Vulnerabilities | ⭐⭐⭐ | A single GPT-4 agent autonomously exploits 87% of 15 critical-severity real-world one-day CVEs, while GPT-3.5, every tested open-source model, and commercial scanners (ZAP, Metasploit) all score 0%. The agent runs a tool-calling loop against live vulnerable systems using only the CVE description and public documentation as input. This is the foundational capability baseline — use it to calibrate model selection and understand the ceiling a single-model agent can reach before adding multi-agent complexity. | 2404.08144 |
| Teams of LLM Agents can Exploit Zero-Day Vulnerabilities | ⭐⭐⭐ | A hierarchical team of LLM agents (HPTSA) autonomously exploits 14 real-world zero-day vulnerabilities, achieving 4.3× improvement over single-agent baselines that consistently fail due to poor long-horizon exploration across wide attack surfaces. A planning agent decomposes the attack and orchestrates specialized subagents, each focused on a specific vulnerability class, with results fed back to the planner for strategy adjustment. This is the key paper showing zero-day exploitation — unlike one-day CVE reproduction — requires multi-agent decomposition, not a more capable single model. | 2406.01637 |
| LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks | ⭐⭐⭐ | hackingBuddyGPT autonomously executes Linux privilege escalation attacks, with GPT-4-Turbo achieving 33–83% success on a structured benchmark — comparable to human pen-testers at 75% — while empirically measuring how context management strategies and LLM-driven reflection boost success across multiple vulnerability classes. The agent runs an observe-think-act loop, compressing past actions into summaries that keep the context window bounded without losing the thread of what has already been tried. The open benchmark and agent code are the standard reference for privilege escalation automation, and the context-compression findings transfer directly to any bot that needs to stay coherent over a long engagement. | 2310.11409 |
| RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents | ⭐⭐⭐ | Given only a target IP, RapidPen autonomously scans, selects attack vectors, and achieves shell access within 200–400 seconds at $0.30–$0.60 per run, with a 60% success rate on Hack The Box targets when augmented by a RAG knowledge base of prior successful exploits. The architecture pairs ReAct-style planning with retrieval-augmented exploit patterns and a direct command-execution feedback loop that terminates on shell confirmation. The IP-only input — no CVE hint, no human assistance — makes this the most realistic single-agent offensive benchmark published to date. | 2502.16730 |
| AEGIS: White-Box Attack Path Generation using LLMs and MCTS | ⭐⭐⭐ | AEGIS combines Monte Carlo Tree Search with an LLM reasoning engine to discover attack paths by exploring white-box exploit sequences — the MCTS backbone systematically expands the search tree based on observed execution outcomes, preventing the LLM from tunnel-visioning on a single dead-end branch. Candidate actions are generated by the LLM, executed against the target, and outcomes feed back to guide further search. For builders, this is the architecture for a hacking bot that discovers non-obvious multi-step exploit chains rather than replaying known patterns against targets that don't match prior experience. | 2601.22720 |
| CVE-Factory: Scaling Expert-Level Agentic Tasks for Code Security Vulnerability | ⭐⭐⭐ | An automated multi-agent pipeline transforms sparse CVE NVD entries into fully executable, sandboxed exploit tasks at 95% solution correctness and 96% environment fidelity verified against human experts, then uses those tasks to fine-tune Qwen3-32B from 5.3% to 35.8% task completion — surpassing Claude 4.5 Sonnet. Specialized agents handle CVE analysis, Docker environment construction, and solution verification, running end-to-end from CVE metadata to training-ready sample without human curation. For builders, this is how to generate an unlimited, continuously-updated supply of exploit training data from the live CVE feed rather than hand-curating reproductions. | 2602.03012 |
2. Agent Architecture
| Paper | ⭐ | Description | arXiv |
|---|---|---|---|
| CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery | ⭐⭐ | Self-evolving multi-agent system using shared persistent memory, asynchronous execution, and heartbeat-based interventions, achieving 3–10× improvement rates over fixed baselines. Agents evolve their own strategies without human intervention across math, algorithmic, and systems tasks. The architecture for a hacking bot that gets better at its job autonomously over time. | 2604.01658 |
| ROMA: Recursive Open Meta-Agent Framework for Long-Horizon Multi-Agent Systems | ⭐⭐⭐ | Decomposes large tasks into parallel subtask trees across multiple agents without hitting context limits. Handles long-horizon workflows by distributing work recursively rather than sequentially. Use this as the planning backbone for multi-stage hacking operations (recon → enumeration → exploitation → post-exploitation). | 2602.01848 |
| Agyn: A Multi-Agent System for Team-Based Autonomous Software Engineering | ⭐⭐ | Assigns specialized agents to coordination, research, implementation, and review roles for autonomous software engineering. Role separation prevents agents from conflating strategy with execution. Maps directly to a hacking team: planner, OSINT/recon, exploit-writer, validator. | 2602.01465 |
| MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safety | ⭐⭐ | RL game where an attacker agent and a defender agent co-evolve, stress-testing safety alignment against novel, never-seen attack patterns. The attacker continuously generates new attack strategies in response to what the defender blocks. Use this to train your bot to bypass defenses that don't exist yet. | 2602.01539 |
| StackPlanner: A Centralized Hierarchical Multi-Agent System with Task-Experience Memory Management | ⭐⭐ | Hierarchical planner that decouples coordination from execution and uses RL-driven experience reuse across sessions. Stores successful task decompositions and replays them when structurally similar tasks recur. For a hacking bot running multiple targets, this prevents re-solving already-solved planning problems. | 2601.05890 |
| D-CIPHER: Dynamic Collaborative Intelligent Multi-Agent System for Offensive Security | ⭐⭐⭐ | D-CIPHER's Planner decomposes CTF and HackTheBox problems and dynamically launches heterogeneous Executor agents specialized by vulnerability class, plus an Auto-prompter that generates high-quality initial prompts from problem context — achieving 22.0% on NYU CTF Bench, 22.5% on Cybench, and 44.0% on HackTheBox, solving 65% more MITRE ATT&CK techniques than prior work. Dynamic feedback loops allow mid-task replanning when an exploit branch fails, unlike static pipelines that exhaust budget on dead ends. D-CIPHER provides the clearest published role-separation schema for a hacking team, mapping directly to the recon → exploit → post-exploitation pipeline. | 2502.10931 |
| Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks | ⭐⭐⭐ | Incalmo autonomously red-teams enterprise networks of 22–50 hosts, succeeding on 37 of 40 emulated MHBench networks while state-of-the-art single-agent systems succeed on only 3 of 40. An LLM planner issues high-level declarative tasks executed by domain-specific task agents, with auxiliary services tracking asset state and keeping the planner's context clean across long stepping-stone pivot chains. For builders targeting enterprise environments rather than isolated CTFs, this is the only published architecture that directly solves the multi-host pivot problem at realistic network scale. | 2501.16466 |
| What Makes a Good LLM Agent for Real-world Penetration Testing? | ⭐⭐⭐ | Excalibur analyzes failure modes across 28 published LLM pentesting systems and introduces a Task Difficulty Assessment (TDA) mechanism that guides exploration-exploitation tradeoffs across four dimensions — horizon estimation, evidence confidence, context load, and historical success — achieving up to 91% task completion on CTF benchmarks and compromising 4/5 hosts on a GOAD Active Directory environment. TDA prevents agents from grinding through high-context branches that lead nowhere by dynamically reallocating effort based on real-time signal rather than uniform exhaustive search. For builders, this is the most actionable diagnostic framework for understanding why your agent stalls and where to invest in better reasoning versus better tooling. | 2602.17622 |
| Environment-Grounded Multi-Agent Workflow for Autonomous Penetration Testing | ⭐⭐⭐ | Introduces a shared graph-based engagement state that all agents write to and read from throughout a pentest session — the graph captures network topology, discovered channels, attempted exploits, and confirmed vulnerabilities as live ground truth accessible to every agent without each holding the whole engagement history in context. Achieves 100% task completion on ROS/ROS2 CTF challenges. For builders, the shared graph is the clearest published answer to how multiple hacking agents share state across a long engagement without context overflow. | 2603.24221 |
| Red-MIRROR: Agentic LLM-based Autonomous Penetration Testing with Memory-Reflection Backbone | ⭐⭐⭐ | Red-MIRROR introduces a Shared Recurrent Memory Mechanism (SRMM) that persists attack state across all agents, combined with Dual-Phase Reflection that validates payloads both before execution (blocking syntactically wrong tool calls) and after (catching hallucinated success where the agent believes it exploited a system it did not), achieving 86% success on the XBOW benchmark versus 50% for the next best system. RAG provides external exploit knowledge while SRMM maintains session continuity across a full engagement. For builders, SRMM + pre/post validation reflection is a complete coordination pattern for multi-agent web exploitation that directly addresses the two failure modes — fragmented state and hallucinated success — that most long-horizon hacking agents hit in practice. | 2603.27127 |
| APT-Agent: Automated LLM-Driven Penetration Testing Framework | ⭐⭐⭐ | APT-Agent introduces a hallucination rectification module that detects when the agent invents command syntax or hostnames that don't exist, rerouting to correction before tool execution, plus a command-specific memory architecture that persists operational context per-command-type across a multi-step attack sequence — together achieving 84% end-to-end success on Metasploitable 2 versus 19% for PentestGPT. The rectification module catches hallucinations at the tool interface rather than letting invalid commands consume tool budget and pollute the execution trace. For builders, hallucination recovery plus per-command memory directly fixes the two most common failure modes in long-horizon hacking agents. | 2605.24949 |
3. Memory & Knowledge Retention
| Paper | ⭐ | Description | arXiv |
|---|---|---|---|
| ProcMEM: Learning Reusable Procedural Memory from Experience via Non-Parametric PPO for LLM Agents | ⭐⭐ | Saves step-by-step attack procedures from past agent runs and retrieves them for reuse without retraining, trained via non-parametric PPO. Procedural memory stores how to accomplish a goal, not just that a goal was accomplished. Your bot's exploit playbook that grows automatically with each successful engagement. | 2602.01869 |
| AutoRefine: From Trajectories to Reusable Expertise for Continual LLM Agent Refinement | ⭐⭐ | Extracts two forms of reusable knowledge from execution histories: specialized subagents for recurring procedural tasks and skill patterns for static knowledge lookup, with continuous pruning and merging. Turns every hacking session into training data for the next one. Use this as the long-term learning layer sitting above your short-term memory. | 2601.22758 |
| Continuum Memory Architectures for Long-Horizon LLM Agents | ⭐⭐ | Defines the class of memory systems for agents that need persistent, temporally chained state across multiple sessions — as opposed to stateless RAG lookups. Specifies the formal requirements a memory system must satisfy for long-horizon operation. Mandatory reading before designing the persistence layer for a multi-session hacking bot. | 2601.09913 |
4. Tool Use & Execution
| Paper | ⭐ | Description | arXiv |
|---|---|---|---|
| CUA-Skill: Develop Skills for Computer Using Agent | ⭐⭐⭐ | Large-scale computer-use skill library with parameterized execution, composition graphs, dynamic retrieval, and memory-aware failure recovery for desktop and terminal applications. Skills are composable and parameterized — one "run nmap" skill works for any target, not just a hardcoded one. The reference implementation for giving your bot reliable terminal and GUI access. | 2601.21123 |
| Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoning | ⭐⭐⭐ | Agents synthesize, verify, and evolve their own executable tools at inference time instead of relying on static pre-defined tool libraries. New tools are validated before use and pruned when they fail. For a hacking bot, this means dynamically writing a new scanner or exploit script when existing tools fail on a novel target. | 2601.07641 |
| InfiAgent: An Infinite-Horizon Framework for General-Purpose Autonomous Agents | ⭐⭐⭐ | Keeps reasoning context bounded regardless of operation length by externalizing persistent state into a file-centric abstraction. The agent reads and writes state files instead of accumulating context, so it never runs out of context during a long engagement. Mandatory for any multi-day autonomous hacking operation. | 2601.03204 |
| Code Agent can be an End-to-End System Hacker | ⭐⭐⭐ | Benchmarks computer-use agent frameworks on realistic OS-level attack tasks — file exfiltration, privilege escalation, persistence, and lateral movement — using a multi-level threat model with varying attacker knowledge, showing that current CUAs execute multi-step system attacks with minimal instruction across all knowledge levels. Results break down attack success by category and chain depth, mapping which capabilities CUAs unlock versus which require custom tooling. The essential baseline for understanding what attack categories a computer-use agent handles autonomously before you invest in custom tool integrations. | 2510.06607 |
| HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities | ⭐⭐⭐ | HackWorld benchmarks computer-use agents on real web application penetration testing tasks requiring visual understanding and dynamic content interaction — capabilities that CLI-only agents cannot replicate — across realistic web targets with real deployed software. CUAs are tasked with discovering and exploiting vulnerabilities through graphical interfaces, covering multiple vulnerability classes and deployment configurations. Establishes the baseline for what a visually-aware hacking bot can autonomously exploit in web apps and provides the harness for measuring improvement over time. | 2510.12200 |
5. Benchmarks & Threat Landscape
| Paper | ⭐ | Description | arXiv |
|---|---|---|---|
| Internal Safety Collapse in Frontier Large Language Models | ⭐⭐ | Reveals that AI agents produce exploits and dangerous data as a side effect of normal professional tasks — no adversarial prompting needed. Tests 8+ frontier models across 56 cross-domain scenarios. Tells you which backbone model your bot can use without needing to spend effort on jailbreaks. | 2603.23509 |
| Learning to Inject: Automated Prompt Injection via Reinforcement Learning | ⭐⭐⭐ | Uses RL to auto-generate prompt injection attacks that transfer across multiple frontier LLM models. Attacks are discovered without white-box access and generalize to unseen models. Essential if your bot targets LLM-integrated applications — this is the attack technique it needs. | 2602.05746 |
| Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale | ⭐⭐ | Analyzes 42,447 agent skills from two major marketplaces, mapping the attack surface across prompt injection, data exfiltration, privilege escalation, and supply chain risks. Shows how real-world agent deployments fail, with prevalence data across vulnerability classes. Tells you where to aim your bot for maximum impact in the wild. | 2601.10338 |
| SoK: DARPA's AI Cyber Challenge (AIxCC): Competition Design, Architectures, and Lessons Learned | ⭐⭐⭐ | The first systematic post-competition analysis of DARPA AIxCC (2023–2025) — the largest autonomous CRS competition ever run — covering design decisions, finalist architectures, and real performance factors derived from source code, execution traces, and organizer/team interviews beyond what the scoreboard reveals. Identifies genuine technical advances (LLM-guided fuzzing, multi-agent patch generation) and open research gaps (defense bypass, multi-host pivoting) that competition scores alone cannot surface. Required reading before designing a serious autonomous vulnerability discovery system — the SoK consolidates what all finalist architectures learned and maps the open research agenda. | 2602.07666 |
| Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios | ⭐⭐ | Benchmarks frontier models on a 32-step corporate network attack scenario at varying inference-time compute budgets, finding log-linear performance scaling with compute — no plateau observed — and that successive model generations improve step-completion at fixed token budgets without the scaling curve flattening. The multi-step chained attack format captures realistic pivot-chain exploitation that single-step benchmarks miss entirely. For builders, this maps the compute budget vs. capability tradeoff for chained attack scenarios and tells you how much a model-generation upgrade is worth before you commit to your backbone choice. | 2603.11214 |
| CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities | ⭐⭐ | The first web vulnerability benchmark built on critical-severity CVEs running in production-realistic Docker sandboxes — real deployed software, real authentication, multi-component environments — showing the best available agent resolves only 13% of vulnerabilities, a stark gap from CTF performance. Evaluation exposes authentication barriers and complex environment configuration as the two consistent blockers where all current agents fail. For builders targeting web exploitation, the 13% ceiling is the honest real-world capability baseline as of mid-2025 — the number to beat before claiming production readiness. | 2503.17332 |
Field Reports
Practitioner write-ups from people who built and shipped real hacking bots. Unlike academic papers, these report on what actually broke in production, what tooling decisions worked, and what the agent loop looks like when it runs against real targets.
| Post | ⭐ | Description | Source |
|---|---|---|---|
| How the Hacking Agents Work — s1r1us (Hacktron.ai) | ⭐⭐⭐ | Frames vulnerability discovery as adversarial theorem proving using the Curry-Howard correspondence: an exploit is a proof that an unintended program state is reachable. Proposes two discovery modes — variant analysis (proving known bug classes exist in new code, ~60–70% of web vulns) and novel research (generating new propositions from deep system understanding). The key architectural insight: top-tier hackers never load full codebases into working memory; they build compressed abstract representations (control-flow skeletons) and reason over those. Maps directly to agent design: an Understanding Agent builds the abstraction, a Reasoning Agent simulates what breaks it. | s1r1us.ninja |
| Building Effective LLM Agents | AI Cyber Challenge — Xint, Theori (3rd place, DARPA AIxCC) | ⭐⭐⭐ | Four battle-tested patterns from building RoboDuck, the CRS that placed 3rd at DARPA's AI Cyber Challenge: (1) decompose tasks into sub-agents so each agent gets only its own context — the sub-agent call doubles as context compression; (2) curate custom tools instead of raw bash — e.g., read_definition, find_references backed by clang AST/joern, with hard limits to prevent context floods; (3) structure outputs with XML tags or a terminate tool and include decoy fields that force the model to reason about trigger conditions (reduces false positives); (4) adapt prompting to each model — <rule> blocks for Claude, forced tool_choice=required for o4-mini. Open-source CRS at theori-io/aixcc-afc-archive. | theori.io |
| Buttercup is Now Open Source — Trail of Bits (2nd place, DARPA AIxCC) | ⭐⭐⭐ | Buttercup CRS ran against 48 challenges on 23 real open-source repositories at DARPA AIxCC 2025 with zero human intervention, finding 28 vulnerabilities and applying 19 patches at 90%+ accuracy. The post exposes four concrete components: vulnerability discovery (OSS-Fuzz + Clusterfuzz + LLM-augmented seed generation that directs the fuzzer WHERE to look for SQL injection and path traversal patterns), contextual analysis (tree-sitter + CodeQuery for structural understanding), a 7-agent patch generation pipeline, and a published spend breakdown ($21.1K LLM, $18.5K compute, $181 per scored point) with documented failure modes — AI unpredictability requires fuzzer feedback to evaluate probabilistic patch outputs, and a strict $100/challenge LLM budget forced efficient allocation. The full system is open source at trailofbits/buttercup, and the per-point cost breakdown is the clearest published cost model for a production-grade autonomous vulnerability pipeline. | blog.trailofbits.com |
| Atlantis Infrastructure Deep Dive — Team Atlanta (1st place, DARPA AIxCC) | ⭐⭐⭐ | Atlantis CRS won DARPA AIxCC 2025 ($4M), submitting 107 PoVs and 41 passing patches across 48 challenges on 23 real open-source repositories. The post exposes the Kubernetes per-challenge scaling architecture, module breakdown with actual submission stats (Atlantis-Multilang 69.2%, Atlantis-C 16.8%, Atlantis-Java 14%), LiteLLM-based budget enforcement with full cost tables (OpenAI $20K, Anthropic $22K, Gemini $7.9K of $50K total LLM spend), and a documented pre-deadline fatal bug: the string "fuzz" in CP directory paths matched an ossfuzz filter and silently blocked all patch submissions — discovered and fixed with minutes to spare. The full budget tables, per-module performance breakdown, and root-cause failure postmortem are the clearest published record of what a winning CRS actually cost and what nearly sank it. | team-atlanta.github.io |
| Defense at AI Speed: Microsoft's MDASH Multi-Model Agentic Security System — Microsoft Autonomous Code Security Team | ⭐⭐⭐ | Microsoft's ACS team deployed MDASH against production Windows networking and authentication code, finding 16 new CVEs including 4 Critical RCE flaws in tcpip.sys and IKEv2, and scoring #1 on the public CyberGym benchmark (88.45% on 1,507 real-world vulnerability tasks). MDASH runs 100+ specialized AI agents in an ensemble of frontier and distilled models that debate and prove exploitability end-to-end — built by engineers from Team Atlanta, directly porting the AIxCC champion architecture into a production pipeline. This is the definitive "from competition to production" case study: the same agent architecture that won AIxCC is now finding real Patch Tuesday CVEs in Microsoft's production Windows codebase. | microsoft.com |
| Can AI Attack the Cloud? Lessons From Building an Autonomous Pentesting PoC — Palo Alto Networks Unit 42 | ⭐⭐⭐ | Unit 42 built "Zealot," a LangGraph-based autonomous pentesting PoC that executed a full SSRF → metadata-service credential theft → lateral movement → BigQuery exfiltration chain against a sandboxed GCP environment with no human intervention. A supervisor agent routes work to three specialist agents (Infrastructure, Application Security, Cloud Security) sharing a single attack-state object — a design forced by the failure of decentralized approaches, which produced redundant and conflicting actions when agents each maintained separate state. The concrete lesson: narrow specialist agents with isolated scope adapt to unexpected recon findings where generalist agents drift, and supervisor-as-single-source-of-truth is the minimum viable architecture for multi-cloud attack chains. | unit42.paloaltonetworks.com |
Contributing
Before adding a paper, read CLAUDE.md — it defines each tier precisely and gives an inclusion checklist. Every entry must answer YES to all four checklist items.
License
CC0 1.0 — public domain.
Contributors
vitorfhc
9 commits
Similar repos
Curated AI-agent papers for building autonomous security bots — offensive AI, multi-agent architecture, memory, and tool use.
8
9 commits
updated May 28, 2026
README
Awesome Autonomous Hacking Agents
A curated reading list of AI-agent papers for building autonomous security bots — agents that discover, analyze, and exploit vulnerabilities with minimal human input.
Contents
- Categories
- Papers
- 1. Core Offensive AI — agents that actually hack things
- 2. Agent Architecture — how to structure the system
- 3. Memory & Knowledge Retention — how the bot learns over time
- 4. Tool Use & Execution — how the bot takes actions in the world
- 5. Benchmarks & Threat Landscape — what's realistic today
- Field Reports — practitioners who shipped real systems
- Contributing
Categories
Each category answers a different question a builder would ask. Read them in order — later categories assume you've already thought through earlier ones.
| # | Category | The Question It Answers |
|---|---|---|
| 1 | Core Offensive AI | Can this paper teach me how an agent actually hacks something? |
| 2 | Agent Architecture | How do I structure agents to handle complex multi-step hacking tasks? |
| 3 | Memory & Knowledge Retention | How does my bot remember what it learned and reuse it? |
| 4 | Tool Use & Execution | How does my bot run commands, write scripts, and take real-world actions? |
| 5 | Benchmarks & Threat Landscape | What can frontier models realistically do today, and what will I face in the wild? |
| — | Field Reports | How did people who actually shipped a hacking bot build it? |
See CLAUDE.md for detailed inclusion criteria.
Papers
1. Core Offensive AI
| Paper | ⭐ | Description | arXiv |
|---|---|---|---|
| To Defend Against Cyber Attacks, We Must Teach AI Agents to Hack | ⭐⭐ | Argues that AI-agent-driven cyberattacks are inevitable and that building frontier offensive AI responsibly is essential defensive infrastructure. Surveys the landscape of what agents can already do offensively, with a blueprint for responsible capability development. Read this first — it frames the entire space. | 2602.02595 |
| A Dual-Loop Agent Framework for Automated Vulnerability Reproduction | ⭐⭐⭐ | Takes a CVE description as input and produces a working exploit using two feedback loops — an outer strategy loop and an inner code loop. The dual-loop pattern separates high-level attack planning from low-level code generation, preventing strategy drift when code fails. Directly copy this architecture for your bot's exploit-generation pipeline. | 2602.05721 |
| Capture the Flags: Family-Based Evaluation of Agentic LLMs | ⭐⭐ | Generates families of equivalent CTF challenges via code transformations to test whether an agent truly understands exploits or just memorizes patterns. Exposes the difference between genuine exploit understanding and surface-level pattern matching. Use this benchmark architecture to evaluate your own bot. | 2602.05523 |
| TxRay: Agentic Postmortem of Live Blockchain Attacks | ⭐⭐⭐ | Reconstructs exploit lifecycles from limited on-chain evidence and generates runnable PoC reproductions. The agent chains together sparse signals into a coherent attack narrative, then translates that narrative into executable code. A concrete end-to-end pipeline from intelligence to working exploit. | 2602.01317 |
| Identifying Adversary Tactics and Techniques in Malware Binaries with an LLM Agent | ⭐⭐⭐ | Uses an LLM agent to map MITRE ATT&CK techniques in stripped malware binaries through incremental context retrieval. Solves the problem of limited context windows on large binaries by feeding evidence incrementally. Use this when your bot needs to reason about existing malware or analyze unfamiliar binaries. | 2602.06325 |
| VirtualCrime: Evaluating Criminal Potential of LLMs via Sandbox Simulation | ⭐⭐ | Evaluates what LLMs can execute in sandboxed offensive scenarios without specialized prompting. Gives empirical ground truth on which models succeed at real-world attack tasks and which prompting strategies are effective. Read before choosing a backbone model for your bot. | 2601.13981 |
| AgenticSCR: An Autonomous Agentic Secure Code Review for Immature Vulnerabilities Detection | ⭐⭐⭐ | Autonomous vulnerability detection via tool invocation and security-focused semantic memory for pre-commit code review. The memory architecture stores "what to look for" as indexed security patterns rather than raw rules. Reuse the memory design for your bot's vulnerability-discovery phase. | 2601.19138 |
| Multimodal Multi-Agent Ransomware Analysis Using AutoGen | ⭐⭐⭐ | A working AutoGen-based multi-agent system for malware analysis that divides static analysis, dynamic analysis, and reporting across specialized agents. Shows how to wire AutoGen agents together for a security-specific task with concrete role definitions. A practical reference implementation to fork. | 2601.20346 |
| Sifting the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering | ⭐⭐ | Head-to-head benchmark of Aider, OpenHands, and SWE-agent on vulnerability triage tasks across multiple backbone models. Measures how agent design — not just model capability — affects security task performance. Read this before picking which agent framework to build on. | 2601.22952 |
| LLM Agents can Autonomously Exploit One-day Vulnerabilities | ⭐⭐⭐ | A single GPT-4 agent autonomously exploits 87% of 15 critical-severity real-world one-day CVEs, while GPT-3.5, every tested open-source model, and commercial scanners (ZAP, Metasploit) all score 0%. The agent runs a tool-calling loop against live vulnerable systems using only the CVE description and public documentation as input. This is the foundational capability baseline — use it to calibrate model selection and understand the ceiling a single-model agent can reach before adding multi-agent complexity. | 2404.08144 |
| Teams of LLM Agents can Exploit Zero-Day Vulnerabilities | ⭐⭐⭐ | A hierarchical team of LLM agents (HPTSA) autonomously exploits 14 real-world zero-day vulnerabilities, achieving 4.3× improvement over single-agent baselines that consistently fail due to poor long-horizon exploration across wide attack surfaces. A planning agent decomposes the attack and orchestrates specialized subagents, each focused on a specific vulnerability class, with results fed back to the planner for strategy adjustment. This is the key paper showing zero-day exploitation — unlike one-day CVE reproduction — requires multi-agent decomposition, not a more capable single model. | 2406.01637 |
| LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks | ⭐⭐⭐ | hackingBuddyGPT autonomously executes Linux privilege escalation attacks, with GPT-4-Turbo achieving 33–83% success on a structured benchmark — comparable to human pen-testers at 75% — while empirically measuring how context management strategies and LLM-driven reflection boost success across multiple vulnerability classes. The agent runs an observe-think-act loop, compressing past actions into summaries that keep the context window bounded without losing the thread of what has already been tried. The open benchmark and agent code are the standard reference for privilege escalation automation, and the context-compression findings transfer directly to any bot that needs to stay coherent over a long engagement. | 2310.11409 |
| RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents | ⭐⭐⭐ | Given only a target IP, RapidPen autonomously scans, selects attack vectors, and achieves shell access within 200–400 seconds at $0.30–$0.60 per run, with a 60% success rate on Hack The Box targets when augmented by a RAG knowledge base of prior successful exploits. The architecture pairs ReAct-style planning with retrieval-augmented exploit patterns and a direct command-execution feedback loop that terminates on shell confirmation. The IP-only input — no CVE hint, no human assistance — makes this the most realistic single-agent offensive benchmark published to date. | 2502.16730 |
| AEGIS: White-Box Attack Path Generation using LLMs and MCTS | ⭐⭐⭐ | AEGIS combines Monte Carlo Tree Search with an LLM reasoning engine to discover attack paths by exploring white-box exploit sequences — the MCTS backbone systematically expands the search tree based on observed execution outcomes, preventing the LLM from tunnel-visioning on a single dead-end branch. Candidate actions are generated by the LLM, executed against the target, and outcomes feed back to guide further search. For builders, this is the architecture for a hacking bot that discovers non-obvious multi-step exploit chains rather than replaying known patterns against targets that don't match prior experience. | 2601.22720 |
| CVE-Factory: Scaling Expert-Level Agentic Tasks for Code Security Vulnerability | ⭐⭐⭐ | An automated multi-agent pipeline transforms sparse CVE NVD entries into fully executable, sandboxed exploit tasks at 95% solution correctness and 96% environment fidelity verified against human experts, then uses those tasks to fine-tune Qwen3-32B from 5.3% to 35.8% task completion — surpassing Claude 4.5 Sonnet. Specialized agents handle CVE analysis, Docker environment construction, and solution verification, running end-to-end from CVE metadata to training-ready sample without human curation. For builders, this is how to generate an unlimited, continuously-updated supply of exploit training data from the live CVE feed rather than hand-curating reproductions. | 2602.03012 |
2. Agent Architecture
| Paper | ⭐ | Description | arXiv |
|---|---|---|---|
| CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery | ⭐⭐ | Self-evolving multi-agent system using shared persistent memory, asynchronous execution, and heartbeat-based interventions, achieving 3–10× improvement rates over fixed baselines. Agents evolve their own strategies without human intervention across math, algorithmic, and systems tasks. The architecture for a hacking bot that gets better at its job autonomously over time. | 2604.01658 |
| ROMA: Recursive Open Meta-Agent Framework for Long-Horizon Multi-Agent Systems | ⭐⭐⭐ | Decomposes large tasks into parallel subtask trees across multiple agents without hitting context limits. Handles long-horizon workflows by distributing work recursively rather than sequentially. Use this as the planning backbone for multi-stage hacking operations (recon → enumeration → exploitation → post-exploitation). | 2602.01848 |
| Agyn: A Multi-Agent System for Team-Based Autonomous Software Engineering | ⭐⭐ | Assigns specialized agents to coordination, research, implementation, and review roles for autonomous software engineering. Role separation prevents agents from conflating strategy with execution. Maps directly to a hacking team: planner, OSINT/recon, exploit-writer, validator. | 2602.01465 |
| MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safety | ⭐⭐ | RL game where an attacker agent and a defender agent co-evolve, stress-testing safety alignment against novel, never-seen attack patterns. The attacker continuously generates new attack strategies in response to what the defender blocks. Use this to train your bot to bypass defenses that don't exist yet. | 2602.01539 |
| StackPlanner: A Centralized Hierarchical Multi-Agent System with Task-Experience Memory Management | ⭐⭐ | Hierarchical planner that decouples coordination from execution and uses RL-driven experience reuse across sessions. Stores successful task decompositions and replays them when structurally similar tasks recur. For a hacking bot running multiple targets, this prevents re-solving already-solved planning problems. | 2601.05890 |
| D-CIPHER: Dynamic Collaborative Intelligent Multi-Agent System for Offensive Security | ⭐⭐⭐ | D-CIPHER's Planner decomposes CTF and HackTheBox problems and dynamically launches heterogeneous Executor agents specialized by vulnerability class, plus an Auto-prompter that generates high-quality initial prompts from problem context — achieving 22.0% on NYU CTF Bench, 22.5% on Cybench, and 44.0% on HackTheBox, solving 65% more MITRE ATT&CK techniques than prior work. Dynamic feedback loops allow mid-task replanning when an exploit branch fails, unlike static pipelines that exhaust budget on dead ends. D-CIPHER provides the clearest published role-separation schema for a hacking team, mapping directly to the recon → exploit → post-exploitation pipeline. | 2502.10931 |
| Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks | ⭐⭐⭐ | Incalmo autonomously red-teams enterprise networks of 22–50 hosts, succeeding on 37 of 40 emulated MHBench networks while state-of-the-art single-agent systems succeed on only 3 of 40. An LLM planner issues high-level declarative tasks executed by domain-specific task agents, with auxiliary services tracking asset state and keeping the planner's context clean across long stepping-stone pivot chains. For builders targeting enterprise environments rather than isolated CTFs, this is the only published architecture that directly solves the multi-host pivot problem at realistic network scale. | 2501.16466 |
| What Makes a Good LLM Agent for Real-world Penetration Testing? | ⭐⭐⭐ | Excalibur analyzes failure modes across 28 published LLM pentesting systems and introduces a Task Difficulty Assessment (TDA) mechanism that guides exploration-exploitation tradeoffs across four dimensions — horizon estimation, evidence confidence, context load, and historical success — achieving up to 91% task completion on CTF benchmarks and compromising 4/5 hosts on a GOAD Active Directory environment. TDA prevents agents from grinding through high-context branches that lead nowhere by dynamically reallocating effort based on real-time signal rather than uniform exhaustive search. For builders, this is the most actionable diagnostic framework for understanding why your agent stalls and where to invest in better reasoning versus better tooling. | 2602.17622 |
| Environment-Grounded Multi-Agent Workflow for Autonomous Penetration Testing | ⭐⭐⭐ | Introduces a shared graph-based engagement state that all agents write to and read from throughout a pentest session — the graph captures network topology, discovered channels, attempted exploits, and confirmed vulnerabilities as live ground truth accessible to every agent without each holding the whole engagement history in context. Achieves 100% task completion on ROS/ROS2 CTF challenges. For builders, the shared graph is the clearest published answer to how multiple hacking agents share state across a long engagement without context overflow. | 2603.24221 |
| Red-MIRROR: Agentic LLM-based Autonomous Penetration Testing with Memory-Reflection Backbone | ⭐⭐⭐ | Red-MIRROR introduces a Shared Recurrent Memory Mechanism (SRMM) that persists attack state across all agents, combined with Dual-Phase Reflection that validates payloads both before execution (blocking syntactically wrong tool calls) and after (catching hallucinated success where the agent believes it exploited a system it did not), achieving 86% success on the XBOW benchmark versus 50% for the next best system. RAG provides external exploit knowledge while SRMM maintains session continuity across a full engagement. For builders, SRMM + pre/post validation reflection is a complete coordination pattern for multi-agent web exploitation that directly addresses the two failure modes — fragmented state and hallucinated success — that most long-horizon hacking agents hit in practice. | 2603.27127 |
| APT-Agent: Automated LLM-Driven Penetration Testing Framework | ⭐⭐⭐ | APT-Agent introduces a hallucination rectification module that detects when the agent invents command syntax or hostnames that don't exist, rerouting to correction before tool execution, plus a command-specific memory architecture that persists operational context per-command-type across a multi-step attack sequence — together achieving 84% end-to-end success on Metasploitable 2 versus 19% for PentestGPT. The rectification module catches hallucinations at the tool interface rather than letting invalid commands consume tool budget and pollute the execution trace. For builders, hallucination recovery plus per-command memory directly fixes the two most common failure modes in long-horizon hacking agents. | 2605.24949 |
3. Memory & Knowledge Retention
| Paper | ⭐ | Description | arXiv |
|---|---|---|---|
| ProcMEM: Learning Reusable Procedural Memory from Experience via Non-Parametric PPO for LLM Agents | ⭐⭐ | Saves step-by-step attack procedures from past agent runs and retrieves them for reuse without retraining, trained via non-parametric PPO. Procedural memory stores how to accomplish a goal, not just that a goal was accomplished. Your bot's exploit playbook that grows automatically with each successful engagement. | 2602.01869 |
| AutoRefine: From Trajectories to Reusable Expertise for Continual LLM Agent Refinement | ⭐⭐ | Extracts two forms of reusable knowledge from execution histories: specialized subagents for recurring procedural tasks and skill patterns for static knowledge lookup, with continuous pruning and merging. Turns every hacking session into training data for the next one. Use this as the long-term learning layer sitting above your short-term memory. | 2601.22758 |
| Continuum Memory Architectures for Long-Horizon LLM Agents | ⭐⭐ | Defines the class of memory systems for agents that need persistent, temporally chained state across multiple sessions — as opposed to stateless RAG lookups. Specifies the formal requirements a memory system must satisfy for long-horizon operation. Mandatory reading before designing the persistence layer for a multi-session hacking bot. | 2601.09913 |
4. Tool Use & Execution
| Paper | ⭐ | Description | arXiv |
|---|---|---|---|
| CUA-Skill: Develop Skills for Computer Using Agent | ⭐⭐⭐ | Large-scale computer-use skill library with parameterized execution, composition graphs, dynamic retrieval, and memory-aware failure recovery for desktop and terminal applications. Skills are composable and parameterized — one "run nmap" skill works for any target, not just a hardcoded one. The reference implementation for giving your bot reliable terminal and GUI access. | 2601.21123 |
| Beyond Static Tools: Test-Time Tool Evolution for Scientific Reasoning | ⭐⭐⭐ | Agents synthesize, verify, and evolve their own executable tools at inference time instead of relying on static pre-defined tool libraries. New tools are validated before use and pruned when they fail. For a hacking bot, this means dynamically writing a new scanner or exploit script when existing tools fail on a novel target. | 2601.07641 |
| InfiAgent: An Infinite-Horizon Framework for General-Purpose Autonomous Agents | ⭐⭐⭐ | Keeps reasoning context bounded regardless of operation length by externalizing persistent state into a file-centric abstraction. The agent reads and writes state files instead of accumulating context, so it never runs out of context during a long engagement. Mandatory for any multi-day autonomous hacking operation. | 2601.03204 |
| Code Agent can be an End-to-End System Hacker | ⭐⭐⭐ | Benchmarks computer-use agent frameworks on realistic OS-level attack tasks — file exfiltration, privilege escalation, persistence, and lateral movement — using a multi-level threat model with varying attacker knowledge, showing that current CUAs execute multi-step system attacks with minimal instruction across all knowledge levels. Results break down attack success by category and chain depth, mapping which capabilities CUAs unlock versus which require custom tooling. The essential baseline for understanding what attack categories a computer-use agent handles autonomously before you invest in custom tool integrations. | 2510.06607 |
| HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities | ⭐⭐⭐ | HackWorld benchmarks computer-use agents on real web application penetration testing tasks requiring visual understanding and dynamic content interaction — capabilities that CLI-only agents cannot replicate — across realistic web targets with real deployed software. CUAs are tasked with discovering and exploiting vulnerabilities through graphical interfaces, covering multiple vulnerability classes and deployment configurations. Establishes the baseline for what a visually-aware hacking bot can autonomously exploit in web apps and provides the harness for measuring improvement over time. | 2510.12200 |
5. Benchmarks & Threat Landscape
| Paper | ⭐ | Description | arXiv |
|---|---|---|---|
| Internal Safety Collapse in Frontier Large Language Models | ⭐⭐ | Reveals that AI agents produce exploits and dangerous data as a side effect of normal professional tasks — no adversarial prompting needed. Tests 8+ frontier models across 56 cross-domain scenarios. Tells you which backbone model your bot can use without needing to spend effort on jailbreaks. | 2603.23509 |
| Learning to Inject: Automated Prompt Injection via Reinforcement Learning | ⭐⭐⭐ | Uses RL to auto-generate prompt injection attacks that transfer across multiple frontier LLM models. Attacks are discovered without white-box access and generalize to unseen models. Essential if your bot targets LLM-integrated applications — this is the attack technique it needs. | 2602.05746 |
| Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale | ⭐⭐ | Analyzes 42,447 agent skills from two major marketplaces, mapping the attack surface across prompt injection, data exfiltration, privilege escalation, and supply chain risks. Shows how real-world agent deployments fail, with prevalence data across vulnerability classes. Tells you where to aim your bot for maximum impact in the wild. | 2601.10338 |
| SoK: DARPA's AI Cyber Challenge (AIxCC): Competition Design, Architectures, and Lessons Learned | ⭐⭐⭐ | The first systematic post-competition analysis of DARPA AIxCC (2023–2025) — the largest autonomous CRS competition ever run — covering design decisions, finalist architectures, and real performance factors derived from source code, execution traces, and organizer/team interviews beyond what the scoreboard reveals. Identifies genuine technical advances (LLM-guided fuzzing, multi-agent patch generation) and open research gaps (defense bypass, multi-host pivoting) that competition scores alone cannot surface. Required reading before designing a serious autonomous vulnerability discovery system — the SoK consolidates what all finalist architectures learned and maps the open research agenda. | 2602.07666 |
| Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios | ⭐⭐ | Benchmarks frontier models on a 32-step corporate network attack scenario at varying inference-time compute budgets, finding log-linear performance scaling with compute — no plateau observed — and that successive model generations improve step-completion at fixed token budgets without the scaling curve flattening. The multi-step chained attack format captures realistic pivot-chain exploitation that single-step benchmarks miss entirely. For builders, this maps the compute budget vs. capability tradeoff for chained attack scenarios and tells you how much a model-generation upgrade is worth before you commit to your backbone choice. | 2603.11214 |
| CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities | ⭐⭐ | The first web vulnerability benchmark built on critical-severity CVEs running in production-realistic Docker sandboxes — real deployed software, real authentication, multi-component environments — showing the best available agent resolves only 13% of vulnerabilities, a stark gap from CTF performance. Evaluation exposes authentication barriers and complex environment configuration as the two consistent blockers where all current agents fail. For builders targeting web exploitation, the 13% ceiling is the honest real-world capability baseline as of mid-2025 — the number to beat before claiming production readiness. | 2503.17332 |
Field Reports
Practitioner write-ups from people who built and shipped real hacking bots. Unlike academic papers, these report on what actually broke in production, what tooling decisions worked, and what the agent loop looks like when it runs against real targets.
| Post | ⭐ | Description | Source |
|---|---|---|---|
| How the Hacking Agents Work — s1r1us (Hacktron.ai) | ⭐⭐⭐ | Frames vulnerability discovery as adversarial theorem proving using the Curry-Howard correspondence: an exploit is a proof that an unintended program state is reachable. Proposes two discovery modes — variant analysis (proving known bug classes exist in new code, ~60–70% of web vulns) and novel research (generating new propositions from deep system understanding). The key architectural insight: top-tier hackers never load full codebases into working memory; they build compressed abstract representations (control-flow skeletons) and reason over those. Maps directly to agent design: an Understanding Agent builds the abstraction, a Reasoning Agent simulates what breaks it. | s1r1us.ninja |
| Building Effective LLM Agents | AI Cyber Challenge — Xint, Theori (3rd place, DARPA AIxCC) | ⭐⭐⭐ | Four battle-tested patterns from building RoboDuck, the CRS that placed 3rd at DARPA's AI Cyber Challenge: (1) decompose tasks into sub-agents so each agent gets only its own context — the sub-agent call doubles as context compression; (2) curate custom tools instead of raw bash — e.g., read_definition, find_references backed by clang AST/joern, with hard limits to prevent context floods; (3) structure outputs with XML tags or a terminate tool and include decoy fields that force the model to reason about trigger conditions (reduces false positives); (4) adapt prompting to each model — <rule> blocks for Claude, forced tool_choice=required for o4-mini. Open-source CRS at theori-io/aixcc-afc-archive. | theori.io |
| Buttercup is Now Open Source — Trail of Bits (2nd place, DARPA AIxCC) | ⭐⭐⭐ | Buttercup CRS ran against 48 challenges on 23 real open-source repositories at DARPA AIxCC 2025 with zero human intervention, finding 28 vulnerabilities and applying 19 patches at 90%+ accuracy. The post exposes four concrete components: vulnerability discovery (OSS-Fuzz + Clusterfuzz + LLM-augmented seed generation that directs the fuzzer WHERE to look for SQL injection and path traversal patterns), contextual analysis (tree-sitter + CodeQuery for structural understanding), a 7-agent patch generation pipeline, and a published spend breakdown ($21.1K LLM, $18.5K compute, $181 per scored point) with documented failure modes — AI unpredictability requires fuzzer feedback to evaluate probabilistic patch outputs, and a strict $100/challenge LLM budget forced efficient allocation. The full system is open source at trailofbits/buttercup, and the per-point cost breakdown is the clearest published cost model for a production-grade autonomous vulnerability pipeline. | blog.trailofbits.com |
| Atlantis Infrastructure Deep Dive — Team Atlanta (1st place, DARPA AIxCC) | ⭐⭐⭐ | Atlantis CRS won DARPA AIxCC 2025 ($4M), submitting 107 PoVs and 41 passing patches across 48 challenges on 23 real open-source repositories. The post exposes the Kubernetes per-challenge scaling architecture, module breakdown with actual submission stats (Atlantis-Multilang 69.2%, Atlantis-C 16.8%, Atlantis-Java 14%), LiteLLM-based budget enforcement with full cost tables (OpenAI $20K, Anthropic $22K, Gemini $7.9K of $50K total LLM spend), and a documented pre-deadline fatal bug: the string "fuzz" in CP directory paths matched an ossfuzz filter and silently blocked all patch submissions — discovered and fixed with minutes to spare. The full budget tables, per-module performance breakdown, and root-cause failure postmortem are the clearest published record of what a winning CRS actually cost and what nearly sank it. | team-atlanta.github.io |
| Defense at AI Speed: Microsoft's MDASH Multi-Model Agentic Security System — Microsoft Autonomous Code Security Team | ⭐⭐⭐ | Microsoft's ACS team deployed MDASH against production Windows networking and authentication code, finding 16 new CVEs including 4 Critical RCE flaws in tcpip.sys and IKEv2, and scoring #1 on the public CyberGym benchmark (88.45% on 1,507 real-world vulnerability tasks). MDASH runs 100+ specialized AI agents in an ensemble of frontier and distilled models that debate and prove exploitability end-to-end — built by engineers from Team Atlanta, directly porting the AIxCC champion architecture into a production pipeline. This is the definitive "from competition to production" case study: the same agent architecture that won AIxCC is now finding real Patch Tuesday CVEs in Microsoft's production Windows codebase. | microsoft.com |
| Can AI Attack the Cloud? Lessons From Building an Autonomous Pentesting PoC — Palo Alto Networks Unit 42 | ⭐⭐⭐ | Unit 42 built "Zealot," a LangGraph-based autonomous pentesting PoC that executed a full SSRF → metadata-service credential theft → lateral movement → BigQuery exfiltration chain against a sandboxed GCP environment with no human intervention. A supervisor agent routes work to three specialist agents (Infrastructure, Application Security, Cloud Security) sharing a single attack-state object — a design forced by the failure of decentralized approaches, which produced redundant and conflicting actions when agents each maintained separate state. The concrete lesson: narrow specialist agents with isolated scope adapt to unexpected recon findings where generalist agents drift, and supervisor-as-single-source-of-truth is the minimum viable architecture for multi-cloud attack chains. | unit42.paloaltonetworks.com |
Contributing
Before adding a paper, read CLAUDE.md — it defines each tier precisely and gives an inclusion checklist. Every entry must answer YES to all four checklist items.
License
CC0 1.0 — public domain.
Similar repos
Contributors
vitorfhc
9 commits