---

中文版

本 README 采用 全量中文 + 全量 English 的双语格式。上半部分是完整中文版，下半部分是完整英文版；两种语言覆盖相同的项目定位、截图、快速开始、报告工作流、制品清单、验证门禁、开发命令、路线图、安全说明和许可证。

如果只需要中文交付或内部转发，可以使用 README.zh-CN.md。主 README 保持中英完整镜像，避免两个语言版本事实漂移。

项目定位

FORGEDAN 基于论文 FORGEDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models，但当前项目主线已经从单纯的越狱算法演示扩展为 LLM 安全评估报告交付框架。

本仓库仍保留进化式越狱攻击、模型适配器、WebScan、REST API 和 Vue dashboard。当前更重要的目标是生成可审计、可复现、可交接的报告包，包括 YAML 套件、确定性扫描器与评分器、证据矩阵、风险登记、覆盖率摘要、JSON Schema 合约、QA 回执、脱敏发布包，以及复制或分享后仍可重新校验的 ZIP 归档。

这不是商业化安全平台，而是报告交付项目。核心价值是报告证据质量：输入可追溯、输出可复现、制品可校验、交付可签收、归档可复核。

交付速览

使用截图

下面截图来自 examples/ready-for-handoff-suite.yml 生成并提交到仓库的样例报告包。完整样例见 docs/sample-report-pack/ready-for-handoff。

报告包总览

QA 交接回执

归档校验

快速开始

前置要求

安装

git clone https://github.com/Coff0xc/LLM-Security-Assessment-Framework.git
cd LLM-Security-Assessment-Framework

# 后端最小安装
pip install -e .

# Web dashboard 和 WebScan 依赖
pip install -e ".[web]"

# 全量 provider、web、monitoring 和 dev 依赖
pip install -e ".[all]"

# 前端
cd frontend
npm install

生成可交付报告包

这是当前最推荐的 smoke path：先做无模型预检，再生成报告包，随后校验制品、生成 QA 回执、打包 ZIP，并在交付后重新校验归档。

python -m forgedan.cli suite preflight examples/ready-for-handoff-suite.yml --strict --output reports/preflight-ready
python -m forgedan.cli suite run examples/ready-for-handoff-suite.yml --output reports/suite-ready
python -m forgedan.cli suite validate-report reports/suite-ready/suite-result.json
python -m forgedan.cli suite verify-bundle reports/suite-ready/suite-manifest.json
python -m forgedan.cli suite qa-report reports/suite-ready/suite-manifest.json --output reports/suite-ready/qa --strict-handoff
python -m forgedan.cli suite archive reports/suite-ready/suite-manifest.json --output reports/suite-ready/handoff.zip
python -m forgedan.cli suite verify-archive reports/suite-ready/handoff.zip

安装 console script 后，也可以使用 forgedan suite ... 形式运行。

运行零配置攻击 Demo

python -m forgedan.cli run --quick -g "test prompt" -m mock:test

运行 Web Dashboard

python -m forgedan.cli web
cd frontend
npm run dev

后端默认在 :5000，前端默认在 :5173。

报告工作流

报告包制品

JSON Schema 与验证

报告制品 schema 位于 schemas/，用于机器校验、交付验收和 CI 回归保护。

validate-report 不只检查 JSON Schema，还会复算 Source Inventory、usage cost、risk register totals、coverage totals、comparison regression counts、QA receipt readiness、manifest binding 和 Markdown/HTML sidecar 摘要，尽量避免手工修改后出现机器数据与人工报告不一致。

常用验证命令：

python -m forgedan.cli suite validate-report reports/suite-ready/suite-result.json
python -m forgedan.cli suite verify-bundle reports/suite-ready/suite-manifest.json
python -m forgedan.cli suite qa-report reports/suite-ready/suite-manifest.json --output reports/suite-ready/qa --strict-handoff
python -m forgedan.cli suite verify-archive reports/suite-ready/handoff.zip

核心能力

常用 CLI

攻击方法

模型适配器

WebScan 模式

架构概览

forgedan/
├── suite.py              # 报告套件运行器、制品、验证器、QA、归档复核
├── scanners.py           # 确定性 prompt、response、tool、model-artifact 扫描器
├── scorers.py            # 可复用确定性 suite scorer
├── finding_taxonomy.py   # finding IDs、优先级、OWASP LLM 映射
├── attacks/              # 攻击算法与注册表
├── adapters/             # 托管、本地、中文、vision、vLLM、HuggingFace、mock adapters
├── api/                  # Flask Blueprint REST API
├── webscan/              # crawler、web scanner、LLM interaction tester
├── engine.py             # 进化算法引擎
├── mutator.py            # mutation strategies 和 MAB 选择
├── fitness.py            # 语义适应度
└── judge.py              # 双 judge 机制

schemas/                  # 报告制品 JSON Schema 合约
examples/                 # 可运行 suite 和 fixture
docs/                     # 同类项目扫描、lint roadmap、样例报告包
tests/                    # pytest coverage
frontend/                 # Vue 3 SPA dashboard

forgedan/suite.py 是报告交付主入口，负责 suite 运行、报告制品写入、schema/manifest 校验、QA 回执和归档复核。schemas/ 约束报告制品，examples/ 保存可运行 fixture，docs/ 保存样例报告包和项目说明，frontend/ 保存 Vue 3 仪表盘。

API 端点

后端使用 Flask Blueprint 暴露 REST API，适合自动化脚本、演示环境和内部评估流程接入。

POST   /api/attacks/run
GET    /api/attacks/{id}/status
GET    /api/attacks/{id}/result
POST   /api/attacks/{id}/stop

GET    /api/models/providers
POST   /api/models/test

POST   /api/webscan/crawl
POST   /api/webscan/scan
POST   /api/webscan/llm-test

POST   /api/reports/generate
GET    /api/reports/{id}
GET    /api/reports/{id}/download

GET    /api/datasets
POST   /api/datasets/upload

GET    /api/health
GET    /api/metrics

文档导航

开发与验证

pip install -e ".[dev]"

# 本地全量 pytest
python -m pytest -q -W error::DeprecationWarning -p no:cacheprovider --basetemp .tmp-test

# CI 报告包门禁
python -m forgedan.cli suite run examples/ready-for-handoff-suite.yml --output reports/suite-ready
python -m forgedan.cli suite verify-bundle reports/suite-ready/suite-manifest.json
python -m forgedan.cli suite qa-report reports/suite-ready/suite-manifest.json --output reports/suite-ready/qa --strict-handoff
python -m forgedan.cli suite verify-archive reports/suite-ready/handoff.zip

# flake8 选定门禁
python -m flake8 forgedan/ --select=E9,F63,F7,F82,E722,F401,F841 --show-source --statistics

# 格式化门禁
python -m black --check forgedan tests

# 前端构建
cd frontend
npm install
npm run build

当前路线图

• 进化算法引擎与 mutation strategy
• 6 种攻击方法：FORGEDAN、AutoDAN、PAIR、GCG、Crescendo、TAP
• 托管、中文、本地、vLLM、HuggingFace、vision 和 mock adapter
• YAML suite runner、导入 case、自定义 scorer、响应缓存、Source Inventory 和 policy gates
• Prompt Injection、越狱框架、系统提示泄漏、敏感信息/PII、Agent/MCP/工具策略风险和模型制品扫描
• Markdown/HTML 报告、脱敏发布包、证据矩阵、case matrix、风险登记、覆盖率摘要、发布说明和 bundle index
• JSON Schema 合约、语义校验、manifest verification、QA receipt 和 archive verification
• 已提交 ready-for-handoff 样例报告包和截图
• CI 覆盖 tests、报告包校验、严格 QA 交接、归档校验、selected flake8、Black 和 frontend build
• 增加更多真实 Agent/MCP manifest fixture，校准默认 trust-score policy
• 仅在能提升报告证据质量时加入 HarmBench/JailbreakBench 示例
• 当报告范围需要时扩展更深的 model serialization 分析

GitHub About 建议

仓库侧栏描述建议：

面向报告交付的 LLM 安全评估框架，用于生成可复现红队套件、证据包、QA 回执、Schema 合约和可校验归档。

建议 Topics：

llm-security, ai-red-team, prompt-injection, jailbreak, owasp-llm, mcp-security, agent-security, security-reporting, risk-register, audit-evidence, json-schema, pytest, python

引用

如果在研究中使用 FORGEDAN，请引用：

@article{cheng2025forgedan,
  title={FORGEDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models},
  author={Cheng, Siyang and Liu, Gaotian and Mei, Rui and Wang, Yilin and Zhang, Kejia and Wei, Kaishuo and Yu, Yuqi and Wen, Weiping and Wu, Xiaojie and Liu, Junhua},
  journal={arXiv preprint arXiv:2511.13548},
  year={2025}
}

贡献

欢迎提交改进。修改报告行为时，请同步维护报告制品、Schema、测试和文档。通用贡献流程见 CONTRIBUTING.md。

安全说明

本项目仅用于授权安全测试、研究复现和报告交付。原始 prompt、response、trace、cache 和 evidence 可能包含敏感数据，应按评估范围和交接规则管理。

许可证

本项目使用 MIT License，详见 LICENSE。

---

English Version

This README uses a complete Chinese + complete English bilingual structure. The first half is the full Chinese version; the second half is the full English version. Both versions cover the same positioning, screenshots, quick start, report workflow, artifact list, validation gates, development commands, roadmap, security notes, and license.

Use README.zh-CN.md for Chinese-only handoff or internal sharing. The main README keeps the Chinese and English versions mirrored so project facts do not drift.

Project Positioning

FORGEDAN is based on the paper FORGEDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models, but the current project direction has expanded from a jailbreak algorithm demo into a report-delivery framework for LLM security assessments.

The repository still contains evolutionary jailbreak attacks, model adapters, WebScan utilities, a REST API, and a Vue dashboard. Its current priority is to produce auditable, reproducible, handoff-ready report packs: YAML suites, deterministic scanners and scorers, evidence matrices, risk registers, coverage summaries, JSON Schema contracts, QA receipts, redacted publication packs, and ZIP archives that can be verified after copying or sharing.

This is a report-delivery project, not a commercial security platform. Its core value is report evidence quality: traceable inputs, reproducible outputs, verifiable artifacts, signed-off handoff, and re-checkable archives.

Delivery Snapshot

Screenshots

The screenshots below come from the checked-in sample generated by examples/ready-for-handoff-suite.yml. The rendered sample is available at docs/sample-report-pack/ready-for-handoff.

Report Pack Overview

QA Receipt

Archive Verification

Quick Start

Prerequisites

Install

git clone https://github.com/Coff0xc/LLM-Security-Assessment-Framework.git
cd LLM-Security-Assessment-Framework

# Minimal backend install
pip install -e .

# Web dashboard and WebScan dependencies
pip install -e ".[web]"

# Full provider, web, monitoring, and dev extras
pip install -e ".[all]"

# Frontend
cd frontend
npm install

Generate a Ready-for-Handoff Report Pack

This is the recommended smoke path: run a no-model preflight, generate the report pack, validate artifacts, write the QA receipt, create a ZIP, and verify the archive after handoff.

python -m forgedan.cli suite preflight examples/ready-for-handoff-suite.yml --strict --output reports/preflight-ready
python -m forgedan.cli suite run examples/ready-for-handoff-suite.yml --output reports/suite-ready
python -m forgedan.cli suite validate-report reports/suite-ready/suite-result.json
python -m forgedan.cli suite verify-bundle reports/suite-ready/suite-manifest.json
python -m forgedan.cli suite qa-report reports/suite-ready/suite-manifest.json --output reports/suite-ready/qa --strict-handoff
python -m forgedan.cli suite archive reports/suite-ready/suite-manifest.json --output reports/suite-ready/handoff.zip
python -m forgedan.cli suite verify-archive reports/suite-ready/handoff.zip

After installing the console script, the same commands can be run as forgedan suite ....

Run a Zero-Config Attack Demo

python -m forgedan.cli run --quick -g "test prompt" -m mock:test

Run the Web Dashboard

python -m forgedan.cli web
cd frontend
npm run dev

The backend defaults to :5000; the frontend defaults to :5173.

Report Workflow

Report Pack Artifacts

JSON Schema and Verification

Report artifact schemas live in schemas/ and support machine validation, handoff acceptance, and CI regression protection.

validate-report checks more than JSON Schema. It recalculates source inventory, usage cost, risk-register totals, coverage totals, comparison regression counts, QA receipt readiness, manifest binding, and Markdown/HTML sidecar summaries so edited reports cannot silently diverge from machine data.

Common verification commands:

python -m forgedan.cli suite validate-report reports/suite-ready/suite-result.json
python -m forgedan.cli suite verify-bundle reports/suite-ready/suite-manifest.json
python -m forgedan.cli suite qa-report reports/suite-ready/suite-manifest.json --output reports/suite-ready/qa --strict-handoff
python -m forgedan.cli suite verify-archive reports/suite-ready/handoff.zip

Key Capabilities

CLI Reference

Attack Methods

Model Adapters

WebScan Modes

Architecture

forgedan/
├── suite.py              # Suite runner, artifacts, validators, QA, archive verifier
├── scanners.py           # Deterministic prompt, response, tool, model-artifact scanners
├── scorers.py            # Reusable deterministic suite scorers
├── finding_taxonomy.py   # Finding IDs, priorities, OWASP LLM mappings
├── attacks/              # Attack algorithms and registry
├── adapters/             # Hosted, local, Chinese, vision, vLLM, HuggingFace, mock adapters
├── api/                  # Flask Blueprint REST API
├── webscan/              # Crawler, web scanner, LLM interaction tester
├── engine.py             # Evolutionary algorithm engine
├── mutator.py            # Mutation strategies and MAB selection
├── fitness.py            # Semantic fitness evaluation
└── judge.py              # Dual-judge mechanism

schemas/                  # Report artifact JSON Schema contracts
examples/                 # Runnable suites and fixtures
docs/                     # Landscape, lint roadmap, sample report pack
tests/                    # pytest coverage
frontend/                 # Vue 3 SPA dashboard

forgedan/suite.py is the report-delivery entry point. It handles suite execution, report artifact writing, schema/manifest validation, QA receipts, and archive verification. schemas/ defines artifact contracts, examples/ stores runnable fixtures, docs/ keeps sample report packs and project documentation, and frontend/ contains the Vue 3 dashboard.

API Endpoints

The backend exposes a Flask Blueprint REST API for automation scripts, demos, and internal assessment workflows.

POST   /api/attacks/run
GET    /api/attacks/{id}/status
GET    /api/attacks/{id}/result
POST   /api/attacks/{id}/stop

GET    /api/models/providers
POST   /api/models/test

POST   /api/webscan/crawl
POST   /api/webscan/scan
POST   /api/webscan/llm-test

POST   /api/reports/generate
GET    /api/reports/{id}
GET    /api/reports/{id}/download

GET    /api/datasets
POST   /api/datasets/upload

GET    /api/health
GET    /api/metrics

Documentation Map

Development

pip install -e ".[dev]"

# Full pytest suite used locally
python -m pytest -q -W error::DeprecationWarning -p no:cacheprovider --basetemp .tmp-test

# CI report-pack gates
python -m forgedan.cli suite run examples/ready-for-handoff-suite.yml --output reports/suite-ready
python -m forgedan.cli suite verify-bundle reports/suite-ready/suite-manifest.json
python -m forgedan.cli suite qa-report reports/suite-ready/suite-manifest.json --output reports/suite-ready/qa --strict-handoff
python -m forgedan.cli suite verify-archive reports/suite-ready/handoff.zip

# Selected flake8 gate
python -m flake8 forgedan/ --select=E9,F63,F7,F82,E722,F401,F841 --show-source --statistics

# Formatter gate
python -m black --check forgedan tests

# Frontend build
cd frontend
npm install
npm run build

Roadmap

• Evolutionary engine and mutation strategies
• Six attack methods: FORGEDAN, AutoDAN, PAIR, GCG, Crescendo, TAP
• Hosted, Chinese, local, vLLM, HuggingFace, vision, and mock adapters
• YAML suite runner with imported cases, custom scorers, response cache, source inventory, and policy gates
• Prompt injection, jailbreak framing, system prompt leakage, secrets/PII, Agent/MCP/tool policy risk, and model artifact scanning
• Markdown/HTML reports, redacted publication packs, evidence matrices, case matrices, risk registers, coverage summaries, release notes, and bundle indexes
• JSON Schema contracts, semantic validation, manifest verification, QA receipts, and archive verification
• Checked-in ready-for-handoff sample report pack with screenshots
• CI gates for tests, report-pack validation, strict QA handoff, archive verification, selected flake8, Black, and frontend build
• Add more realistic Agent/MCP manifest fixtures and calibrate the default trust-score policy
• Add HarmBench/JailbreakBench examples only where they improve report evidence quality
• Expand model serialization analysis when the report scope needs deeper artifact review

GitHub About

Suggested repository description:

Report-first LLM security assessment framework for reproducible red-team suites, evidence packs, QA receipts, schemas, and archive verification.

Suggested topics:

llm-security, ai-red-team, prompt-injection, jailbreak, owasp-llm, mcp-security, agent-security, security-reporting, risk-register, audit-evidence, json-schema, pytest, python

Citation

If you use FORGEDAN in research, please cite:

@article{cheng2025forgedan,
  title={FORGEDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models},
  author={Cheng, Siyang and Liu, Gaotian and Mei, Rui and Wang, Yilin and Zhang, Kejia and Wei, Kaishuo and Yu, Yuqi and Wen, Weiping and Wu, Xiaojie and Liu, Junhua},
  journal={arXiv preprint arXiv:2511.13548},
  year={2025}
}

Contributing

Contributions are welcome. When changing report behavior, keep report artifacts, schemas, tests, and documentation aligned. See CONTRIBUTING.md for the general contribution flow.

Security

Use this project only for authorized security testing, research reproduction, and report delivery. Raw prompts, responses, traces, caches, and evidence may contain sensitive data; handle them according to the assessment scope and handoff rules.

License

This project is released under the MIT License. See LICENSE.

---