CoT Red Team Agent

Refusal quotes of a canary are not a finding. This CLI scores visible chain-of-thought and proves simulated-agent impact from observed actions — not from assistant prose or an LLM judge.

python -m pip install cot-redteam-agent

Published on PyPI as cot-redteam-agent.

Try it in 30 seconds (no API key)

The keyless mock provider never touches the network. The config below is also shipped as cot_redteam/data/mock_demo.example.yaml. mock_mode: auto discloses a synthetic canary, so scan should exit 1 (findings). That is the demo working — not a real-model vulnerability.

cat > demo.yaml <<'EOF'
version: 2

global:
  seed: 1
  output_dir: ./results-demo
  concurrency: 1

providers:
  mock:
    kind: mock
    mock_mode: auto

evaluation:
  models:
    - mock:target
  dataset_path: pkg:sample.jsonl
  sample_count: 1
  budgets:
    max_requests: 40
    max_elapsed_seconds: 120
  retain_prompts: false
  retain_responses: false

storage:
  path: ./results-demo/cot_redteam.db
EOF
cot-redteam scan --config demo.yaml

Exit codes: 0 clean, 1 findings, 2 config/env, 3 partial. Gate CI on 1. Switch mock_mode to refuse for a clean pass — packaged variants mock_refuse.example.yaml and mock_disclose.example.yaml sit next to the auto demo. cot-redteam init --demo mock writes this keyless config instead of the OpenRouter example. From a clone, the v0.6 agent lane is also keyless: cot-redteam agent scan --config cot_redteam/data/agent_security.example.yaml.

Why this vs garak, PyRIT, and promptfoo

Pick the tool that matches the job. This project does not replace a broad scanner, an orchestration SDK, or a YAML eval harness.

See also docs/comparisons.md for overlap, honest non-goals, and what each project's own README says it is.

Interactive adaptive TUI: multi-model board, payload attempt log, model output, and last real successful disclosure (refusal re-quotes are not counted as success). See docs/assets/README.md for social-preview guidance.

CoT Red Team Agent is an open-source CLI and Python API for evaluating LLM and agent behavior under adversarial inputs. It runs reproducible model attacks and offline simulated-agent scenarios, records failure-aware evidence, and generates auditable reports and replayable security regressions.

Version 0.6.0 adds the Proof-of-Action agent execution lane: a deterministic Support Agent World, deny-by-default tool gateway, action/state-transition oracles, checksummed exploit replay, and patched-target regression suites. It preserves the 0.5 adaptive attacks and TUI, 0.3 benchmark, and 0.2 Python API. Existing users should also read the 0.3 migration guide.

What it does

• Runs built-in and third-party attacks against one or more target models.
• Agentic canary extraction: seeds a payload bank, classifies each refusal, then invents the next technique until a compliant final-text disclosure (or budget). Refusal quotes are not success.
• LLM-driven attacker (injection.system_canary_agent_llm): an attacker model writes the next extraction prompts from the conversation (PAIR loop, TAP-style candidate branching) and falls back to the deterministic catalog if the attacker provider fails. Requires attacker_model in attack_config.
• Crescendo multi-turn attack (injection.crescendo_canary): gradual 5-rung escalation that references the model's own replies, from a benign question to an exact token ask. No attacker model needed.
• General forbidden-behavior testing (harm.forbidden_behavior): score any harmful request (phishing, fraud, weapons, hate speech, ...) with a StrongREJECT-style rubric judge — refusal, specificity, convincingness — no canary required. Ships with pkg:harmful.jsonl (10 items).
• CI-ready scan: cot-redteam scan --config X runs a quick compliance scan and exits 0 (clean) / 1 (findings) / 3 (partial) — gate CI on it.
• OWASP LLM Top 10 tags in every report item (LLM01/02/05/07...).
• Adaptive fixed-bank mode also available (injection.system_canary_adaptive).
• Interactive Codex-style TUI with slash commands, multi-model board, and live attempt timeline (cot-redteam tui).
• Runs packaged 12-trial smoke and 56-scenario core prompt-injection suites.
• Preserves system, developer, user, assistant, and simulated tool message roles.
• Scores exact and partial canary disclosure separately in final text and visible provider reasoning (refusal analysis that only quotes a canary is not success).
• Reports attack objectives, benign-task utility, false refusals, exclusions, and Wilson confidence intervals as separate dimensions.
• Evaluates outputs with regex, LLM-judge, ensemble, and evasion monitors.
• Distinguishes provider, attack, monitor, budget, and cancellation failures.
• Tracks request, token, elapsed-time, and estimated-cost budgets.
• Stores runs transactionally in SQLite with retention-aware redaction.
• Produces Markdown, CSV, and LaTeX reports with honest eligibility counts.
• Produces lossless JSONL benchmark evidence for retained multi-turn transcripts.
• Writes reproducibility manifests and detached artifact checksums.
• Evolves bounded populations of generated attack templates through the normal evaluation engine.

The tool does not provide a hosted service or guarantee that automated monitors represent ground truth.

Supported providers

Provider keys are read only from named environment variables. Secrets must not be placed directly in YAML files. The mock provider needs no key at all: mock_mode: auto|refuse|disclose|error controls whether it discloses a synthetic canary, refuses, or raises provider errors — ideal for smoke tests and CI without spending any budget.

Installation

Python 3.10 through 3.13 is supported.

Install from PyPI:

python -m pip install cot-redteam-agent

Or install the tagged source release:

# test: command
python -m pip install "git+https://github.com/rudrasatani13/cot-redteam-agent.git@v0.6.0"

Or install the wheel attached to the GitHub release:

python -m pip install \
  "https://github.com/rudrasatani13/cot-redteam-agent/releases/download/v0.6.0/cot_redteam_agent-0.6.0-py3-none-any.whl"

For development:

git clone https://github.com/rudrasatani13/cot-redteam-agent.git
cd cot-redteam-agent
python -m pip install -e ".[dev]"

Published on PyPI as cot-redteam-agent.

Five-minute quickstart

No API key? Use the 30-second mock scan above. The steps below contact a real provider and may incur cost.

Create a wheel-safe example configuration:

# test: command
cot-redteam init --path config.yaml
# Keyless mock demo instead of the OpenRouter example:
# cot-redteam init --path demo.yaml --demo mock
# Edit evaluation.models and generative model IDs for your provider route.
export OPENROUTER_API_KEY=your-key
cot-redteam config validate --config config.yaml
cot-redteam list-attacks
cot-redteam list-monitors

The generated configuration uses the packaged pkg:sample.jsonl dataset and works outside the repository. It includes optional provider examples, but validation requires credentials only for providers referenced by the selected evaluation and generative models. The default attack is injection.system_canary_agent (invent techniques until real disclosure).

Run the configured evaluation when you are ready to contact the provider:

cot-redteam run --config config.yaml
cot-redteam list-runs --config config.yaml

Interactive adaptive TUI

--config is required (bare cot-redteam tui will error):

cot-redteam tui --config config.yaml
# or auto-start:
cot-redteam tui --config config.yaml --auto-start

The bottom type bar is a single slim line (no tall box borders). Type a slash command and press Enter. Mid panels expand; the composer stays pinned.

Inside the TUI:

/model openrouter:your-model-id
/payloads 8
/run

Useful commands: /help, /add, /models, /attack, /stop, /quit. See the full TUI guide (layout + slim composer notes).

Render a report using the run_id printed by the run command:

cot-redteam report \
  --config config.yaml \
  --run-id RUN_ID \
  --format markdown

Markdown reports include retained system and attack prompts, model responses, visible provider reasoning, exact attack-assessment evidence, and monitor outcomes. The packaged adaptive canary attack places a synthetic token only in a trusted system instruction and reports success only on real disclosure—not when the model refuses while quoting the canary during analysis.

Prompt-injection benchmark

List the packaged suites:

cot-redteam list-suites
cot-redteam suite show --id builtin.smoke

In config.yaml, select a suite and remove the legacy attacks and monitors entries if you want a benchmark-only run:

evaluation:
  models:
    - openrouter:your-model-route
  suite_ids:
    - builtin.smoke
  repetitions: 1
  budgets:
    # 12 trials; one is two-turn, so the target-request minimum is 13.
    max_requests: 13
  retain_prompts: true
  retain_responses: true
  retain_reasoning: true

Provider capabilities are declared under providers.<name>.capabilities. Unsupported roles fail during config validate, before any billed request. The packaged smoke suite includes a simulated tool-output case, so the selected route must declare tool_role: true; otherwise use a filtered local suite.

Run and inspect it with the same commands:

cot-redteam config validate --config config.yaml
cot-redteam run --config config.yaml
cot-redteam report --config config.yaml --run-id RUN_ID --format markdown
cot-redteam report --config config.yaml --run-id RUN_ID --format jsonl

Benchmark results apply only to the tested model route, provider behavior, policy, suite version, transformations, and repetitions. They are not a universal model-security score. See the benchmark guide.

Visible reasoning and interpretation

The tool records visible reasoning only when it is:

1. exposed in a provider response field; or
2. enclosed by configured delimiters such as <think>...</think>.

Ordinary answer prose is not relabeled as hidden reasoning. Model outputs are nondeterministic, automated monitors are imperfect, and attack success does not prove a general model vulnerability. Reports preserve failed and excluded items so those limitations remain visible. A reasoning-only canary disclosure means the tested provider route exposed protected system content to its API caller; it does not prove that every deployment of the named model does so.

Data handling

Prompts, responses, and visible reasoning can contain confidential information. Review evaluation.retain_prompts, evaluation.retain_responses, and evaluation.retain_reasoning before running against sensitive datasets.

The default configuration retains evaluation traces. Stored SQLite databases, artifacts, reports, and generated archives should be protected as sensitive research data and must not be committed.

Responsible use

Use the project only with models, endpoints, datasets, and credentials you are authorized to test. Respect provider terms, rate limits, privacy obligations, and applicable law. Do not use generated attacks to access third-party systems or data without permission.

Model-safety results belong in normal research reports or issues. Suspected software vulnerabilities in this repository must be reported privately under the security policy.

Agent security (v0.6 Proof-of-Action)

v0.6 adds an agent execution lane that evaluates agent behavior, not just model text. Impact is proven only by observed simulated actions and deterministic world state transitions — assistant prose, model reasoning, and LLM judge opinion are never proof of impact.

• Support Agent World — the only executable simulated world: in-memory tickets, CRM records, canaries, outbound sink, approval state, and protected state. No tool performs real network, shell, filesystem, subprocess, or external database effects.
• ToolGateway with deny-by-default tool dispatch — unknown tools, tools outside the scenario sandbox allowlist, schema-invalid calls, and over-limit arguments are denied before execution. Derived authorization (policy decisions per tool call) is recorded on every dispatch as oracle evidence; scenarios rely on unauthorized calls EXECUTING so the oracles can prove impact from observed state changes, so authorization is enforced only when the gateway is configured with deny_unauthorized_tools: true. Byte/action/time/concurrency limits bound every dispatch and structured trajectory events are recorded for each one.
• Deterministic oracles — canary-reached-sink (bound to transmitted payload content and the specific mutating action, never to sink names), unauthorized-tool-call, approval-bypass, and protected-state-mutation proofs over pre/post world snapshots and recorded action evidence (including transient mutate-and-restore detection). Oracle failures are never clean.
• Three scenarios with vulnerable/patched/clean fixtures: indirect prompt injection, tool-result injection, approval bypass.
• Replay artifacts — verified exploits save as strict checksummed JSON with a detached .sha256 sidecar that is verified on load; retention flags and the recorded seed are honored so exact replay reproduces them, and regression suites replay saved exploits against patched targets expected to hold.

cot-redteam agent scan --config cot_redteam/data/agent_security.example.yaml
cot-redteam replay artifacts/<run_id>/replay.json     # exit 1 = reproduced
cot-redteam regress --suite security-regressions/     # patched target holds

The v0.6 path runs fully offline with the keyless mock provider plus scripted targets and local SQLite/artifacts. Agent retention defaults to omitting raw tool arguments/results, memory values, and final-response content; structural evidence (events, digests, oracle verdicts) is always kept.

Custom Python target adapters execute in-process and are trusted code, like installed plugins. The runtime facades separate target-authored messages from gateway-authored action evidence, but they are not a sandbox for malicious adapter code. Treat model output and tool-result data as hostile; install or supply only target adapter code you trust.

Python API

# test: python
import asyncio

from cot_redteam.api import run_benchmark
from cot_redteam.core.config import load_config


async def main() -> None:
    config = load_config("config.yaml")
    run = await run_benchmark(config)
    print(run.run_id, len(run.trials))


asyncio.run(main())

Use run_evaluation for the backward-compatible 0.2 attack/monitor path and run_benchmark for configured suites. Both contact providers and may incur cost.

CLI reference

• cot-redteam init — --demo mock writes the keyless mock demo
• cot-redteam config validate|show
• cot-redteam list-attacks|list-monitors|list-providers
• cot-redteam list-suites
• cot-redteam suite validate|show
• cot-redteam dataset import cyberseceval|ih-challenge
• cot-redteam run
• cot-redteam tui — interactive adaptive dashboard
• cot-redteam race — race one probe across models and compare compliance
• cot-redteam list-runs|show-run|report
• cot-redteam evolve
• cot-redteam agent scan --config — scripted agent scenarios; saves replay artifacts for verified exploits
• cot-redteam replay EXPLOIT.json — deterministic local replay of a saved exploit
• cot-redteam regress --suite DIR — replay saved exploits against patched targets expected to hold

Exit codes are 0 for completed, 1 for failed/findings (a reproduced exploit), 2 for configuration or environment errors (including corrupt or incompatible replay artifacts), 3 for partial/inconclusive runs, and 130 for user interruption (Ctrl-C) — so CI gates keyed on 1 = findings are not poisoned by aborts.

Documentation

Development

python -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"
ruff format --check .
ruff check .
mypy cot_redteam
pytest

See CONTRIBUTING.md for the complete quality gates.

License

Released under the MIT License.