Neuromance

A Rust library and runtime for controlling and orchestrating LLMs.

Overview

Neuromance provides high-level abstractions for building LLM-powered applications in Rust.

• neuromance - Main library providing unified interface for LLM orchestration
• neuromance-common - Common types and data structures for conversations, messages, and tools
• neuromance-client - Client implementations for various LLM providers
• neuromance-agent - Agent framework for autonomous task execution with LLMs
• neuromance-tools - Tool execution framework with MCP support
• neuromance-repl - Embedded Python REPL (PyO3) with stateful sessions and Rust-backed callbacks

Neuromance also provides a container runtime for kubernetes.

• neuromance-runtime - Container runtime binary; runs an agent in oneshot or serve mode

Development

Prerequisites

• Nix with flakes enabled

Nix provides the pinned Rust toolchain (stable channel, via rust-toolchain.toml) and all build dependencies. No separate Rust install is needed.

Dev shell

nix develop

Building

nix build              # library
nix build .#neuromance-runtime

Checks

nix flake check        # fmt + clippy + tests + build (all crates)

Or run individual checks (cached via crane):

nix run .#fmt
nix run .#clippy
nix run .#test

Container images

nix build .#neuromance-image           # minimal runtime
nix build .#neuromance-toolkit-image   # adds busybox, git, curl, jq, node, python3, shell

Generic image verbs take the image short-name (neuromance or neuromance-toolkit) as the first argument:

nix run .#list                                        # list available images
nix run .#load    -- <image>                          # load into local docker
nix run .#push    -- <image> <registry> <namespace> [tag]
nix run .#release -- <image> <registry> <namespace>   # pushes :imageTag + :floatingTag
nix run .#inspect -- <image>                          # opens dive
nix run .#info    -- <image>                          # name, tags, store path, size

Workspace Structure

neuromance/
├── crates/
│   ├── neuromance/           # Main library
│   ├── neuromance-common/    # Common types and data structures
│   ├── neuromance-client/    # Client implementations
│   ├── neuromance-agent/     # Agent framework
│   ├── neuromance-tools/     # Tool execution framework
│   ├── neuromance-repl/      # Embedded Python REPL
│   ├── neuromance-db/        # Postgres persistence for conversations
│   └── neuromance-runtime/   # Container runtime binary
├── Cargo.toml                # Workspace configuration
└── README.md

Providers

Endpoints, credentials, and default models are grouped into named [[providers]] entries. The [agent] references one by name; each subagent inherits the agent's provider unless it names its own. A provider supplies credentials via exactly one of two paths — api_key_env (a raw key read from the environment) or an inline [providers.proxy] table (a sealed token routed through a tokenizer proxy). At least one provider is required.

mode = "serve"

[[providers]]
name = "primary"
model = "openai:gpt-4o"          # default model; an agent/subagent may override it
api_key_env = "OPENAI_API_KEY"   # raw key from the environment

[agent]
id = "research-agent"
provider = "primary"             # references a [[providers]] entry
# model omitted — inherits the provider's "openai:gpt-4o"
system_prompt = "..."

The provider: prefix on a model string (openai:, anthropic:, chat_completions:, …) selects the client type and the default endpoint; the provider's base_url overrides that endpoint. A model with a generic prefix (chat_completions:, responses:) has no default endpoint, so its provider must set base_url.

Tokenizer proxy

When a provider is deployed behind a tokenizer proxy, the agent pod never holds the plaintext credential. The pod reads a sealed token from a projected secret, sends it to the proxy under X-Tokenizer-Token, and the proxy decrypts the token and injects the real provider key server-side before forwarding to the upstream.

The provider's base_url and [providers.proxy].base_url carry distinct URLs:

• base_url is the upstream LLM endpoint. Falls back to the provider default from the model prefix (e.g. openai:gpt-4o → https://api.openai.com/v1).
• [providers.proxy].base_url is the tokenizer proxy itself. The client attaches it as an HTTP forward proxy so requests leave the pod in absolute-form (RFC 7230 §5.3.2) with the upstream authority in the request URL — no side-band routing header is needed.

[[providers]]
name = "primary"
model = "openai:gpt-4o"
# Upstream provider URL. Optional when the model prefix has a default
# (here, openai → api.openai.com). Set this to pin a different upstream,
# e.g. https://openrouter.ai/api/v1.
base_url = "https://openrouter.ai/api/v1"
# api_key_env is omitted — the proxy is the credential source.

[providers.proxy]
# Tokenizer proxy Service inside the cluster. Attached as the HTTP forward proxy.
base_url = "http://tokenizer-proxy.windowlickers.svc.cluster.local:8080"
# Projected Secret volume holding the sealed token.
token_file = "/var/run/neuromance/tokens/llm"
# Header carrying the sealed token. Defaults to X-Tokenizer-Token.
token_header = "X-Tokenizer-Token"

Conversation persistence

With an optional [database] section, the runtime writes conversation history through to postgres as tasks run — a shared durable record any number of agents can write into. Messages are stored as an append-only log (idempotent per message id, ordered by a per-conversation sequence), so the full history including tool calls and tool results survives restarts and context compaction. In-memory state stays authoritative for serving; persistence is best-effort and never blocks a running task, while a failed connection at startup fails fast.

[database]
# Environment variable holding the postgres URL — the URL embeds a
# credential, so it never lives in this file (same policy as a provider's
# api_key_env).
url_env = "DATABASE_URL"
max_connections = 5            # optional, default 5
acquire_timeout_seconds = 5    # optional, default 5
run_migrations = true          # optional, default true

Migrations are embedded in neuromance-db and applied automatically at startup. Set run_migrations = false when an external owner (an operator or a shared schema service) manages the database — the runtime then connects to and uses the existing schema without attempting any DDL. The embedded migrations remain the source of truth for neuromance's tables regardless of who applies them. See the neuromance-db crate docs for the schema and the cargo sqlx prepare workflow when changing queries.

Subagents

A [[subagents]] section declares subagents the main agent can delegate to. A subagent that pins neither provider nor model runs whatever the main agent runs for that task — a serve task's per-task provider/model override included — so it only needs an id and system_prompt. Pinning either one keeps the subagent's own identity and ignores the per-task override: the effective model is then the subagent's model, then the chosen provider's default model, then the agent's configured model.

A subagent is provisioned with the same toolset as the main agent — the capability tools from [[tools]], the execute_python bridge, and the delegate tools — so it can both use tools and delegate further. Its conversation loop is wired the same way too: it persists through [database], compacts under [context], carries the [skills] menu in its system prompt with $mention expansion in-loop, receives [rules], and is told its [workspace] directory. Two things are deliberately left off. Subagents never stream — a delegate tool returns only the final content. And subagent tool calls auto-approve inside the pod: they run autonomously within a single parent delegation, with no interactive approver in the loop, so the pod boundary (e.g. kata containers) is the isolation, the same way the whole agent pod is already sandboxed.

Self-delegation. Setting runtime.self_delegation = true gives the main agent a task delegate tool that runs a fresh copy of itself — the same system prompt, provider, model, and turn budget — on a scoped subtask, with no [[subagents]] entry to write. It is exactly a synthesized subagent named task, so it joins the same max_delegation_depth tower, gets the same shared toolset, and (when execute_python is configured) the same run_subagent/spawn_agents bridge. task is reserved while this is on: an explicit subagent or tool of that name is rejected at startup. Combine it with [[subagents]] freely — the task self-clone is added alongside any named subagents.

One divergence: a serve task that supplies its own system_prompt overrides the main agent's seed message only. The task clone still carries agent.system_prompt, because a subagent's prompt is fixed when the delegation tower is built and the per-task prompt is not known then.

Nested delegation is bounded by runtime.max_delegation_depth, which counts subagent hops from the main agent (depth 0). At 1 the main agent reaches subagents but those subagents carry no delegate tools; at 2 (the default) a subagent may delegate one further hop, and so on. The deepest subagents are still fully tool-capable — they simply cannot delegate. The bound is enforced structurally (a finite tower of subagent instances built at startup), so the chain depth cannot run away; it is capped at 5.

The bound is on depth alone. Each delegated run carries its own agent.max_turns budget rather than drawing down the parent's, so within the depth limit an agent may still delegate many times — and spawn_agents runs a batch concurrently. Set a finite agent.max_turns to bound the work a single task can generate, especially with self_delegation on, where the clone inherits the same instructions that told the parent to delegate.

Every configured subagent is reachable two ways:

• As a delegate tool — an agent gets a tool named after the subagent's id, taking instructions and optional context. Like other tools, a delegate tool is not auto-approved, so the main agent's tool calls under approval.mode = "auto" are subject to the same startup safety gate (set approval.allow_unsafe_tools = true or use async approval).
• From the Python REPL — when an execute_python tool is also configured (requires the python-repl build feature), the runtime builds the REPL with the subagents bridged in as run_subagent(name, instructions, context=None) and spawn_agents([Agent(name, instructions), ...]), so the agent can write its own orchestration in Python. The bridge runs in restricted mode; an execute_python entry with restricted = false alongside subagents is rejected. The deepest subagents, having no children to delegate to, get a plain execute_python (no run_subagent/spawn_agents). Each subagent run gets its own fresh interpreter — including each concurrent run in a spawn_agents fan-out — so interpreter state never bleeds across runs.

[runtime]
# Max subagent hops in a delegation chain. Optional; defaults to 2.
max_delegation_depth = 2

[[subagents]]
id = "researcher"
system_prompt = "You research a question and report findings."
# provider, model, max_turns all optional; provider defaults to the agent's,
# model defaults to the provider's default then the agent's effective model.

[[subagents]]
id = "critic"
system_prompt = "You critique a draft and list concrete fixes."
description = "Delegate a draft to the critic for review."
model = "anthropic:claude-opus-4-8"
max_turns = 4

# Capability tools and the Python bridge are shared by the main agent and every
# subagent. Bridge the subagents into Python (run_subagent / spawn_agents):
[[tools]]
name = "execute_python"

Sandboxed tool execution

Tool execution can run in a separate sandbox process so that a misbehaving agent — one that escapes the in-process Python sandbox or abuses bash — has no path to the database or the LLM credentials. The orchestrator and the sandbox run as two containers in one pod under different service accounts: the orchestrator (sa/mancer) holds the DB and provider credentials and runs the agent loop, approval, and persistence; the sandbox (sa/mancer-sandbox) executes the capability tools (bash, file tools, grep/find/ls, execute_python) against the workspace, with an egress NetworkPolicy blocking Postgres and no place in the database AuthorizationPolicy.

The same binary runs both roles, selected by subcommand:

neuromance-runtime           # orchestrator (default; also `run`)
neuromance-runtime sandbox   # sandbox tool executor

Both processes read the same config file. The [sandbox] section configures the boundary:

[sandbox]
# Address the sandbox process binds its gRPC server to. Loopback-only: the
# two containers share the pod network namespace, so the channel never leaves
# the pod. Optional; defaults to 127.0.0.1:50051.
listen_addr = "127.0.0.1:50051"
# Endpoint the orchestrator dials. When set, the orchestrator advertises the
# sandbox's tools to the LLM and routes every approved tool call there instead
# of executing in-process. Unset (or no [sandbox] section) keeps tools local.
endpoint = "http://127.0.0.1:50051"

The orchestrator fetches the sandbox's tool definitions once at startup (retrying while the sandbox container comes up) and mirrors each tool's auto-approval requirement, so the startup approval gate behaves exactly as it does for local tools. Approval is always decided by the orchestrator before a call crosses the boundary; the sandbox only executes. A tool that runs but fails is returned as a normal tool error; a transport failure degrades to a failed tool call rather than crashing the orchestrator.

The channel is loopback-only within the pod (Istio mesh covers transport; the channel construction is left pluggable for future SPIFFE/SPIRE identities). With [workspace] configured, the workspace volume is an emptyDir shared by both containers: the sandbox executes tools inside it, while the orchestrator — which holds the storage and git credentials — seeds and snapshots it. Tool code still has no reachable credentials.

The gRPC build needs protoc (provided by the Nix dev shell and build inputs).

Limitation (this release): the Python run_subagent/spawn_agents bridge cannot cross the sandbox boundary — it needs the interpreter and the subagent tower in one process. A config that sets sandbox.endpoint together with an execute_python tool and any delegation ([[subagents]] or runtime.self_delegation) is rejected at startup. Delegation and a standalone execute_python each work under the sandbox individually.

Workspaces

Each conversation gets a working directory at <root>/<conversation_id>, created when its first task runs and reused by continuations. The directory is announced in the seed system message and injected as bash's default cwd (an explicit cwd in a tool call always wins), on both the in-process and sandbox execution paths. Subagents inherit the parent's workspace, and each delegated task carries the same directory note — a subagent's system prompt is fixed at startup, before any conversation has a workspace, so the note is attached to the task instead.

[workspace]
# Absolute root under which per-conversation directories are created.
# In k8s this is an emptyDir mounted at the same path in the orchestrator
# and sandbox containers.
root = "/workspace"

# S3-compatible object storage (garage). Required by object seeds and
# persistence; credentials come from the environment, never this file.
[workspace.storage]
endpoint = "http://garage.storage.svc.cluster.local:3900"
bucket = "workspaces"
# region defaults to "garage"
access_key_id_env = "GARAGE_KEY_ID"
secret_access_key_env = "GARAGE_SECRET"

# Snapshot/restore across replicas. Optional — omit for pod-local
# workspaces only.
[workspace.persistence]
# Object key prefix; a conversation's snapshot lives at
# <prefix><conversation_id>.tar.zst. Defaults to "snapshots/".
prefix = "snapshots/"
# Globs excluded from snapshots, relative to the workspace dir.
exclude = ["target/**", "**/node_modules/**", "**/__pycache__/**"]

# Tokenizer-proxy auth for git seeds, shaped like [providers.proxy].
# Omit for anonymous clones. Remotes must be http:// so the proxy can
# read the sealed header (it upgrades to TLS upstream); a non-http:// git
# seed url is rejected at config load.
#
# Proxied clones run the `git` CLI, so they need a git binary on PATH:
# use the toolkit image (`neuromance-toolkit`), which carries one. The
# minimal image does not. Anonymous clones use libgit2 in process and
# run on either image.
[workspace.git_proxy]
base_url = "http://tokenizer-proxy.windowlickers.svc.cluster.local:8080"
token_file = "/var/run/neuromance/tokens/git"

# Named seed definitions. Selected per task by the `workspace` field on
# POST /tasks/new (or `[oneshot].workspace`); with no field, the sole
# definition is used when exactly one is configured. Seeding runs exactly
# once, when the conversation's workspace is first created.
[[workspace.definitions]]
name = "org-dev"

# Repositories pre-cloned into the workspace.
[[workspace.definitions.git]]
url = "http://git.windowlicke.rs/windowlickers/neuromance.git"
ref = "main"                      # branch, tag, or SHA; optional
# dest defaults to the repo basename ("neuromance")

# tar.zst archives from workspace.storage, unpacked into the workspace —
# scratch files and other goodies.
[[workspace.definitions.objects]]
key = "seeds/scratch.tar.zst"
dest = "scratch"                  # optional; defaults to the workspace dir

With persistence on, the worker snapshots the workspace after every run (success, failure, or drain cancellation) and restores the latest snapshot before a continuation that lands on a different replica — a marker file carrying the snapshot's ETag lets a task that lands on the replica which produced the snapshot skip the restore. Snapshot refs are written through to postgres when [database] is configured (the workspace_snapshots table), for audit and retention sweeps; the object store stays authoritative for restore. A restored workspace is never re-seeded, and a continuation naming a different workspace keeps the one it was seeded with (logged, not an error).

Retention is expiry-based and external: point a garage lifecycle rule at the snapshot prefix rather than deleting from the runtime. Size the pod's terminationGracePeriodSeconds above runtime.shutdown_grace_seconds plus the time to pack and upload a snapshot of your workspace's working set — the drain-time snapshot runs inside that window.

Structured outputs

A task can require the agent's final answer to be a JSON object matching a schema. The schema is enforced by the provider, not by a prompt: the runtime passes it through to the model's native structured-output parameter (response_format on Chat Completions, text.format on the Responses API, output_config.format on Anthropic). A provider that does not support it fails the run rather than answering unconstrained.

In serve mode the schema is per task, in the POST /tasks/new body:

{
  "task": "Rate the pull request in the workspace.",
  "output_schema": {
    "title": "verdict",
    "type": "object",
    "properties": {
      "score": { "type": "integer" },
      "summary": { "type": "string" }
    },
    "required": ["score", "summary"],
    "additionalProperties": false
  }
}

In oneshot mode it is config, under [oneshot].output_schema, with the same shape.

The schema must be a closed JSON object: the root declares "type": "object", and every object subschema — including those under properties, $defs, items, and anyOf — sets "additionalProperties": false. Both are checked locally, so an unenforceable schema is rejected at intake (400, before a task is minted) or at startup validation rather than mid-run, and the error names the path of the offending subschema. OpenAI strict mode additionally requires every property to appear in required; that one is enforced by the provider, not here. Nesting deeper than 64 levels is rejected too. The optional title names the schema on the wire; it defaults to output and must be 1–64 characters of a-z, A-Z, 0-9, _ or -, the range OpenAI accepts.

The constraint applies to whatever turn carries it, so it never rides on a turn that offers tools: the model would answer in the schema where it should call a tool. The agent runs its tool loop unconstrained, and when the model stops calling tools it is asked once more — with the schema and no tools — for the answer. That final tool-free turn is the one that must parse as a JSON object; the unconstrained answer it replaces is discarded and never reaches the conversation history. A task with no tools configured skips the extra call: its first turn is already the final one.

Conformance beyond that is the provider's job. The runtime checks that the final turn parses as a JSON object; it does not re-validate the object against the schema, so a provider that ignores the constraint returns a structured field that satisfies no more than "is an object".

A successful task returns the parsed object in a structured field alongside the prose output, on GET /tasks/{id} and in the oneshot JSON. With [database] configured it is stored in the tasks.structured JSONB column. A run whose final turn is not a schema-valid object fails with schema_violation; a refusal fails with schema_refusal and a response cut off at the token limit fails with schema_truncated. None of the three are retried — the provider already enforced the schema, so a retry would only repeat the failure.

Tool bootstrap

Some tools cache their credentials to disk rather than reading them from the environment on each call. The agent pod has no persistent storage, so those tools must be logged in at container start. Each [[bootstrap]] entry names a command the runtime runs once before tasks begin; failures are logged but never fatal — a tool that can't be set up just isn't available, the same as any other tool error.

The runtime has no per-tool knowledge: the full command and arguments are supplied in config, so the deployer (typically an operator) bakes them in. A secret is never placed in args — config.toml ships in a ConfigMap. Instead, set token_env to the name of an environment variable (sourced from a Secret); the runtime feeds that variable's value to the command on stdin, so the credential never lands in argv.

[[bootstrap]]
name = "forgejo"                 # label for logs only
command = "fj"                   # executable, must be on PATH
args = [
  "--host", "git.example.com",
  "auth", "add-tokenizer",
  "--proxy", "http://tokenizer-proxy.windowlickers.svc.cluster.local:8080",
  "--name", "forgejo",
]
# Optional. When set, the value of this env var is fed on stdin, never in args.
token_env = "FORGEJO_TOKEN"

Context compaction

An optional [context] section enables automatic conversation compaction in the runtime. Once the conversation grows past a ratio of the model's context window, older turns are summarized by the LLM and replaced with the summary, preserving the system prompt and the most recent turns.

Conversation size is measured from the provider-reported usage of the most recent response, so no tokenizer is downloaded at startup. One known lag: the first request of a resumed conversation is sent uncompacted because no usage exists yet in that run — compaction at the end of the previous run keeps stored histories under target.

Subagents compact too, on the same settings and against their own conversations. Each subagent run gets its own compactor, so concurrent runs in a spawn_agents fan-out never influence each other's decisions. Long delegated runs therefore summarize instead of walking into the context window, at the cost of the summarization calls that implies.

[context]
# Model context window in tokens. Required; everything else is optional.
context_window_size = 128000
# Compact once usage exceeds this ratio of the window (default 0.8).
compaction_threshold_ratio = 0.8
# Aim for this ratio of the window after compaction (default 0.5).
target_ratio = 0.5
# Recent user+assistant turns preserved verbatim (default 3).
preserve_recent_turns = 3
# one_shot (default) | hierarchical | truncate
strategy = "one_shot"

Skills

An optional [skills] section gives the agent skills — directories containing a SKILL.md (YAML frontmatter with name and description, plus optional agentskills.io fields, followed by a Markdown body of reusable instructions). Skills use progressive disclosure: a cheap menu of name: description (file: …) is folded into the system prompt, and a skill's full body lives on disk, read only when needed. This keeps the system prompt stable and cache-friendly.

Skills are discovered from on-host directory roots (each immediate subdirectory containing a SKILL.md is a skill) and/or a corpus-shaped HTTP endpoint (GET /skills for the menu, GET /skills/{id} for a body). On-host roots take precedence over the endpoint when a skill name appears in both.

At startup the runtime materializes every discovered skill to a local temp directory and rebuilds the catalog over it, so the corpus is only a boot-time dependency. The menu then lists each skill's on-disk path, and the agent loads a skill by reading that file with its read tool — no fetch at reasoning time. In addition, when mention is enabled a $name, skill://id, or [text](skill://id) link in user input (for example a /thing-skill slash command) injects the body inline as a message; common shell variables ($PATH, $HOME, …) are ignored.

Subagents see the same catalog. The menu is appended to each subagent's configured system_prompt (a subagent has no seed message to fold it into), and $mention expansion works inside a delegated run the same way it does for the main agent.

[skills]
# On-host skill directories, highest precedence first.
roots = ["/etc/neuromance/skills", "./skills"]
# Optional corpus-shaped endpoint serving skills over HTTP.
endpoint = "https://corpus.internal/api/v1/skills"
# Optional env var holding a bearer token for the endpoint.
endpoint_token_env = "CORPUS_TOKEN"
# Expand `$name` mentions in user input into the skill body (default true).
mention = true
# Byte budgets for the menu and each mention-injected body (default 8192 each).
menu_budget_bytes = 8192
body_budget_bytes = 8192

Rules

An optional [rules] section gives the agent rule files — Markdown files (.md/.mdc) with optional YAML frontmatter that inject instructions by location rather than by intent. Where a skill is summoned when its description matches the task, a rule is pushed in automatically: a rule marked always_apply is injected once at the start of every conversation, and a rule with globs is injected the first time a tool touches a file whose path matches one of its patterns. Each rule is injected at most once per conversation.

Recognized frontmatter keys are globs (a YAML sequence or a comma-separated scalar; paths is accepted as an alias), always_apply (alwaysApply is also accepted), and description. A file with no frontmatter is a body-only rule with no globs. Glob patterns follow gitignore-style semantics and are matched workspace-relative, so *.rs matches a Rust file in any directory and src/**/*.rs matches only under src/.

Rules are discovered from on-host directory roots (walked recursively; a rule's id is its path relative to the root) and/or a corpus-shaped HTTP endpoint (GET /rules for the listing, GET /rules/{id} for a body). On-host roots take precedence over the endpoint when a rule id appears in both.

Subagents receive rules from the same catalog on the same terms: always-apply rules at the start of a delegated run, glob-matched rules the first time the subagent's own tool calls touch a matching path.

[rules]
# On-host rule directories, highest precedence first (searched recursively).
roots = ["/etc/neuromance/rules", "./rules"]
# Optional corpus-shaped endpoint serving rules over HTTP.
endpoint = "https://corpus.internal/api/v1/rules"
# Optional env var holding a bearer token for the endpoint.
endpoint_token_env = "CORPUS_TOKEN"
# Byte budget for each injected rule body (default 8192).
body_budget_bytes = 8192

A rule file looks like:

---
globs: "*.rs"
description: Rust conventions
---
Follow the repository's error-handling rules: no unwrap/expect outside tests.

Observability

The runtime exports traces, logs, and GenAI metrics over OTLP, and operational metrics over a Prometheus scrape endpoint.

Enabling export

OTEL_EXPORTER_OTLP_ENDPOINT is the master switch. When it is unset, the runtime emits stderr logs and Prometheus metrics only.

RUST_LOG gates span creation, not just log output. A RUST_LOG=warn deployment with OTLP configured exports no gen_ai spans at all. Use RUST_LOG=info, the runtime default.

Spans

Instrumentation follows the OpenTelemetry GenAI semantic conventions, pinned to v1.38.0. Attribute keys live in crates/neuromance-common/src/telemetry/genai.rs; update that file and its version pin together.

invoke_agent {agent}              neuromance-agent    INTERNAL
└── chat_turn                     neuromance          INTERNAL
    ├── chat {model}              neuromance-client   CLIENT   (one per retry attempt)
    └── execute_tool {tool}       neuromance          INTERNAL

chat_turn is not a GenAI operation. It is one iteration of the orchestration loop, spanning both the model call and the tool executions that follow, and it keeps the turn grouping that makes a long agent run readable.

The usage attributes on invoke_agent are the whole run's, so one span prices a run without a join across its chat children. Cached and cache-creation tokens are broken out because they bill at different rates, and gen_ai.request.model is the agent's configured model — a compactor configured with a different model bills against its own chat spans. The counts exclude subagents: each delegated run carries its own invoke_agent span, which is what keeps a sum over a delegation tree from counting the same tokens twice. A run that fails after calling the provider still reports what it spent.

server.address names the upstream provider, not the tokenizer proxy: the proxy is transport plumbing, and reporting it would make every provider look like one server. A streaming call keeps its chat span open until the stream ends, so the span times the whole generation and carries the final token counts. A retried request produces one chat span per attempt.

Metrics

Two pipelines run side by side, and nothing is emitted to both.

• gen_ai.client.token.usage and gen_ai.client.operation.duration go out natively over OTLP, with the units and bucket boundaries the conventions advise. A Prometheus round-trip would turn the dots into underscores and drop the buckets. Attributes: gen_ai.operation.name, gen_ai.provider.name, gen_ai.request.model, server.address, server.port, plus gen_ai.response.model and error.type when known, and gen_ai.token.type (input / output) on the token histogram.
• neuromance_* operational metrics (queue depth, task outcomes, tool durations, approvals, persistence) are Prometheus, scraped from GET /metrics on runtime.health_addr (default :8081).

A collector needs both a receiver for the OTLP push and a scrape job for /metrics:

receivers:
  otlp:
    protocols:
      grpc: { endpoint: 0.0.0.0:4317 }
  prometheus:
    config:
      scrape_configs:
        - job_name: neuromance
          scrape_interval: 30s
          static_configs:
            - targets: ["neuromance:8081"]

service:
  pipelines:
    traces:  { receivers: [otlp], exporters: [clickhouse] }
    logs:    { receivers: [otlp], exporters: [clickhouse] }
    metrics: { receivers: [otlp, prometheus], exporters: [clickhouse] }

Capturing prompts and completions

OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true adds gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions, gen_ai.tool.call.arguments, and gen_ai.tool.call.result to the spans above.

This ships conversation content to your trace backend. Everything a user types, everything a model answers, and every argument and result of every tool call. API keys are not captured — they live in SecretString and are never part of a captured structure — but user data in prompts is. The flag is off by default; turn it on per-deployment, and only where the trace store has the same access controls as the conversations themselves.

Model Context Protocol (MCP) Support

Neuromance supports the Model Context Protocol for connecting to external tool servers via the neuromance-tools crate.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Authors

• Evan Dobry (@ecdobry)

Acknowledgments

Named after William Gibson's novel Neuromancer.