jev-cli

Analyze JSON, NDJSON, and JSONC system artifacts with TypeSafe AI's Jev decision model, and get typed, calibrated answers back instead of prose.

$ uv run jcli analyze app.ndjson --pack logs.triage --window 2 --mark-uncertain 0.6 -o table
where           question     type    answer                conf
app.ndjson:1-2  is_incident  noul    0.11 (no)            0.78*
app.ndjson:1-2  severity     score   0.2 (Informational)   0.71
app.ndjson:1-2  category     choice  application           0.42
app.ndjson:3-4  is_incident  noul    0.93 (yes)           0.86*
app.ndjson:3-4  severity     score   1.87 (Outage)         0.71
app.ndjson:3-4  category     choice  dependency            0.88
* noul answers report derived certainty |p-0.5|*2, not an API confidence score.
3 call(s)  1290 in / 36 out tokens

Every answer is anchored to the lines it came from, and every answer carries a number saying how sure the model is — so you can gate on it.

Install

./scripts/install.sh               # puts `jcli` on your PATH via uv
export TYPESAFE_API_KEY=...        # or --api-key-file, or ~/.config/jcli/config.toml
jcli --help

It needs uv and prints the install command if uv is missing. Nothing goes into the system Python; nothing needs sudo. After installing it verifies the binary runs, reports its version, and checks that the built-in packs and transforms actually shipped — a wheel that builds but omits its YAML data is the failure worth catching at install time, not on first use.

To work in the repo without installing, uv sync and prefix commands with uv run.

The idea

Jev takes a state plus a map of typed questions and evaluates all of them in one call, in parallel. So asking eight questions costs about what asking one costs. jcli leans on that: selecting two packs does not make two requests, it makes one request carrying both packs' questions.

$ jcli analyze auth.ndjson -p logs.triage -p security.audit --dry-run | jq '. | length'
1                                   # one window -> one call
$ jcli analyze auth.ndjson -p logs.triage -p security.audit --dry-run \
    | jq '.[0].request.questions | keys | length'
8                                   # ...carrying all eight questions

Three question types, straight from the API:

Input

jcli analyze session.log                    # text log; transform auto-detected
jcli analyze app.ndjson                     # NDJSON, streamed line by line
jcli analyze cluster.json                   # JSON object, or array of records
jcli analyze config.jsonc                   # comments and trailing commas welcome
cat events.ndjson | jcli analyze -          # stdin
jcli analyze a.ndjson b.ndjson              # several sources; windows never straddle files

The dialect is auto-detected; --format json|ndjson|jsonc overrides.

How records are grouped

One flag, --window:

Every mode is additionally capped by --max-state-bytes. The API's true ceiling is not published; jcli probe measures it against the live service so you can set the flag from measurement rather than superstition.

Filtering, narrowing, scrubbing

jcli analyze session.log --filter agent=abc123           # just one agent's records
jcli analyze audit.ndjson --filter 'level=error' --filter 'status>=500'   # ANDed
jcli analyze session.log --filter 'tool!=Bash' --filter 'desc~cargo'
jcli analyze k8s-events.json -s reason -s message        # project down to what matters
jcli analyze audit.ndjson --redact-preset secrets        # scrub before anything leaves the process

--filter FIELD OP VALUE uses the same operators as --fail-on: = != ~ (contains, case-insensitive) !~ >= > <= <. Dotted paths reach nested fields (user.id=u1). Repeating the flag ANDs the terms; for an "or", run the two separately.

It is record-aware, which grep is not: on a log whose fields contain embedded newlines, grep agent=X silently truncates every multi-line record. Filtering also runs before --select and --redact, so you can filter on an identifier and then scrub it.

A filter that matches nothing is an error listing the values the field actually holds — an empty table is a poor way to discover a typo.

Redaction runs strictly before the first byte reaches the SDK. Matched values become [redacted] rather than disappearing, so "did this request carry credentials" is still answerable.

Text logs: transforms

Most system artifacts are not JSON. A transform declares the shape of a text log once, in YAML, and everything downstream — chunking, redaction, packs, confidence gating — works unchanged.

jcli transforms list
jcli transforms show syslog --sample /var/log/auth.log   # see what it makes of a real file
jcli analyze /var/log/auth.log -p security.audit         # transform auto-detected
jcli analyze weird.log -t ./my-transform.yaml            # or point at your own

-t is an override, not a requirement. When the input is not JSON, every transform is scored by parse rate over a sample; a single clear winner is used and announced on stderr. If nothing matches, or two match equally, the run stops and lists the candidates with their scores — a wrong transform yields plausible records with the wrong fields, so guessing is worse than asking.

name: bracketed-kv
description: A bracketed-timestamp key=value log whose last field is quoted free text.

# A record begins here. Lines that do not match belong to the record above, which
# is what lets a field contain embedded newlines.
record_start: '^\[\d{4}-\d{2}-\d{2}T'

pattern: |-
  (?s)^\[(?P<ts>[^\]]*)\]\s+
  agent=(?P<agent>\S+)\s+ tool=(?P<tool>\S+)\s+
  desc="(?P<desc>.*)"\s*$
flags: [verbose]

null_values: ["-"]        # a bare dash becomes null
coerce: { ts: timestamp } # int | float | bool | timestamp | string

syslog is the only transform that ships. Everything else is a file you write and pass with -t ./my-transform.yaml, or drop into ~/.config/jcli/transforms/ to have it discovered and auto-detected like a built-in. A worked example lives in tests/fixtures/bracketed-kv.yaml.

Named groups become JSON keys. Two details matter in practice, both learned from real emitter output rather than assumed:

• Records are not lines. record_start frames them; continuation lines are joined.
• Quoted fields are not shell-quoted. Emitters write command lines into a desc="..." field without escaping the quotes inside, so logfmt, csv, and shlex parsers all truncate at the first inner quote. A greedy group running to the record's final quote is the fix.

Nothing is dropped silently. A record the pattern rejects is kept verbatim under _unparsed and counted in a warning — a log whose format drifted mid-file is itself a finding, and an empty result set is a bad way to discover it.

Transforms layer like packs: built-ins, then ~/.config/jcli/transforms/*.yaml, then -t ./my-transform.yaml.

Question packs

Four ship built in:

jcli packs list
jcli packs show security.audit --resolved    # exactly what would be sent
jcli packs init mine --from logs.triage      # scaffold your own
jcli packs validate ./mine.yaml

Asking inline

--ask / -a defines a question on the command line, no file needed. Two forms:

# shorthand: a noul. The '=' comes first, so a colon in the question is fine.
jcli analyze session.log -a 'stuck=Is the agent repeating itself?'

# full form: the same YAML a pack file uses, so any type works
jcli analyze session.log -a 'focus: {type: score, instructions: "How concentrated?",
                                   criteria: [scattered, clustered, focused]}'

It needs no --pack, merges with one when given, and can reword a built-in question by naming its id. Quote any value containing :, , or ? — YAML flow mappings require it.

Overriding

Packs layer, lowest to highest:

1. built-ins
2. ~/.config/jcli/questions/*.yaml — a file whose name: matches a built-in replaces it
3. --questions FILE — deep-merged overlays, repeatable
4. --set QID.path=value — repeatable, JSON-valued when parseable
5. --drop QID

jcli analyze app.ndjson -p logs.triage \
  --set 'category.criteria.application.what=A bug in our own service code' \
  --drop actionable

A --set only replaces the leaf it names; sibling keys survive. Overrides are re-validated afterwards, because a --set can break a rubric as easily as fix it.

Writing a pack

instructions and criteria accept nested structure, not just strings (Advanced: structure):

name: logs.triage
version: 1
questions:
  severity:
    type: score
    instructions: How severe is the worst condition in these records?
    criteria:                          # ordered, lowest first
      - summary: Informational
        signals: ["debug/info only", "no error fields"]
      - summary: Degraded
        signals: ["elevated latency", "partial retries"]
      - summary: Outage
        signals: ["sustained 5xx", "dependency down", "data loss"]

  category:
    type: choice
    instructions: Which subsystem is the most likely origin?
    criteria:                          # at least two options
      infra:
        what: The host, network, disk, or scheduler beneath the service.
        examples: ["OOMKilled", "disk full", "node NotReady"]
      security:
        what: Authentication, authorization, or tampering signal.
        not_for: Ordinary 401s from routinely expired tokens.

  is_incident:
    type: noul
    instructions: Is this a live production incident?
    criteria:
      "true": User-visible failure or SLO breach in progress.
      "false": Routine errors, retries, or known-benign noise.

Confidence

choice and score answers carry a confidence the API derives from the shape of the probability distribution. noul answers do not — a noul is a single probability and nothing else.

jcli does not invent one. It reports a separate certainty for nouls, computed as |p − 0.5| × 2 and shown under its own column with a *. A noul of 0.5 is maximally unsure; a noul of 0.05 is a confident no. Conflating that with the API's calibrated confidence would make --confident mean two different things in one table.

Nouls do carry a probabilities map — {"true": p, "false": 1 - p} — so every answer type has the same shape and one consumer handles all three:

{"question": "is_incident", "type": "noul", "value": 0.93, "certainty": 0.86,
 "probabilities": {"true": 0.93, "false": 0.07}}

That is a different claim from a confidence. For a two-outcome question the returned number fixes the complement exactly, so writing it out adds no information and invents none.

In the table, a noul renders as 0.93 (yes) / 0.11 (no) — the same value (meaning) shape score has always used, so a confident no is never misread as a weak yes. The label is mechanical (which side of 0.5), inventing no threshold; how strongly it leans is the adjacent certainty column.

--probabilities / -P adds a distribution column to table and summary:

$ jcli analyze app.ndjson -p logs.triage -o table --probabilities
where           question     type    answer                conf   distribution
app.ndjson:1-2  is_incident  noul    0.11 (no)            0.78*   false 89% · true 11%
app.ndjson:1-2  severity     score   0.2 (Informational)   0.71   Outage 85% · Degraded 10% · ...
app.ndjson:1-2  category     choice  application           0.42   dependency 55% · infra 24% · ...

It earns its keep on choice and score, where the runner-up is real information — an even four-way split is a different situation from a clear winner at the same confidence. On a noul the two halves restate one number; they are shown anyway so every row reads the same way. JSON, NDJSON, and CSV output are unaffected by the flag.

jcli analyze app.ndjson -p logs.triage --mark-uncertain 0.7   # keep weak answers, flagged
jcli analyze app.ndjson -p logs.triage --confident 0.7        # drop them

The threshold is part of the flag, so there is no way to ask for dropping without saying how confident is confident enough.

TypeSafe's guidance: act automatically above ~0.9 for high-stakes decisions, never act below 0.5.

Output

--output json|ndjson|table|csv|summary. Defaults to table on a TTY, json in a pipe.

{
  "source": "auth.ndjson",
  "span": {"source": "auth.ndjson", "indices": [0, 4], "count": 5, "lines": [1, 5]},
  "pack": "security.audit",
  "question": "threat_class",
  "type": "choice",
  "value": "privilege_escalation",
  "confidence": 0.93,
  "probabilities": {"privilege_escalation": 0.93, "recon": 0.05, "benign": 0.02},
  "below_threshold": false
}

Using it as a gate

jcli analyze audit.ndjson -p security.audit \
   --fail-on 'suspicious>=0.8 and blast_radius>=1.5'
echo $?    # 6

Grammar is QID OP VALUE joined by and / or. A term matches if any window satisfies it.

Exit 5 is deliberate: a 500 on window 37 of 200 does not throw away 199 good answers. A 401 does abort immediately, because it would fail identically on all 200.

Credentials

Resolved in order: --api-key → --api-key-file → $TYPESAFE_API_KEY → ~/.config/jcli/config.toml → keyring (with the keyring extra).

The token is wrapped so it cannot be printed by accident — repr, f-strings, and dataclass reprs all render ***. A config or key file that is group- or world-readable is refused, not warned about. jcli config show names which source supplied the key without revealing it.

$ jcli config show
config file    ~/.config/jcli/config.toml
base_url       https://api.typesafe.ai
model          jev-latest
api_key        found via $TYPESAFE_API_KEY

Defaults

Repeating the same pack and transform on every run gets old. ~/.config/jcli/config.toml absorbs it:

api_key = "..."

[defaults]
pack = ["agent.session"]
transform = "syslog"    # explicit -t still wins; auto-detect is the fallback
window = 25
output = "table"

An explicit flag always beats a config default, which always beats the built-in default. jcli config prints what resolved, and names any key it ignored.

Development

uv run pytest            # 119 tests, no network
uv run pytest -m live    # smoke tests against the real API (needs TYPESAFE_API_KEY, costs money)
uv run ruff check .
uv run mypy

--dry-run prints the exact request bodies without a key and without touching the network — the fastest way to see what a flag combination actually does.

jcli analyze audit.ndjson -p security.audit --dry-run | jq '.[0].request.state'