decide

Make decisions from the command line. No code required.

Use decision models like Typesafe's Jev to make smart decisions in your scripts, pipelines, and agent skills.

Decision models let you ask and answer yes/no, multiple choice, and multiple level questions. Decision models can return an answer, the model's confidence in its answer, the probability of the answer being true, and even a full distribution across possible answers.

Install and Setup

$ brew install vsekhar/tap/decide
$ decide --set-config --model typesafe:jev-latest --api-key -
API key: <paste your API key>

You can also set DECIDE_MODEL and DECIDE_MODEL_API_KEY in the environment, or write them to $HOME/.config/decide/config.

Usage

# Yes/no decision (default)
$ decide "Is Atlanta the capital of Georgia?"
yes

# Choose from among options
$ decide "What kind of weather is typical in Florida?" \
    --option rainy \
    --option sunny \
    --option snowy
sunny

# Provide context
$ decide --context @ticket.txt \
         "Which team handles this ticket?" \
         --option shipping \
         --option billing \
         --option returns
returns

# Choose a level on a scale specified "least" to "most"
$ decide --context @ticket.txt \
         "How urgent is this ticket?" \
         --level not_urgent \
         --level somewhat_urgent \
         --level urgent
not_urgent

# Improve decisions by explaining options to the model
$ decide --context @ticket.txt \
         "How urgent is this ticket?" \
         --level not_urgent="Customer feedback or feature request" \
         --level somewhat_urgent="Customer problem, but customer not blocked" \
         --level urgent="Customer blocked"
somewhat_urgent

# Compose context from multiple sources, refer by name in questions and options
$ decide --context ticket=@ticket.txt \
         --context refund_policy=@refund_policy.txt \
         "Should we issue a refund?" \
         --yes yes="Allowed by refund_policy and requested in ticket"
no

# Give the model structured context; --context-json parses its value as JSON
$ decide --context-json order='{"total": 45.00, "days_since_delivery": 12}' \
         --context refund_policy=@refund_policy.txt \
         "Should we issue a refund?"
yes

# Improve performance and cost by asking multiple questions at once against the same context
$ decide --context @ticket.txt \
     "Which team handles this ticket?" \
         --option shipping \
         --option billing \
         --option returns \
     "How urgent is this ticket?" \
         --level not_urgent \
         --level somewhat_urgent \
         --level urgent \
     "Should we issue a refund?"
returns
somewhat_urgent
yes

# Name questions; a named question prints as name=answer
$ decide --context @ticket.txt \
     "Which team handles this ticket?" \
        --name team \
        --option shipping \
        --option billing \
        --option returns \
     "Should we issue a refund?" \
         --name refund
team=returns
refund=yes

Advanced usage

Question files

Questions and their details can be read from files. This is useful for placing questions under version control.

# Text question files mimic the command line
$ cat triage.txt
"Which team handles this ticket"
    --option shipping
    --option billing
    --option returns

"How urgent is this ticket"
    --level not_urgent
    --level somewhat_urgent
    --level urgent

"Should we issue a refund"

$ decide --context @ticket.txt --questions @triage.txt
returns
somewhat_urgent
yes

Multiple question files can be specified, and questions on the command line and in files can be mixed. Questions from a file are inserted where the corresponding --questions flag appears on the command line.

# JSON question files can use names, richer descriptions and structured instructions.
$ cat triage.json
{
  "questions": [
    {
      "name": "team",
      "instructions": "Which team handles this ticket?",
      "options": [
        {
          "id": "shipping",
          "summary": "Delivery issues",
          "examples": ["Package is late", "Tracking says delivered but nothing arrived"],
          "signals": ["Names a carrier or a tracking number"]
        },
        {
          "id": "billing",
          "summary": "Payment problems",
          "not_for": "Money back for an item the customer returned; that is returns",
          "examples": ["Charged twice", "Card declined at checkout"]
        },
        {
          "id": "returns",
          "summary": "Exchanges and refunds",
          "not_for": "Damage in transit; that is shipping",
          "examples": ["Wrong size", "Wants money back for a returned item"]
        }
      ]
    },
    {
      "name": "urgency",
      "instructions": "How urgent is this ticket?",
      "levels": [
        {"id": "not_urgent",      "summary": "Customer feedback or feature request"},
        {"id": "somewhat_urgent", "summary": "Customer problem, but customer not blocked"},
        {"id": "urgent",          "summary": "Customer blocked",
                                  "signals": ["cannot", "stuck", "deadline", "today"]}
      ]
    },
    {
      "name": "refund",
      "instructions": {
        "question": "Should we issue a refund?",
        "rules": [
          "Apply `refund_policy` to the `ticket`.",
          "When the policy is silent, answer no."
        ]
      },
      "yes": {"id": "Yes", "summary": "The policy allows a refund for this case"},
      "no":  {"id": "No",  "summary": "The policy forbids it, or the customer does not ask for money back"},
      "min-confidence": 0.7,
      "fallback": "No"
    }
  ]
}

$ decide --context ticket=@ticket.txt \
         --context refund_policy=@refund_policy.txt \
         --questions @triage.json

team=returns
urgency=somewhat_urgent
refund=Yes

Standard input

# - reads the whole of standard input as a context or as a question file, once per run
$ cat ticket.txt | decide --context ticket=- --questions @triage.txt
$ cat triage.txt | decide --context @ticket.txt --questions -

A run reads standard input once, so - may appear once on a line, whether as a context, a question file, or the API key. @- names a file called -.

Confidence bars

# A question below its --min-confidence bar prints an empty answer; the others print theirs
$ decide --context @ticket.txt \
     "Which team handles this ticket?" \
         --name team \
         --option shipping \
         --option billing \
         --option returns \
     "Should we issue a refund?" \
         --name refund \
         --min-confidence 0.9
team=returns
refund=

stderr>  Unsure: question 2 ("Should we issue a refund?") has confidence 0.74, below the bar of 0.90
$ echo $?
2

The run exits 2 when any question is below its bar, after every line has printed. Check the exit code before you read stdout.

Statistics: confidence and probabilities

# Request stats (confidence, probability) for a question with --stats
# Request full distribution (probabilities of all answers) for question with --distribution
$ decide --context @ticket.txt \
         "Which team handles this ticket?" \
             --name team \
             --option shipping \
             --option billing \
             --option returns \
             --distribution \
         "How urgent is this ticket?" \
             --level not_urgent \
             --level somewhat_urgent \
             --level urgent \
             --stats \
         "Should we issue a refund?" \
             --name refund
team=returns	confidence:0.910 probability:0.910	shipping:0.060	billing:0.030	returns:0.910
somewhat_urgent	confidence:0.780 probability:0.550 score:1.150
refund=yes

# Fields are tab-separated; values inside the stats field are space-separated
$ decide ... --stats | cut -f2 | cut -d' ' -f1 | cut -d: -f2                             # confidence
$ decide ... --distribution | cut -f3- | tr '\t' '\n' | grep '^billing:' | cut -d: -f2   # p('billing')

The chosen answer is always the answer with the highest probability and --stats prints that value in the probability field.

The confidence value measures how sure the model is that it can answer the question with the given options. This is usually the value you want to check: low confidence means you should be cautious in acting on the model's decision.

Compare the two responses to identical questions with different options:

$ % decide "What is the capital of Georgia?" \
         --option Atlanta \
         --option Chicago \
         --distribution
Atlanta confidence:1.000 probability:1.000      Atlanta:1.000   Chicago:0.000

$ decide "What is the capital of Georgia?" \
         --option Atlanta \
         --option Tbilisi \
         --distribution
Tbilisi confidence:0.520 probability:0.760      Atlanta:0.240   Tbilisi:0.760

Notice that confidence is a function of the question as well as the given options. When the options consist of only US cities, "Georgia" is resolved to the US state and the model can answer confidently. When the options include Tbilisi, the model is no longer confident it can correctly answer the (ambiguous) question with the given options.

Scripting

# Branch in a script via exit codes (-q suppresses printed output)
if decide --context="$body" "Is this message spam?" -q; then
  mv "$file" spam/
fi

# Customize output strings for yes/no
$ decide --context @ticket.txt \
         "Should we issue a refund?" \
         --yes "Hell yeah" \
         --no "Forget it"
Hell yeah

# Branch with custom output strings
if answer=$(decide --context="$body" "Is this message spam?" --yes spam --no ham); then
  mv "$file" "$answer/"
fi

Streaming

# Stream named JSON context, decide for each line on STDIN
cat events.jsonl | decide --context policy=@policy.txt \
                          --context-json event=- \
                          --each \
                          --questions @triage.decide \
                          --json > triage_decisions.jsonl

Each line of standard input is one event, and the - context holds it. Blank lines are skipped. With --json, every event prints one line, so line N of the output answers the Nth non-blank line of the input; an event the model server failed, or a line that is not valid JSON or UTF-8, or too large for the model, prints an error record. Without --json, an event prints the lines a single run prints, and an event with an error prints nothing, so read stderr, which names the input line, and use --json for output a program reads.

JSON output

Full parseable details of a decision can be obtained via JSON output:

$ decide --context @ticket.txt \
         "Which team handles this ticket?" \
         --option shipping \
         --option billing \
         --option returns \
         --json
{"q1":{"kind":"choice","answer":"returns","confidence":0.91,"probabilities":{"shipping":0.06,"billing":0.03,"returns":0.91}}}

$ decide --context ticket=@ticket.txt \
         --context refund_policy=@refund_policy.txt \
         --questions @triage.json \
         --json
{"team":{"kind":"choice","answer":"returns","confidence":0.91,"probabilities":{"shipping":0.06,"billing":0.03,"returns":0.91}},
 "urgency":{"kind":"rating","answer":"somewhat_urgent","score":1.15,"confidence":0.78,"probabilities":{"not_urgent":0.15,"somewhat_urgent":0.55,"urgent":0.3}},
 "refund":{"kind":"verdict","answer":"Yes","verdict":true,"confidence":0.74,"probabilities":{"Yes":0.87,"No":0.13}}}

JSON is output on one line (JSONL-style). The example above is wrapped for readability.

Agent skills

decide is ideal for agents: one command, no server, succinct invocation language and return values, machine-readable output (text or JSON).

See the decide skill in this repo for an example.

Command line configuration

# Model and API key can be specified (or overridden) on the command line for zero-config usage
$ decide --model openrouter:typesafe/jev-1.13 \
         --api-key "$KEY" \
         "Is Atlanta the capital of Georgia?"
yes

Errors

# Handle non-decision errors (loss of network, etc.) using `--fallback`
$ sudo ip link set eth0 down
$ decide --context "$body" \
         "Which team handles this ticket?" \
         --option shipping \
         --option billing \
         --option returns \
         --min-confidence 0.7 \
         --fallback human

stderr>  Error: cannot reach decision model server
human

# A fallback also stands in for an answer below its bar, and the run is decided
$ decide --context @ticket.txt \
         "Which team handles this ticket?" \
         --name team \
         --option shipping \
         --option billing \
         --option returns \
         --min-confidence 0.95 \
         --fallback human \
         "Should we issue a refund?" \
         --name refund
team=human
refund=yes

stderr>  Unsure: question 1 ("Which team handles this ticket?") has confidence 0.91, below the bar of 0.95
$ echo $?
0

# Define a safe path for a script; a yes/no fallback is its yes or no value
if decide --context "$body" "Is this message spam?" -q --fallback no; then
  mv "$file" spam/     # never reached on an error
fi

--fallback on a question names what it prints when its answer is below its --min-confidence bar, or when the run has a remote error. A question that prints its fallback counts as decided. A remote error prints every fallback when every question has one; when any question has none, nothing prints and the run exits 11. A setup error never takes a fallback.

Development

To build the tool or run its tests, see DEVELOPMENT.md and TESTING.md.

Appendix

Exit codes

For a choice, a rating, or a batch, decisions are printed to stdout and the exit code reports errors:

When the run has one yes/no question, the exit code carries the answer as well:

Only 0 and 1 carry an answer. A script that branches on the exit code should put the action on the yes side, or switch on $?.

Exit 2 prints every line, and the unsure ones are empty. Check the code before you read stdout.

--fallback on a question makes an unsure answer a decision, and a remote error too when every question has one. With one yes/no question the fallback is its yes or no value, and the code follows that side.

In a stream, a setup error stops the run at once. 2, 10, and 11 are per event: an unsure event prints its empty answers or its fallbacks; an event the model server failed prints its fallbacks, or an error record with --json; a line that is not valid JSON or UTF-8, or too large for the model, prints an error record with --json and nothing without; stderr names the input line; the stream goes on; and the final code is the highest code any event produced. A decided event counts as 0, so one yes/no question does not answer with the exit code in a stream.