DFIP

Decision-focused inference on Modal — a research preview. DFIP provides a small Python client, pinned model adapters, and reproducible experiments for System One models (also known as decision models). The client runs locally; weights and GPU execution stay in your Modal account.

All client deployments use the released native inference paths. CUDA graph replay, Triton candidate projection, and shared batching are separate research paths. The experiments show useful gains on some resident workloads, but do not establish a general acceleration method or production readiness.

Install and run

Requires Python 3.10+, a Modal account with GPU access, and access to the selected Hugging Face checkpoint. Install from this repository:

git clone https://github.com/superagent-ai/dfip.git
cd dfip
python -m venv .venv
source .venv/bin/activate
python -m pip install -e .
modal setup

# Choose names that do not replace an existing deployment or reuse its cache.
export DFIP_APP=dfip-preview
export DFIP_VOLUME=dfip-preview-models
dfip setup --model liquid
dfip predict --model liquid --input examples/request.json

setup downloads only the selected pinned checkpoint into a Modal Volume, then deploys authenticated Modal methods. The quick start uses an L40S. Preparing and calling models incurs Modal usage; containers scale to zero after 60 seconds of inactivity, and the first call includes model loading. No public HTTP endpoint is created. Keep the environment variables set for subsequent commands.

For a gated checkpoint, accept its Hugging Face terms and create a Modal secret named dfip-hf containing HF_TOKEN. Set DFIP_HF_SECRET=dfip-hf before running dfip prepare --model <alias>. Credentials belong in that secret, not in request files. Preparation injects the secret into the download worker.

Python client

from dfip import Client

client = Client("liquid")
result = client.predict([
    {
        "state": "I was charged twice. Please refund the duplicate payment.",
        "questions": {
            "team": {
                "type": "choice",
                "instructions": "Which team should handle this request?",
                "criteria": {
                    "billing": "Payments and refunds",
                    "technical": "Software bugs",
                },
            },
        },
    },
])
print(result[0]["answers"]["team"]["choice"])

await client.predict_async(records) supports asynchronous calls. client.diagnostics() reports the remote execution policy. Text requests support choice, noul (yes/no), and score questions. Outputs retain each released model's answers format; extra fields differ between models. See the request example for the input format.

Client models

dfip models prints immutable checkpoint revisions. Aliases and the listed Hugging Face IDs are accepted. Prepare each model before calling it; downloading all models is unnecessary for the quick start. Arbitrary Hugging Face models are not supported automatically. GLiNER2.5-Decide is a research adapter and is not available through this client registry.

Research results

The paper and analysis study PPLX-Decider, Liquid d1-3B, Clef-Flash, and GLiNER2.5-Decide. The manuscript is a review draft and has not been submitted to arXiv.

The bracketed ranges are paired 95% bootstrap confidence intervals where shown. These phases are not pooled. Resident timings exclude client transport and cold starts. Changing external traffic reverses the original Decider/Clef replay advantage through recapture, queues, drops, and a retained CUDA failure. Clef also changes probabilities and some confidence-threshold outcomes. GLiNER's compiled shared scheduler improves resident completion time, but has worse p95 latency than compiled native batching in its short arrival replay. These limits are part of the evidence, not resolved serving guarantees.

Regenerate the four-model tables from the archived results without a GPU:

python -m pip install -e '.[paper]'
PYTHONPATH=src python scripts/analyze_focused.py

Reproducibility instructions and the benchmark archive link the protocols, raw records, failed runs, source snapshots, and Modal reproduction commands. Run new GPU experiments one GPU application at a time, in separate research apps and Volumes. Earlier studies remain in the archive for provenance and do not contribute to the focused paper's claims. The opt-in adaptive serving pilot is documented separately in adaptive serving.

Configuration and checks

Set these consistently for preparation, deployment, evaluation, and inference. Client("liquid", app_name="my-dfip", environment_name="staging") selects a deployment explicitly. Modal's environment selection applies to deployment.

After checkpoint preparation, the following checks run on Modal and incur usage:

dfip smoke --model liquid --output liquid-smoke.json
dfip validate --model liquid --output liquid-validation.json
dfip benchmark --model liquid --quality 400 --repetitions 100 --output liquid-benchmark.json

smoke and benchmark support all five client models; validate supports the four in-process adapters. These commands check the packaged default path. Experimental comparisons use the separate research entry points.

For local development:

python -m pip install -e '.[dev]'
pytest
ruff check src tests

CI checks the CPU client and packaging on Python 3.10 and 3.12. GPU studies run explicitly on Modal. See contributing, support, security policy, and release notes. Citation metadata is in CITATION.cff.

DFIP source is MIT licensed. Model weights and downloaded datasets are not bundled. The research archive includes pinned upstream source snapshots under their own terms; see third-party notices. Model licenses apply independently, including Liquid's commercial-use conditions. Pinned upstream custom Python code executes remotely where required by a model.