forecast_eval — documentation hub

A trading-evaluation harness for futures forecasting models, targeting TXF (Taiwan Stock Exchange Futures) with toto2 as the reference forecaster.

What it answers: given a forecast model, how much PnL is realizable after realistic execution costs, and how much of the model's predictive edge survives the trader and execution layers to the final result?

This folder is the fast-orientation layer: one page per Python file, each telling you what the file owns and which file to open next. For the full design rationale see SPEC.md; for how to run things see usage.md; for environment/interpreter notes see CLAUDE.md.

---

Architecture at a glance

flowchart LR
    Ticks[Tick stream] --> Env[Environment]
    Env -- MarketEvent --> Exec[Execution]
    Exec -- FillEvent --> Port[Portfolio]
    Model[Forecaster<br/>toto2 / naive] -- Forecast --> Emit[SignalEmitter]
    Emit -- BUY/SELL/HOLD --> Trader
    Trader -- OrderEvent --> Exec
    Exit[ExitRule<br/>optional] --> Trader
    Port --> Metrics[MetricsReport]
    Port --> Logger[logging_io]
    Logger --> Report[reports.py]

    classDef opt fill:#f6f6f6,stroke:#bbb,stroke-dasharray:4 3;
    class Exit,Model opt;

Separation of concerns: the forecasting model, the signal logic, the trader, and the execution simulator are independent layers — each swappable behind an abstract interface, so any one can be evaluated in isolation.

The per-tick event loop

Every MarketEvent flows through these five steps in run_backtest (SPEC §4.1):

flowchart TD
    T[next MarketEvent at tick t] --> S1
    S1[1. Check fills<br/>resolve pending limit vs this print] --> S2
    S2[2. Check ExitRule<br/>if position open and rule fires, submit close] --> S3
    S3{3. On forecast boundary<br/>and past warm-up?}
    S3 -- yes --> F[Forecaster.forecast -> SignalEmitter.emit -> Trader.on_signal]
    S3 -- no --> S4
    F --> S4
    S4[4. Session-boundary forced close<br/>if enabled and session changed] --> S5
    S5[5. Record fills / signals / forecasts] --> T

Three invariants make this honest (all enforced in code + tests):

• Look-ahead defense — the forecaster is handed a freshly-built frame of history through t only; a runtime guard rejects any forecast stamped past t.
• Same-tick fill guard — an order placed at tick t never fills against t's own print.
• No-flip rule — a SELL while long closes only (never flips to short); same for BUY while short. No pyramiding in v1.

---

File map — open this when…

__init__.py files (forecast_eval/__init__.py, and the package __init__.py under data/, forecaster/, strategy/, exits/, tests/) are empty or near-empty package markers — nothing to document.

---

Recommended reading order

For a newcomer grasping the codebase from scratch:

1. events.py — the vocabulary (MarketEvent, Forecast, Order, Fill).
2. environment.py — where ticks come from.
3. execution.py — the fill rules (the trickiest, most-tested logic).
4. portfolio.py + trader.py — accounting + the state machine.
5. run.py — how steps 1–4 are wired into the per-tick loop.
6. forecaster/base.py → strategy/base.py → exits/base.py — the three pluggable contracts.
7. metrics.py — how a finished run is scored.
8. The concrete implementations (forecaster/toto2.py, strategy/threshold.py) and entry points (real_data_demo.py, compare_models.py).

Entry points (runnable)