Local, privacy-first engine that reads live financial news and shows which stocks it moves — including non-obvious ripple effects — with calibrated confidence and an event-study track record. Runs on your machine, no API keys.
Python
1
133 commits
updated Aug 16, 2026
A local, privacy-first engine that reads live financial news and shows which stocks it moves — with honest, calibrated confidence and an event-study paper trail.
NewsImpact reads news wires (RSS) and live broadcast channels (OCR), figures out what happened and which companies are affected — including the non-obvious second-order names — and grades every call against real market data. It runs entirely on your machine; no data leaves it, and no API keys are required.
Not investment advice. This is a research and portfolio project. Every metric in this repository is reproducible from the included scripts.
Most "news sentiment" tools wrap a cloud API and imply an accuracy no one can deliver. NewsImpact is built on one principle: be excellent where accuracy is real, and honest where it isn't.
| Layer | What | Real number |
|---|---|---|
| Stance (hawkish/dovish) | our own model, trained on public data | 76.7% accuracy; 86% on confident calls (≥0.85, ~57% coverage) |
| Directional price prediction | physically capped (~55–60% is the state of the art) | shown next to the realized outcome + calibration, never inflated |
| Event-study evidence | real market data (yfinance + statsmodels) | e.g. NVDA post-FOMC abnormal return with t-stat / p-value and caveats |
We do not claim to beat the market. Short-horizon direction is near-random by construction; overclaiming is the fastest way to lose a finance-literate reader. The honesty is the point.
Everything runs locally, key-free, and privacy-first — external APIs are optional enhancements, and the app degrades gracefully without any of them.
| Feature | What it does |
|---|---|
| Power Mode toggle | A header switch scales the engine to your machine (no code/env edits). Laptop (default) is unchanged; Power hears every channel each sweep (GPU if present), reads the full story on every item, walks a deeper exposure graph, and keeps a durable local transcript history to search & backtest. Applied live, all local. Raises coverage/depth — not the honest accuracy ceiling |
| Machine-aware | Detects GPU / CPU cores / RAM and recommends the right mode; degrades gracefully (no GPU → runs on CPU) |
Standard vs. Power at a glance:
| Capability | Standard (default) | ⚡ Power |
|---|---|---|
| Channel audio (ASR) | samples one channel per sweep (opt-in) | hears every channel each sweep |
| Speech device | CPU (int8) | GPU (float16) if present, else CPU |
| Article reading | full story on alerts only | full story on every relevant item |
| Exposure graph | 2-hop | 3-hop, wider |
| Feeds polled / tick | 3 | 8 |
| Transcript history | in-memory (~last 250) | durable, saved & searchable (export CSV) |
| Cross-connect earlier reads | off | on (corroboration / leads that beat the wires) |
| "What Power did" summary | — | live counts (articles, on-air lines, cross-connections) |
| Best for | everyday / low power draw | strong CPU / GPU boxes |
Both modes are fully local; Standard reproduces the original defaults exactly, so switching Power off changes nothing. It raises coverage & depth, not the honest accuracy ceiling.
| Feature | What it does |
|---|---|
| Multi-source ingestion | Broadcaster RSS, SEC EDGAR 8-K (primary filings), Google News (per-ticker/topic), GDELT (global), live TV via OCR, and optional audio via ASR |
| Push-first detection | Conditional-GET polling (HTTP 304) + WebSub support — cheap enough to poll often, ~0s where push is available |
| UCB1 bandit scheduler | Attention flows to sources actually producing news; quiet ones stay sampled (provable regret bound) |
| Relevance gate + dedup | One-pass Aho-Corasick gate (~0.05 ms) + SimHash near-duplicate detection (turns dups into corroboration) |
| Tier-3 depth | On escalation only: async article-body fetch (reads the whole story, not the headline) + optional local-LLM reasoning |
| Feature | What it does |
|---|---|
| Entity linking + disambiguation | "Apple" the company vs. "apple pie" — resolves tickers using context |
| Typed event taxonomy | ~28 market-moving event types (guidance cut, recall, rate hike, M&A…) with direction + severity |
| Stance & sentiment | Own-trained hawkish/dovish model (76.7% acc) + FinBERT + VADER ensemble |
| Feature | What it does |
|---|---|
| Ripple graph | Market-residual correlation surfaces suppliers/competitors/correlated names filings never mention |
| Causal graph (transfer entropy) | A directed, nonlinear "who-leads-whom" graph — X's past genuinely precedes Y's next move — beyond what correlation can see. Shown as directed links in the evidence graph |
| Co-mention graph | Learns live which companies the news keeps pairing — a compounding, non-back-fillable asset |
| Event cascades (Hawkes) | A self-exciting point process flags when a cluster of related alerts is building (vs a one-off) and how long impact persists |
| Learned propagation (OOS-validated) | A small neural net gives a second opinion on second-order names — but only after it beat the correlation baseline in walk-forward, embargoed validation; otherwise it stays disabled (honest by construction) |
| Lead-lag + abnormal volume | Flags names that front-run the tape or show unusual positioning |
| Dempster-Shafer fusion | Combines reliability-weighted evidence into a conviction and an uncertainty |
| Feature | What it does |
|---|---|
| Event study | Every "news → stock" claim backed by abnormal return, t-stat, and p-value on real prices |
| Live track record | Every call logged before the outcome, scored against real prices, with calibration — losses shown |
| Selective prediction | A conviction line splits committed calls from "watching" (low-conviction); accuracy is reported on the committed subset — say less, be right more often |
| Self-calibrating confidence | "70%" is mapped to our own empirical hit-rate (shrunk + monotonic, learned from resolved calls) — it gets truer over time |
| Edge-conditioning | Confidence is down-weighted on mega-caps/ETFs (news priced in fast) and up-weighted where post-news drift survives (smaller, higher-volatility names) |
| Multi-horizon scoring | Every call graded at 1 / 2 / 5 / 20 trading days — the honest truth shown: near-random at 1-day, real drift at 5–20 days |
| Follow / fade | Fresh, surprising news tends to keep drifting (follow); stale, already-priced-in news tends to mean-revert (fade) — tagged on each alert |
| Optional AI read | Add your own free key (Groq / Gemini) or local Ollama in the app — stored in a private, git-ignored local file (never uploaded) and verified with a live check (the button lights up only for a real, working key). Each alert then gets an advisory LLM direction folded into the fusion; off by default, never changes the logged track record |
| Earnings surprise | On earnings/filing alerts, the latest beat / miss vs. the analyst estimate (reported EPS vs consensus) — the surprise that actually drives drift, not just sentiment |
| Abstention | Says "uncertain" instead of guessing when the evidence is weak |
| Feature | What it does |
|---|---|
| Event-driven push (SSE) | UI updates the instant new data lands; direction-aware price flashing |
| War-room UI | Video wall + impact panel + signal scanner + live signals + proof panel + golden-path demo |
| Real-time prices | Finnhub WebSocket when a key is set, end-of-day fallback otherwise |
| Bounded storage | Runtime data auto-capped with configurable retention; disk-usage transparency + one-call reset |
| Graceful degradation | Missing key/model/feed never breaks the app — it downgrades and keeps working |
| Feature | What it does |
|---|---|
| Two-tier portfolio | Track what you own and what you're watching to buy (local, private). Every alert is tagged owned / watching and — via a multi-hop exposure graph — owned-related / watching-related: the indirect, graph-connected hits competitors ignore (your airline when oil spikes), each with the path back to your holding |
| Special attention to your names | The reading layer adds per-ticker feeds for your book, polls them harder, and auto-escalates any story touching your holdings to a full read of the article |
| Impact magnitude | Not just which way but how far: an honest move-size band (e.g. ~1–3%, notable) grounded in the stock's own volatility × event severity × link strength × novelty — a scale, never a promise |
| Show me the sources ("connect the dots") | Every "news → your stock" claim expands into an inspectable evidence graph: the real headline + clickable source links (primary + corroborating outlets) → typed event → theme move → the statistical link (corr, p) → your stock. Verify it yourself |
| Closed-loop daily ritual | A morning brief ("what's moving your holdings") + a scorecard ("recent calls: 4/6 correct") + a daily ledger with a one-click "grade yesterday's calls" — the honest feedback loop no incumbent offers |
Open a source-backed dossier for any ticker from the header (🔎 Dossier; opt-in, off by default, fully local).
| Feature | What it does |
|---|---|
| Company Dossier | One page per stock: an outlook, a catalyst calendar (earnings, ex-dividend, index dates, IPO lock-ups) with an elevated-risk-window banner, connections (suppliers/peers, so indirect news reaches you), fundamentals, market structure (short interest, insider activity, volume anomalies), news sentiment (FinBERT over recent headlines), red flags, and filings — every fact cited with its source and as-of date |
| Honest outlook | Synthesizes the dossier's own evidence into a lean with a confidence tier and a "Why?" attribution; it abstains when evidence is thin or conflicting, and never gives a price target |
| The Desk | Add what you know that the app can't fetch → a weighted, reversible update, capped so a hunch can't dominate, kept separate from the hard data, telling you if the read hinges on you being right. Suspected insider (non-public) info is flagged and treated as a hypothesis, never a trade signal |
| Feature | What it does |
|---|---|
| Developer surface | Stable signal schema (/schema), an endpoint catalog (/api) + OpenAPI (/docs), and drop-in sources — add a feed or a country in a few lines via custom_feeds.json, NEWSIMPACT_COUNTRY, or register_feed() |
| Signal export / backtest | Export every decided call + its real outcome as CSV/JSON (/signals/export) to validate the engine in your own backtests |
live RSS wires + broadcast channels (OCR)
│ push-first ingest (conditional GET / adaptive polling / scene-change gate)
▼
fast relevance gate (Aho-Corasick) → near-duplicate dedup (SimHash)
▼
entity linking + disambiguation → typed event taxonomy
▼
grounding: ripple graph + co-mention graph + historical correlation
▼
Dempster-Shafer conviction fusion (belief + uncertainty)
▼
event study + calibrated prediction + ABSTAIN
▼
FastAPI ──(Server-Sent Events, push-on-change)──▶ Next.js war-room UI
The frontend always works on a seeded golden path (zero dependencies); when the API is up it serves live data and falls back silently if not — so the UI is never broken.
Golden path — press play to stream a real FOMC press conference; the impact panel reacts with affected tickers, calibrated confidence, and event-study numbers:
Live Signals — the adaptive watcher reads wires and channels in real time; typed event chips (earnings beat, guidance raise…), a market read, and importance-ranked alerts:
Proof — every measured, checkable number: stance accuracy, meta-labeling, an NVDA event study, the ripple graph, and the live track record with calibration:
| Area | Choices | Why |
|---|---|---|
| Frontend | Next.js (App Router), TypeScript, Tailwind, Zustand | Fast, typed, small state footprint; SSR-ready |
| Backend | Python, FastAPI, Uvicorn | Async, first-class ML ecosystem, tiny surface area |
| ML / NLP | PyTorch + Transformers, sentence-transformers (MiniLM), FinBERT, VADER | Local inference, no API keys; finance-tuned models |
| Quant | yfinance, NumPy, SciPy, statsmodels | Real market data + event-study statistics |
| Ingest | feedparser (RSS), yt-dlp + PyAV + RapidOCR (channels) | Broad, free sources; broadcast text via OCR |
| Realtime | Server-Sent Events; optional Finnhub WebSocket | Push-on-change UI; live prices when a key is present |
Design decisions favor local-first, keyless-by-default, and graceful degradation: every external dependency is optional and the app stays useful without it.
backend/
newsimpact/ core engine (importable package)
api.py FastAPI app: endpoints + SSE stream + background watcher
ingest.py push-first feed polling (conditional GET, adaptive cadence, WebSub)
fastgate.py Aho-Corasick relevance gate + SimHash near-dup detection
linker.py entity linking + ambiguity disambiguation
events.py typed event taxonomy (guidance cut, recall, rate hike, …)
connect.py text → affected stocks (first- and second-order, grounded)
ripple.py market-residual correlation ("ripple") graph
comention.py live-learned news co-mention relationship graph
fusion.py Dempster-Shafer conviction + uncertainty fusion
event_study.py abnormal return / CAR / t-stat / p-value
track_record.py live prediction log + outcome scoring + calibration
watcher.py adaptive calm/alert news watcher
stance.py / sentiment*.py / themes.py / embeddings.py NLP layers
... (see files for the rest)
scripts/ build_*, train_*, verify_*, and demo scripts
data/ precomputed caches (universe, returns, profiles, graphs)
requirements*.txt
web/
app/ Next.js routes, layout, design tokens
components/ war-room UI (video wall, impact panel, signals, proof, charts)
lib/ API clients, SSE stream, store, types
README.md this file
RUNNING.md day-to-day launch reference
LICENSE MIT
cd web
npm install
npm run dev # http://localhost:3000
Runs entirely on the seeded golden path — the war-room, golden-path playback, and Signal Scanner examples all work with no backend.
# 1) Backend
cd backend
python -m venv .venv
.venv/Scripts/python -m pip install -r requirements.txt
.venv/Scripts/python -m pip install -r requirements-models.txt # torch + transformers
# 2) Build data + the stance model (weights are not committed — see note below)
.venv/Scripts/python -m scripts.build_universe
.venv/Scripts/python -m scripts.train_stance_binary
.venv/Scripts/python -m scripts.generate_golden_path
# optional advanced grounding (both git-ignored artifacts; the app degrades gracefully without them):
.venv/Scripts/python -m scripts.build_causal_graph # directed transfer-entropy graph
.venv/Scripts/python -m scripts.train_propagation # learned propagation (auto-disables if it fails OOS validation)
# 3) Run the API
.venv/Scripts/python -m uvicorn newsimpact.api:app --port 8000
Point the frontend at it:
# web/.env.local
NEXT_PUBLIC_API_BASE_URL=http://localhost:8000
Then cd web && npm run dev. See RUNNING.md for the day-to-day commands.
Model weights are not in the repo. The fine-tuned stance models are ~300 MB each, above GitHub's file limit, so they're git-ignored and regenerated by
scripts.train_stance_binary(uses the public Trillion Dollar Words dataset). The app degrades gracefully without them.
cd backend
.venv/Scripts/python -m scripts.verify_all # exercises every module end-to-end
Built: war-room UI + design system; push-first multi-source ingest (RSS/EDGAR/Google News/GDELT/OCR/ASR); Aho-Corasick gate + SimHash dedup; entity linking with disambiguation; typed event taxonomy; ripple + co-mention graphs; Dempster-Shafer fusion; event study; adaptive watcher; FastAPI + SSE.
Prediction quality: selective prediction (commit vs. abstain) with a committed-only scoreboard; self-calibrating confidence; edge-conditioning; multi-horizon scoring (1/2/5/20d)
Personal: two-tier portfolio (owned + watching) with multi-hop exposure paths; impact magnitude bands; inspectable evidence graph with clickable sources; closed-loop daily brief + ledger ("grade yesterday's calls").
Power Mode: one in-app switch scaling laptop → full-power (all-channel GPU ASR, full-story reads, deeper graph, durable local transcript history).
Company Dossier + The Desk: a source-backed, per-stock research file (outlook, catalyst calendar, connections, fundamentals, market structure, news sentiment, red flags, filings) plus human-in-the-loop reasoning that folds in what you know — opt-in, fully local.
Exploring: using the transcript history to cross-connect breaking stories with what was heard/read earlier; a visible "what Power Mode did for you" summary; adaptive CPU/RAM throttling; broader country source packs.
Built as a portfolio and research project. Not investment advice.
Local, privacy-first engine that reads live financial news and shows which stocks it moves — including non-obvious ripple effects — with calibrated confidence and an event-study track record. Runs on your machine, no API keys.
Python
1
133 commits
updated Aug 16, 2026
A local, privacy-first engine that reads live financial news and shows which stocks it moves — with honest, calibrated confidence and an event-study paper trail.
NewsImpact reads news wires (RSS) and live broadcast channels (OCR), figures out what happened and which companies are affected — including the non-obvious second-order names — and grades every call against real market data. It runs entirely on your machine; no data leaves it, and no API keys are required.
Not investment advice. This is a research and portfolio project. Every metric in this repository is reproducible from the included scripts.
Most "news sentiment" tools wrap a cloud API and imply an accuracy no one can deliver. NewsImpact is built on one principle: be excellent where accuracy is real, and honest where it isn't.
| Layer | What | Real number |
|---|---|---|
| Stance (hawkish/dovish) | our own model, trained on public data | 76.7% accuracy; 86% on confident calls (≥0.85, ~57% coverage) |
| Directional price prediction | physically capped (~55–60% is the state of the art) | shown next to the realized outcome + calibration, never inflated |
| Event-study evidence | real market data (yfinance + statsmodels) | e.g. NVDA post-FOMC abnormal return with t-stat / p-value and caveats |
We do not claim to beat the market. Short-horizon direction is near-random by construction; overclaiming is the fastest way to lose a finance-literate reader. The honesty is the point.
Everything runs locally, key-free, and privacy-first — external APIs are optional enhancements, and the app degrades gracefully without any of them.
| Feature | What it does |
|---|---|
| Power Mode toggle | A header switch scales the engine to your machine (no code/env edits). Laptop (default) is unchanged; Power hears every channel each sweep (GPU if present), reads the full story on every item, walks a deeper exposure graph, and keeps a durable local transcript history to search & backtest. Applied live, all local. Raises coverage/depth — not the honest accuracy ceiling |
| Machine-aware | Detects GPU / CPU cores / RAM and recommends the right mode; degrades gracefully (no GPU → runs on CPU) |
Standard vs. Power at a glance:
| Capability | Standard (default) | ⚡ Power |
|---|---|---|
| Channel audio (ASR) | samples one channel per sweep (opt-in) | hears every channel each sweep |
| Speech device | CPU (int8) | GPU (float16) if present, else CPU |
| Article reading | full story on alerts only | full story on every relevant item |
| Exposure graph | 2-hop | 3-hop, wider |
| Feeds polled / tick | 3 | 8 |
| Transcript history | in-memory (~last 250) | durable, saved & searchable (export CSV) |
| Cross-connect earlier reads | off | on (corroboration / leads that beat the wires) |
| "What Power did" summary | — | live counts (articles, on-air lines, cross-connections) |
| Best for | everyday / low power draw | strong CPU / GPU boxes |
Both modes are fully local; Standard reproduces the original defaults exactly, so switching Power off changes nothing. It raises coverage & depth, not the honest accuracy ceiling.
| Feature | What it does |
|---|---|
| Multi-source ingestion | Broadcaster RSS, SEC EDGAR 8-K (primary filings), Google News (per-ticker/topic), GDELT (global), live TV via OCR, and optional audio via ASR |
| Push-first detection | Conditional-GET polling (HTTP 304) + WebSub support — cheap enough to poll often, ~0s where push is available |
| UCB1 bandit scheduler | Attention flows to sources actually producing news; quiet ones stay sampled (provable regret bound) |
| Relevance gate + dedup | One-pass Aho-Corasick gate (~0.05 ms) + SimHash near-duplicate detection (turns dups into corroboration) |
| Tier-3 depth | On escalation only: async article-body fetch (reads the whole story, not the headline) + optional local-LLM reasoning |
| Feature | What it does |
|---|---|
| Entity linking + disambiguation | "Apple" the company vs. "apple pie" — resolves tickers using context |
| Typed event taxonomy | ~28 market-moving event types (guidance cut, recall, rate hike, M&A…) with direction + severity |
| Stance & sentiment | Own-trained hawkish/dovish model (76.7% acc) + FinBERT + VADER ensemble |
| Feature | What it does |
|---|---|
| Ripple graph | Market-residual correlation surfaces suppliers/competitors/correlated names filings never mention |
| Causal graph (transfer entropy) | A directed, nonlinear "who-leads-whom" graph — X's past genuinely precedes Y's next move — beyond what correlation can see. Shown as directed links in the evidence graph |
| Co-mention graph | Learns live which companies the news keeps pairing — a compounding, non-back-fillable asset |
| Event cascades (Hawkes) | A self-exciting point process flags when a cluster of related alerts is building (vs a one-off) and how long impact persists |
| Learned propagation (OOS-validated) | A small neural net gives a second opinion on second-order names — but only after it beat the correlation baseline in walk-forward, embargoed validation; otherwise it stays disabled (honest by construction) |
| Lead-lag + abnormal volume | Flags names that front-run the tape or show unusual positioning |
| Dempster-Shafer fusion | Combines reliability-weighted evidence into a conviction and an uncertainty |
| Feature | What it does |
|---|---|
| Event study | Every "news → stock" claim backed by abnormal return, t-stat, and p-value on real prices |
| Live track record | Every call logged before the outcome, scored against real prices, with calibration — losses shown |
| Selective prediction | A conviction line splits committed calls from "watching" (low-conviction); accuracy is reported on the committed subset — say less, be right more often |
| Self-calibrating confidence | "70%" is mapped to our own empirical hit-rate (shrunk + monotonic, learned from resolved calls) — it gets truer over time |
| Edge-conditioning | Confidence is down-weighted on mega-caps/ETFs (news priced in fast) and up-weighted where post-news drift survives (smaller, higher-volatility names) |
| Multi-horizon scoring | Every call graded at 1 / 2 / 5 / 20 trading days — the honest truth shown: near-random at 1-day, real drift at 5–20 days |
| Follow / fade | Fresh, surprising news tends to keep drifting (follow); stale, already-priced-in news tends to mean-revert (fade) — tagged on each alert |
| Optional AI read | Add your own free key (Groq / Gemini) or local Ollama in the app — stored in a private, git-ignored local file (never uploaded) and verified with a live check (the button lights up only for a real, working key). Each alert then gets an advisory LLM direction folded into the fusion; off by default, never changes the logged track record |
| Earnings surprise | On earnings/filing alerts, the latest beat / miss vs. the analyst estimate (reported EPS vs consensus) — the surprise that actually drives drift, not just sentiment |
| Abstention | Says "uncertain" instead of guessing when the evidence is weak |
| Feature | What it does |
|---|---|
| Event-driven push (SSE) | UI updates the instant new data lands; direction-aware price flashing |
| War-room UI | Video wall + impact panel + signal scanner + live signals + proof panel + golden-path demo |
| Real-time prices | Finnhub WebSocket when a key is set, end-of-day fallback otherwise |
| Bounded storage | Runtime data auto-capped with configurable retention; disk-usage transparency + one-call reset |
| Graceful degradation | Missing key/model/feed never breaks the app — it downgrades and keeps working |
| Feature | What it does |
|---|---|
| Two-tier portfolio | Track what you own and what you're watching to buy (local, private). Every alert is tagged owned / watching and — via a multi-hop exposure graph — owned-related / watching-related: the indirect, graph-connected hits competitors ignore (your airline when oil spikes), each with the path back to your holding |
| Special attention to your names | The reading layer adds per-ticker feeds for your book, polls them harder, and auto-escalates any story touching your holdings to a full read of the article |
| Impact magnitude | Not just which way but how far: an honest move-size band (e.g. ~1–3%, notable) grounded in the stock's own volatility × event severity × link strength × novelty — a scale, never a promise |
| Show me the sources ("connect the dots") | Every "news → your stock" claim expands into an inspectable evidence graph: the real headline + clickable source links (primary + corroborating outlets) → typed event → theme move → the statistical link (corr, p) → your stock. Verify it yourself |
| Closed-loop daily ritual | A morning brief ("what's moving your holdings") + a scorecard ("recent calls: 4/6 correct") + a daily ledger with a one-click "grade yesterday's calls" — the honest feedback loop no incumbent offers |
Open a source-backed dossier for any ticker from the header (🔎 Dossier; opt-in, off by default, fully local).
| Feature | What it does |
|---|---|
| Company Dossier | One page per stock: an outlook, a catalyst calendar (earnings, ex-dividend, index dates, IPO lock-ups) with an elevated-risk-window banner, connections (suppliers/peers, so indirect news reaches you), fundamentals, market structure (short interest, insider activity, volume anomalies), news sentiment (FinBERT over recent headlines), red flags, and filings — every fact cited with its source and as-of date |
| Honest outlook | Synthesizes the dossier's own evidence into a lean with a confidence tier and a "Why?" attribution; it abstains when evidence is thin or conflicting, and never gives a price target |
| The Desk | Add what you know that the app can't fetch → a weighted, reversible update, capped so a hunch can't dominate, kept separate from the hard data, telling you if the read hinges on you being right. Suspected insider (non-public) info is flagged and treated as a hypothesis, never a trade signal |
| Feature | What it does |
|---|---|
| Developer surface | Stable signal schema (/schema), an endpoint catalog (/api) + OpenAPI (/docs), and drop-in sources — add a feed or a country in a few lines via custom_feeds.json, NEWSIMPACT_COUNTRY, or register_feed() |
| Signal export / backtest | Export every decided call + its real outcome as CSV/JSON (/signals/export) to validate the engine in your own backtests |
live RSS wires + broadcast channels (OCR)
│ push-first ingest (conditional GET / adaptive polling / scene-change gate)
▼
fast relevance gate (Aho-Corasick) → near-duplicate dedup (SimHash)
▼
entity linking + disambiguation → typed event taxonomy
▼
grounding: ripple graph + co-mention graph + historical correlation
▼
Dempster-Shafer conviction fusion (belief + uncertainty)
▼
event study + calibrated prediction + ABSTAIN
▼
FastAPI ──(Server-Sent Events, push-on-change)──▶ Next.js war-room UI
The frontend always works on a seeded golden path (zero dependencies); when the API is up it serves live data and falls back silently if not — so the UI is never broken.
Golden path — press play to stream a real FOMC press conference; the impact panel reacts with affected tickers, calibrated confidence, and event-study numbers:
Live Signals — the adaptive watcher reads wires and channels in real time; typed event chips (earnings beat, guidance raise…), a market read, and importance-ranked alerts:
Proof — every measured, checkable number: stance accuracy, meta-labeling, an NVDA event study, the ripple graph, and the live track record with calibration:
| Area | Choices | Why |
|---|---|---|
| Frontend | Next.js (App Router), TypeScript, Tailwind, Zustand | Fast, typed, small state footprint; SSR-ready |
| Backend | Python, FastAPI, Uvicorn | Async, first-class ML ecosystem, tiny surface area |
| ML / NLP | PyTorch + Transformers, sentence-transformers (MiniLM), FinBERT, VADER | Local inference, no API keys; finance-tuned models |
| Quant | yfinance, NumPy, SciPy, statsmodels | Real market data + event-study statistics |
| Ingest | feedparser (RSS), yt-dlp + PyAV + RapidOCR (channels) | Broad, free sources; broadcast text via OCR |
| Realtime | Server-Sent Events; optional Finnhub WebSocket | Push-on-change UI; live prices when a key is present |
Design decisions favor local-first, keyless-by-default, and graceful degradation: every external dependency is optional and the app stays useful without it.
backend/
newsimpact/ core engine (importable package)
api.py FastAPI app: endpoints + SSE stream + background watcher
ingest.py push-first feed polling (conditional GET, adaptive cadence, WebSub)
fastgate.py Aho-Corasick relevance gate + SimHash near-dup detection
linker.py entity linking + ambiguity disambiguation
events.py typed event taxonomy (guidance cut, recall, rate hike, …)
connect.py text → affected stocks (first- and second-order, grounded)
ripple.py market-residual correlation ("ripple") graph
comention.py live-learned news co-mention relationship graph
fusion.py Dempster-Shafer conviction + uncertainty fusion
event_study.py abnormal return / CAR / t-stat / p-value
track_record.py live prediction log + outcome scoring + calibration
watcher.py adaptive calm/alert news watcher
stance.py / sentiment*.py / themes.py / embeddings.py NLP layers
... (see files for the rest)
scripts/ build_*, train_*, verify_*, and demo scripts
data/ precomputed caches (universe, returns, profiles, graphs)
requirements*.txt
web/
app/ Next.js routes, layout, design tokens
components/ war-room UI (video wall, impact panel, signals, proof, charts)
lib/ API clients, SSE stream, store, types
README.md this file
RUNNING.md day-to-day launch reference
LICENSE MIT
cd web
npm install
npm run dev # http://localhost:3000
Runs entirely on the seeded golden path — the war-room, golden-path playback, and Signal Scanner examples all work with no backend.
# 1) Backend
cd backend
python -m venv .venv
.venv/Scripts/python -m pip install -r requirements.txt
.venv/Scripts/python -m pip install -r requirements-models.txt # torch + transformers
# 2) Build data + the stance model (weights are not committed — see note below)
.venv/Scripts/python -m scripts.build_universe
.venv/Scripts/python -m scripts.train_stance_binary
.venv/Scripts/python -m scripts.generate_golden_path
# optional advanced grounding (both git-ignored artifacts; the app degrades gracefully without them):
.venv/Scripts/python -m scripts.build_causal_graph # directed transfer-entropy graph
.venv/Scripts/python -m scripts.train_propagation # learned propagation (auto-disables if it fails OOS validation)
# 3) Run the API
.venv/Scripts/python -m uvicorn newsimpact.api:app --port 8000
Point the frontend at it:
# web/.env.local
NEXT_PUBLIC_API_BASE_URL=http://localhost:8000
Then cd web && npm run dev. See RUNNING.md for the day-to-day commands.
Model weights are not in the repo. The fine-tuned stance models are ~300 MB each, above GitHub's file limit, so they're git-ignored and regenerated by
scripts.train_stance_binary(uses the public Trillion Dollar Words dataset). The app degrades gracefully without them.
cd backend
.venv/Scripts/python -m scripts.verify_all # exercises every module end-to-end
Built: war-room UI + design system; push-first multi-source ingest (RSS/EDGAR/Google News/GDELT/OCR/ASR); Aho-Corasick gate + SimHash dedup; entity linking with disambiguation; typed event taxonomy; ripple + co-mention graphs; Dempster-Shafer fusion; event study; adaptive watcher; FastAPI + SSE.
Prediction quality: selective prediction (commit vs. abstain) with a committed-only scoreboard; self-calibrating confidence; edge-conditioning; multi-horizon scoring (1/2/5/20d)
Personal: two-tier portfolio (owned + watching) with multi-hop exposure paths; impact magnitude bands; inspectable evidence graph with clickable sources; closed-loop daily brief + ledger ("grade yesterday's calls").
Power Mode: one in-app switch scaling laptop → full-power (all-channel GPU ASR, full-story reads, deeper graph, durable local transcript history).
Company Dossier + The Desk: a source-backed, per-stock research file (outlook, catalyst calendar, connections, fundamentals, market structure, news sentiment, red flags, filings) plus human-in-the-loop reasoning that folds in what you know — opt-in, fully local.
Exploring: using the transcript history to cross-connect breaking stories with what was heard/read earlier; a visible "what Power Mode did for you" summary; adaptive CPU/RAM throttling; broader country source packs.
Built as a portfolio and research project. Not investment advice.