sha2nkt/delay_bahn

Python

125

298 commits

updated Sep 21, 2026

See the code

README

DelayBahn

Live at delaybahn.com Python 3.12+ FastAPI DuckDB

A bahn.de-style train connection search that shows the median arrival delay of the last 7 days (window selectable up to 30) for every connection, so you can book the one that actually runs on time. "Booking" deep-links to the real bahn.de page pre-filled with the journey.

Features

  • Delay statistics per leg: median arrival delay over the last 7/15/30 days, per-day charts, and the official IRIS delay causes behind every badge.
  • Four countries: Germany, Austria, Switzerland, and France — cross-border journeys get statistics on every leg.
  • Transfer-risk warnings: connections the arriving train's delay history makes tight or unlikely are flagged, and journeys can be sorted by connection risk.
  • Five sort modes: departure time, least delay, cheapest price, lowest connection risk, fewest transfers.
  • Compensation checker: reconstructs a past journey as it actually ran — exact delays, missed or cancelled connections, the replacement trains a passenger would have taken — and computes DB Fahrgastrechte compensation (25 %/50 %) with a deep link into the bahn.de claim flow.
  • Same-day results: a journey is checkable minutes after arrival via live IRIS lookups, showing the same value the nightly build will store.
  • DE/EN interface, shareable search URLs, station autocomplete served from the local delay data.

How it works

  • Journey search: the bahn.de web API (www.bahn.de/web/api) provides journey options including transfers and prices — the same API the bahn.de website uses. Station autocomplete is answered from the local delay data, falling back to that API for stations without delay history.
  • Historical delays, Germany: the public HuggingFace dataset piebro/deutsche-bahn-data publishes raw Deutsche Bahn IRIS timetable responses every 6 hours. The pipeline keeps a rolling 31-day mirror and builds a per-stop delay table covering all German stations.
  • Historical delays, Switzerland, France, Austria, and the Netherlands: official istdaten daily files (opentransportdata.swiss), 24/7 pollers on the official SNCF GTFS-RT feed and OVapi's Dutch train GTFS-RT feed, and a 24/7 poller sweeping ÖBB HAFAS (Scotty) station boards for the ~200 busiest Austrian stations produce per-day tables in the same schema; NL history is seeded from the Rijden de Treinen monthly train archive; pipeline/merge_delays.py unions all five countries into the served table.
  • Today's delays: for a journey the nightly pipeline has not ingested yet, delays are read at request time from the DB Timetables API (IRIS plan + fchg) — the same field the pipeline stores, so the answer does not change later. Optional: set DB_API_KEY/DB_CLIENT_ID (e.g. in a .env); without them the site simply stops at the last ingested day.
  • Matching: each train leg of a journey is matched against history by train number + arrival-station EVA + time-of-day proximity (±120 min), one closest match per calendar day. The median arrival delay at the leg destination is taken over the matched days; cancelled days are excluded from the median but counted.

Setup

Prerequisites:

  • Python 3.12+
  • uv (curl -LsSf https://astral.sh/uv/install.sh | sh)
  • git

1. Clone with the submodule

The delay pipeline imports the parser from the deutsche-bahn-data submodule, so it must be checked out:

git clone --recurse-submodules https://github.com/sha2nkt/delay_bahn.git
cd delay_bahn

If you already cloned without --recurse-submodules:

git submodule update --init --recursive

2. Install dependencies

uv sync

This creates .venv/ and installs everything from uv.lock (FastAPI, DuckDB, pandas, huggingface-hub, ...).

3. Build the delay table

The app needs the merged delay table before it can show delay stats — it opens data/delays.duckdb (built by merge_delays.py), or falls back to loading data/delays.parquet into memory if the db file is absent:

uv run python pipeline/build_delay_db.py            # full window (default: 31 days)
uv run python pipeline/build_delay_db.py --days 3   # quick smoke run
uv run python pipeline/merge_delays.py              # write data/delays.parquet + the data/delays.duckdb the app opens

This downloads raw parquet files from the HuggingFace dataset into data/raw_data/ (~5.5 GB for the full window; data/de/delays.parquet adds another ~720 MB), parses each day once into the data/de/parsed/ cache, and merges them into data/de/delays.parquet. Re-run it daily to stay fresh — already-downloaded days are skipped and only days with new raw files are re-parsed. No HuggingFace account or token is needed; the dataset is public. merge_delays.py then combines the per-country tables (DE alone is fine) into data/delays.parquet and materializes the sorted data/delays.duckdb the app serves from.

4. Run the app

uv run uvicorn app.main:app --port 8000

Open http://localhost:8000, search a connection (e.g. Berlin Hbf → München Hbf), sort by "Wenigste Verspätung".

Layout

PathPurpose
pipeline/build_delay_db.pyHF download + XML parse → data/de/delays.parquet
pipeline/build_ch_days.py, pipeline/fr_poller.py, pipeline/consolidate_fr.pySwiss and French per-day producers
pipeline/at_poller.py, pipeline/consolidate_at.py, pipeline/build_at_stations.pyAustrian per-day producer (ÖBB HAFAS board poller + curated station list)
pipeline/nl_poller.py, pipeline/consolidate_nl.py, pipeline/build_nl_stations.py, pipeline/seed_nl_archive.pyDutch per-day producer (OVapi GTFS-RT poller + station crosswalk + archive seeder)
pipeline/it_poller.py, pipeline/consolidate_it.py, pipeline/build_it_stations.pyItalian per-day producer (ViaggiaTreno run tracking + station crosswalk)
pipeline/merge_delays.pyunions the per-country tables → data/delays.parquet + data/delays.duckdb
app/bahn_api.pyasync client for the bahn.de web API
app/delays.pyDuckDB delay-stats lookup (the core matching query)
app/live_delays.pylive same-day lookups via the DB Timetables API
app/main.pyFastAPI endpoints /api/locations, /api/journeys + static serving
static/vanilla HTML/CSS/JS frontend
deutsche-bahn-data/git submodule: data collection project whose parser and dataset we reuse
data/gitignored: raw parquet mirror + delays.parquet + delays.duckdb

Data sources & credits

Repo context for tooling and future work

  • feature_list.md — what the product does, feature by feature, with status
  • progress.md — current state snapshot, verification status, known limitations
  • log.md — append-only change log (newest entry last; never rewrite old entries)

License

This project is licensed under the Creative Commons Attribution-NonCommercial 4.0 International license (CC BY-NC 4.0) — see LICENSE for the full text. You may share and adapt this work for non-commercial purposes with attribution; commercial use requires separate permission.

The external data sources listed above remain under their own licenses (ODbL, CC BY 4.0, CC0, and provider terms).

Contributors

sha2nkt

297 commits

untitaker

1 commits

sha2nkt/delay_bahn

Python

125

298 commits

updated Sep 21, 2026

See the code

README

DelayBahn

Live at delaybahn.com Python 3.12+ FastAPI DuckDB

A bahn.de-style train connection search that shows the median arrival delay of the last 7 days (window selectable up to 30) for every connection, so you can book the one that actually runs on time. "Booking" deep-links to the real bahn.de page pre-filled with the journey.

Features

  • Delay statistics per leg: median arrival delay over the last 7/15/30 days, per-day charts, and the official IRIS delay causes behind every badge.
  • Four countries: Germany, Austria, Switzerland, and France — cross-border journeys get statistics on every leg.
  • Transfer-risk warnings: connections the arriving train's delay history makes tight or unlikely are flagged, and journeys can be sorted by connection risk.
  • Five sort modes: departure time, least delay, cheapest price, lowest connection risk, fewest transfers.
  • Compensation checker: reconstructs a past journey as it actually ran — exact delays, missed or cancelled connections, the replacement trains a passenger would have taken — and computes DB Fahrgastrechte compensation (25 %/50 %) with a deep link into the bahn.de claim flow.
  • Same-day results: a journey is checkable minutes after arrival via live IRIS lookups, showing the same value the nightly build will store.
  • DE/EN interface, shareable search URLs, station autocomplete served from the local delay data.

How it works

  • Journey search: the bahn.de web API (www.bahn.de/web/api) provides journey options including transfers and prices — the same API the bahn.de website uses. Station autocomplete is answered from the local delay data, falling back to that API for stations without delay history.
  • Historical delays, Germany: the public HuggingFace dataset piebro/deutsche-bahn-data publishes raw Deutsche Bahn IRIS timetable responses every 6 hours. The pipeline keeps a rolling 31-day mirror and builds a per-stop delay table covering all German stations.
  • Historical delays, Switzerland, France, Austria, and the Netherlands: official istdaten daily files (opentransportdata.swiss), 24/7 pollers on the official SNCF GTFS-RT feed and OVapi's Dutch train GTFS-RT feed, and a 24/7 poller sweeping ÖBB HAFAS (Scotty) station boards for the ~200 busiest Austrian stations produce per-day tables in the same schema; NL history is seeded from the Rijden de Treinen monthly train archive; pipeline/merge_delays.py unions all five countries into the served table.
  • Today's delays: for a journey the nightly pipeline has not ingested yet, delays are read at request time from the DB Timetables API (IRIS plan + fchg) — the same field the pipeline stores, so the answer does not change later. Optional: set DB_API_KEY/DB_CLIENT_ID (e.g. in a .env); without them the site simply stops at the last ingested day.
  • Matching: each train leg of a journey is matched against history by train number + arrival-station EVA + time-of-day proximity (±120 min), one closest match per calendar day. The median arrival delay at the leg destination is taken over the matched days; cancelled days are excluded from the median but counted.

Setup

Prerequisites:

  • Python 3.12+
  • uv (curl -LsSf https://astral.sh/uv/install.sh | sh)
  • git

1. Clone with the submodule

The delay pipeline imports the parser from the deutsche-bahn-data submodule, so it must be checked out:

git clone --recurse-submodules https://github.com/sha2nkt/delay_bahn.git
cd delay_bahn

If you already cloned without --recurse-submodules:

git submodule update --init --recursive

2. Install dependencies

uv sync

This creates .venv/ and installs everything from uv.lock (FastAPI, DuckDB, pandas, huggingface-hub, ...).

3. Build the delay table

The app needs the merged delay table before it can show delay stats — it opens data/delays.duckdb (built by merge_delays.py), or falls back to loading data/delays.parquet into memory if the db file is absent:

uv run python pipeline/build_delay_db.py            # full window (default: 31 days)
uv run python pipeline/build_delay_db.py --days 3   # quick smoke run
uv run python pipeline/merge_delays.py              # write data/delays.parquet + the data/delays.duckdb the app opens

This downloads raw parquet files from the HuggingFace dataset into data/raw_data/ (~5.5 GB for the full window; data/de/delays.parquet adds another ~720 MB), parses each day once into the data/de/parsed/ cache, and merges them into data/de/delays.parquet. Re-run it daily to stay fresh — already-downloaded days are skipped and only days with new raw files are re-parsed. No HuggingFace account or token is needed; the dataset is public. merge_delays.py then combines the per-country tables (DE alone is fine) into data/delays.parquet and materializes the sorted data/delays.duckdb the app serves from.

4. Run the app

uv run uvicorn app.main:app --port 8000

Open http://localhost:8000, search a connection (e.g. Berlin Hbf → München Hbf), sort by "Wenigste Verspätung".

Layout

PathPurpose
pipeline/build_delay_db.pyHF download + XML parse → data/de/delays.parquet
pipeline/build_ch_days.py, pipeline/fr_poller.py, pipeline/consolidate_fr.pySwiss and French per-day producers
pipeline/at_poller.py, pipeline/consolidate_at.py, pipeline/build_at_stations.pyAustrian per-day producer (ÖBB HAFAS board poller + curated station list)
pipeline/nl_poller.py, pipeline/consolidate_nl.py, pipeline/build_nl_stations.py, pipeline/seed_nl_archive.pyDutch per-day producer (OVapi GTFS-RT poller + station crosswalk + archive seeder)
pipeline/it_poller.py, pipeline/consolidate_it.py, pipeline/build_it_stations.pyItalian per-day producer (ViaggiaTreno run tracking + station crosswalk)
pipeline/merge_delays.pyunions the per-country tables → data/delays.parquet + data/delays.duckdb
app/bahn_api.pyasync client for the bahn.de web API
app/delays.pyDuckDB delay-stats lookup (the core matching query)
app/live_delays.pylive same-day lookups via the DB Timetables API
app/main.pyFastAPI endpoints /api/locations, /api/journeys + static serving
static/vanilla HTML/CSS/JS frontend
deutsche-bahn-data/git submodule: data collection project whose parser and dataset we reuse
data/gitignored: raw parquet mirror + delays.parquet + delays.duckdb

Data sources & credits

Repo context for tooling and future work

  • feature_list.md — what the product does, feature by feature, with status
  • progress.md — current state snapshot, verification status, known limitations
  • log.md — append-only change log (newest entry last; never rewrite old entries)

License

This project is licensed under the Creative Commons Attribution-NonCommercial 4.0 International license (CC BY-NC 4.0) — see LICENSE for the full text. You may share and adapt this work for non-commercial purposes with attribution; commercial use requires separate permission.

The external data sources listed above remain under their own licenses (ODbL, CC BY 4.0, CC0, and provider terms).

Contributors

sha2nkt

297 commits

untitaker

1 commits

Languages

Python

51.3%

JavaScript

27.7%

HTML

11.1%

CSS

9.1%