Turkey Search Demand Map

An interactive analytics map for exploring geo-search demand across Turkey. The web app shows province-level heatmaps, filters requests by time and category, and supports demand forecasting with a machine-learning model.

What it does

• Choropleth map — color Turkish provinces by metrics such as search volume or average organization rating.
• Filters — slice data by hour ranges, weekdays, provinces, categories, and rating.
• Province drill-down — inspect aggregated stats and distributions for a selected region.
• ML forecasting — predict short-term demand for a province (requires a trained model file).
• Localization — English and Turkish UI.

Search-system request logs are cleaned and joined to city boundaries before the map can show demand.

Project layout

turkey/
├── api/                 # FastAPI backend (demand + ML endpoints)
├── cli/                 # Data pipeline (python -m cli.run_map)
├── data/
│   └── tr-cities.json   # Province boundaries (GeoJSON; see note below)
├── ds/                  # Notebooks and offline preprocessing / ML
├── frontend/            # React + Vite + Leaflet UI
├── app.py               # ASGI entrypoint (exports FastAPI app)
└── requirements.txt     # Python dependencies

Map boundaries (data/tr-cities.json)

Province boundaries are not authored in this repo. tr-cities.json was taken from an external project:

Source: https://github.com/alpers/Turkey-Maps-GeoJSON

Prerequisites

• Python 3.10 or newer (3.11+ recommended)
• Node.js 18+ and npm (for the frontend)
• A C compiler may be required on some systems to install GeoPandas / Shapely wheels if prebuilt packages are unavailable.

Installation

1. Python environment

From the repository root:

python3 -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt

2. Frontend dependencies

cd frontend
npm install
cd ..

Preparing data

The UI and API will start without prepared data, but the map and analytics will be empty until you run the pipeline.

Input logs are search-system request logs (JSONL). One command runs the full pipeline:

python -m cli.run_map --logs path/to/logs.jsonl --geojson data/tr-cities.json

Stages (in order):

1. Slim JSONL — extract org_* from model_response_full
2. Spatial join — keep points inside tr-cities.json (adds province name, number)
3. Optional query cleanup (--exclude-queries, --drop-min-count)
4. Translation + categorization — org_type_en and category on data/df_with_cat.csv
5. Province split — data/provinces/{number}.csv with org_name, time, category, org_rating

Query cleanup is skipped unless you opt in (see below). See python -m cli.run_map --help for all flags (--regions-dir, --exclude-queries, etc.).

Translation model: the pipeline uses Helsinki-NLP/opus-mt-tr-en as a free, local example — it is not the best Turkish→English option. Replace it in cli/translate.py if you need higher quality (e.g. a larger Marian/NLLB model or a paid API). Categorization uses BAAI/bge-large-en-v1.5 on the translated labels.

Optional query cleanup

Noisy high-volume prompts (often from other systems) are not removed by default. Use either or both flags:

python -m cli.run_map --logs path/to/logs.jsonl --geojson data/tr-cities.json \
  --exclude-queries data/bad_queries.txt

Example — drop any query that occurs 500+ times:

python -m cli.run_map --logs path/to/logs.jsonl --geojson data/tr-cities.json \
  --drop-min-count 500

Both flags can be combined; exclusions run first, then the frequency threshold.

To re-run only categorization + split on an existing CSV: python ds/ml/categorize_org_types.py --input data/df.csv.

Further model training and notebooks live under ds/.

Running the application

Production-style (single server)

Build the frontend, then start the API (it serves the built static files):

cd frontend && npm run build && cd ..
uvicorn app:app --host 127.0.0.1 --port 8000

Open http://127.0.0.1:8000.

Development (hot reload)

Use two terminals.

Terminal 1 — API:

source .venv/bin/activate
uvicorn app:app --reload --host 127.0.0.1 --port 8000

Terminal 2 — frontend dev server (proxies /api and /tr-cities.json to the API):

cd frontend
npm run dev

Open http://127.0.0.1:5173.

Machine learning

Demand prediction endpoints (/api/ml/*) expect a serialized model at api/rf_model.pkl. This file is not committed (see .gitignore). Train or export the model using notebooks under ds/ml/, then place rf_model.pkl in api/ before using the predict panel in the UI.

API overview

Interactive API docs: http://127.0.0.1:8000/docs when the server is running.

Data science workflow