RowdyAccess

Accessible pedestrian routing for the UTSA Downtown Campus (San Antonio, TX).

Navigation apps give the shortest route. For a wheelchair user or a low-vision pedestrian, that route may be impassable: a corner without a curb ramp, a bike blocking the sidewalk, or a crossing with no signal. RowdyAccess combines OpenStreetMap sidewalk data with computer vision run on street photos we took ourselves (YOLO + RampNet). It then shows the shortest route next to the best route for your profile and explains the difference in plain language.

Scope: about 700 m around 501 W César E. Chávez Blvd.

---

Quick start

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# CPU-only laptop? Install the much smaller CPU build of torch first:
#   pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu

python backend/main.py          # → http://localhost:8000

On first start the server downloads YOLO11n (models/yolo11n.pt) and RampNet (Hugging Face cache). After that everything works offline except the map tiles:

HF_HUB_OFFLINE=1 python backend/main.py        # demo day: no network calls to Hugging Face
PATHABLE_PORT=8765 python backend/main.py      # other port
PATHABLE_DEVICE=cpu python backend/main.py     # force CPU
PATHABLE_NO_DETECTOR=1 python backend/main.py  # routing only (no torch needed at runtime)

Basemap: the app uses OpenStreetMap standard raster tiles and needs no map-provider API key. Without internet, routes and markers still draw on a plain background.

The OSM graph is saved per campus (data/utsa_downtown_walk.graphml, data/utsa_main_walk.graphml) and is never re-downloaded. Delete a file to rebuild it (needs internet).

Data storage (SQLite)

The data the app changes while it runs is kept in a SQLite database, data/rowdyaccess.db (backend/store.py): the map points (live set and the "Restore demo data" baseline), share links and the street-view photo index. Every change is one transaction.

The database file is part of the repository; there are no JSON copies of this data.

python backend/store.py info      # what is in the database
python backend/store.py export    # write it as JSON files into data/export/ (to read or diff; not tracked)
python backend/store.py import    # load data/export/ back into the database (replaces its contents)
PATHABLE_DB=/tmp/test.db python backend/main.py    # use another database file

Reference data that the app only reads stays in JSON: presets.json, entrances.json, streets.json, osm_features.json, via_link_stops.json.

Campuses

The app covers the campuses listed under campuses in data/presets.json (Downtown Campus, Main Campus). The user picks one first; its buildings, parking lots, service area and map view are then loaded. Each place in buildings belongs to a campus ("campus": "main"; without the field: downtown). To add or refresh a campus:

python -c "import sys; sys.path.insert(0, 'backend'); import graph_builder as gb; gb.download_graph('main')"
python backend/entrances.py main      # entrances of its places  → data/entrances.json
python backend/streets.py             # street names             → data/streets.json
python backend/osm_features.py main   # mapped crossings, steps, curb ramps → data/osm_features.json

Walking routes stay within one campus.

Docker

# put your keys in .env first (see .env.example); it is copied into the image
docker compose up --build       # → http://localhost:8000   (admin: http://localhost:8000/admin)

The image builds the React frontend, installs the CPU build of PyTorch and bundles YOLO11n, so the container starts without downloading anything. ./data is mounted into the container: uploads, admin edits and share links stay on the host and survive rebuilds.

• Keys. Your real .env is copied into the image when it is built (/app/.env), so the server and the scripts inside it (docker compose exec app python backend/mapillary_scan.py --campus main) have your keys, and the image runs on another machine without the file. That means the image contains your API keys: keep it private and do not push it to a public registry. With compose, .env is also passed at start, so after editing it docker compose up -d is enough; for a plain docker run, rebuild.
• User id. The container runs as uid 1000. If id -u prints something else, add PATHABLE_UID=<id -u> and PATHABLE_GID=<id -g> to .env before building, otherwise it cannot write ./data.
• Admin token. Set PATHABLE_ADMIN_TOKEN in .env, or read the random one from docker compose logs app.
• Other port. Change the left side of ports in docker-compose.yml ("8080:8000"). A Google Maps key restricted to http://localhost:8000/* then needs the new address added.
• GPU. With the NVIDIA Container Toolkit: docker compose --profile gpu up --build app-gpu. This installs CUDA PyTorch (a much larger image) and runs RampNet on uploaded photos; RampNet is downloaded on first start and kept in the hf-cache volume.
• CPU. RampNet is off for uploads (about 30 s per photo). Uncomment PATHABLE_UPLOAD_RAMPNET in docker-compose.yml to turn it on.

Without compose: docker build -t rowdyaccess . && docker run --rm -p 8000:8000 -v "$PWD/data:/app/data" rowdyaccess

Frontend (React + Material You)

The UI lives in web/ (React 19, Material Web components, dynamic Material You color, MapLibre GL). A built copy is in frontend/dist/, which FastAPI serves at /, so the demo needs no Node.js. To change the UI (needs Node.js 20+; on our dev server: conda activate pathable-node):

cd web
npm install
npm run dev      # http://localhost:5173, API calls are proxied to the backend on :8000
npm run build    # writes ../frontend/dist (commit it)

Notes for the browser:

• Locate me needs a secure page (http://localhost:8000 or HTTPS) and location permission. Away from campus it says how far you are and offers Use campus location for demos.
• Share opens a dialog with the link and a Copy button (works even in embedded previews that block the Clipboard API).

Demo script

The service area (shaded outside, dashed outline) is the bounding box of the campus buildings in data/presets.json plus 150 m (BOUNDARY_PAD_M in backend/main.py); stops outside it are rejected. The sample trip is set in data/presets.json ("demo": start/end, or a "stops" list of building names). The building coordinates come from OSM.

Accessibility features from OpenStreetMap

python backend/osm_features.py    # → data/osm_features.json

Imports what OpenStreetMap already maps inside the service area: crossings (signalized with marking style, audible signals and push buttons where tagged; marked; unmarked), curb types, and steps. They show on the map with a "From OpenStreetMap" link and are merged into the map points by the detector (step 3 below); a team label at the same spot (within 5 m) overrides them. Currently: 53 signalized crossings, 7 marked crosswalks, 5 unmarked crossings, 2 sets of steps.

Finding curb ramps with RampNet on Mapillary street photos

OpenStreetMap has no curb-ramp data here, but Mapillary has thousands of street photos downtown (2015–2021, mostly dashcam; no 360° panoramas around campus). With a Mapillary client token in .env (MAPILLARY_TOKEN="MLY|..."):

python backend/mapillary_scan.py --limit 5   # try a few crossings first
python backend/mapillary_scan.py             # every mapped crossing (~5 min on GPU)

For each crossing it picks the 2 nearest photos whose camera faces it, runs RampNet (and YOLO), turns each ramp's position in the photo into a compass direction, and snaps it to the crossing corner in that direction. RampNet was trained on 360° panoramas, so on ordinary photos its peaks are weaker (0.3–0.5): the scan uses threshold 0.3 and keeps only ramps that line up with a mapped crossing corner. YOLO's objects are logged but not saved (the photos are years old). Results replace earlier scan results in the database (live points and the reset baseline). Restart the server afterwards. Last run: 73 photos at 65 crossings → 13 curb ramps (confidence 0.35–0.82), each with a link to its Mapillary photo.

Street View for map points: python backend/mapillary_scan.py --streetview picks, for every feature in the database, the nearest Mapillary photo that faces it (else the closest within 60 m), saves it as data/photos/sv_<id>.jpg and indexes it in the database. Popups then show that photo with Open Street View, an interactive Mapillary viewer (needs MAPILLARY_TOKEN). Last run: 67 of 80 features.

Collecting photos and running detection

1. Take photos with location on and the camera set to JPEG / "Most Compatible". HEIC also works (pillow-heif), but JPEG is safer.
2. Name photos of street corners corner_*.jpg. If RampNet finds no ramp in a corner photo, it records a no_curb_ramp.
3. Copy the photos into data/photos/ and run:

   python backend/detector.py --photos data/photos
   #   --no-rampnet             YOLO only (e.g. on a CPU laptop; RampNet takes ~30 s per photo on CPU)
   #   --rampnet-mode stretch   README-style resize instead of the panorama embedding (compare both!)
   

   Photos without GPS are skipped and logged. Annotated copies go to data/photos/annotated/. The results go to the database (live points and the reset baseline).
4. Check the pins in the UI. Fix mistakes in data/manual_labels.json, which uses the same format; id, photo and bbox are optional:

   [{"type": "curb_ramp", "lat": 29.42359, "lon": -98.50051, "label": "ramp, NW corner Nueva/Pecos"}]
   

   A manual label replaces any model label of the same kind (ramp / no ramp count as one kind) within 5 m. Re-run step 3, then restart the server.

Shared detection format (every source outputs this):

{"id": "det_0001", "type": "obstacle", "label": "bicycle", "lat": 29.4229, "lon": -98.5012,
 "confidence": 0.91, "photo": "IMG_0412.jpg", "bbox": [x1, y1, x2, y2], "model": "yolo-coco", "verified": false}

Tuning

All weights are in one dictionary: PROFILES in backend/profiles.py. Penalties are "extra meters"; INF means blocked. To see how routes change, run:

python backend/graph_builder.py 29.422982 -98.502770 29.423082 -98.497637

API

Architecture

 phone photos (GPS EXIF)         OpenStreetMap (osmnx, downloaded once)
          │                                   │
          ▼                                   ▼
 detector.py                          graph_builder.py
  ├ EXIF GPS (Pillow)                  ├ walk graph → data/utsa_downtown_walk.graphml
  ├ YOLO11n COCO → obstacle,           ├ crossings = groups of footway=crossing edges
  │   traffic light (weak signal hint) │   + OSM tags: crossing, kerb, tactile_paving, surface
  ├ RampNet → curb_ramp / no_curb_ramp ├ attach detections (≤ 25 m):
  ├ crosswalk.pt (optional)            │   corner types → nearest crossing node
  ├ manual_labels.json override (5 m)  │   obstacle / damaged → nearest edge
  └ annotated JPEGs                    └ per-profile edge costs (profiles.py)
          │                                   │
          └────────► data/rowdyaccess.db ◄────┘
                              │
                     main.py (FastAPI, all in memory)
                     /route /features /upload /label /reset /photos
                              │
                     web/ → frontend/dist (React + Material You, MapLibre GL 3D map)
                     multi-stop directions · markers + photos · route comparison · explanation · upload

How the cost model works:

• Edge cost = length × multipliers + penalties.
• A crossing that is split into several short OSM edges is charged its penalty once: half on each end edge.
• A missing curb ramp blocks every crossing that starts at that corner. A sidewalk along the corner without crossing it is not affected.
• Unknown is its own state. No data means neither good nor bad: an unknown signal gets a smaller penalty than "no signal", and edges without any data get the unknown multiplier. The UI shows them in light gray.
• no_curb_ramp from a manual label, a verified label, or OSM kerb=raised blocks the crossing. One that comes only from RampNet adds a large penalty instead, because a model miss shouldn't cut off a corner.

Credits

All application code was written during the event.

Known limitations

• GPS accuracy. Phone GPS is off by 3–10 m (worse between tall buildings). A corner photo can snap to the wrong corner of an intersection. Hand-verified labels fix this.
• Small area. Only about 700 m around the campus. Points outside are rejected.
• Partially hand-verified labels. Curb ramps rely mostly on team labels. RampNet was trained on 360° Street View panoramas, not phone photos, so we embed each photo in a panorama-sized canvas, and accuracy on our photos is not measured. OpenStreetMap has no curb-ramp data around campus (only curbs tagged "yes"), so ramps come only from our photos and team labels.
• One position per photo. All detections in a photo get the photographer's position, not the object's.
• Parked cars are ignored (COUNT_CARS_AS_OBSTACLES = False in detector.py); otherwise every street photo would block its sidewalk.
• Traffic lights are only a weak hint for pedestrian signals (confidence × 0.5); a car signal isn't a walk signal.
• OSM gaps. wheelchair and incline tags are absent here and kerb / tactile_paving are rare, so much of each route shows as "no data". Crossing tags (signals, markings) are well mapped.
• Live upload speed. On a CPU laptop, uploads run YOLO only (fast); RampNet runs on uploads only with a GPU or PATHABLE_UPLOAD_RAMPNET=1.