GhostRunner: Turning Every Street into a Story

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

GhostRunner: An open-source, local-first audio drama engine that turns the real streets under your feet into the set of a thriller, and a one-laptop side hustle for anyone who wants to sell themed run and walking experiences.

What I Built

GhostRunner is a generative audio companion for runners and walkers. You pick a genre (zombie escape, cyberpunk courier, spy exfiltration), a distance and an intensity. GhostRunner plans a route over real streets, finds the real landmarks along it, and writes an audio drama in which your surroundings are the set:

"Turn right onto Oak Street. See that old brick building on your left? That's where the drop is hidden."

Then you put your phone away and run. The right line plays at the right place, in a radio-dispatcher voice, just before the turn it describes. If you speed up, the tone turns urgent and a pursuer closes in. If you flag, the story hands you a breather. If you take a wrong turn, the dispatcher notices, re-routes you and rewrites the rest of the mission.

Three things make it more than a prompt wrapped around a map:

• It is spatially exact. Every beat is anchored to a distance along the route, so a line about a turn plays before the turn, not after it.
• It is grounded. The narrator can only name landmarks that exist in OpenStreetMap along your route. A validator rejects anything else.
• It is fast, because nothing is generated while you run. The story is compiled and voiced ahead of you. Playing a line is a lookup, not an inference.

And it runs on a laptop with a 4 GB GPU.

The side hustle: selling mission packs

The expensive part of an audio drama has always been making it. A writer, a voice actor and a studio for every route does not scale to "your neighbourhood". GhostRunner turns that into a few minutes of laptop time per route, and that makes a small one-person service possible. The unit of sale is a mission pack: a route, a compiled story and voiced audio, made once and replayed any number of times.

Who I think would pay for one:

• Running clubs and run-shop events. A "Zombie 5K" or "Cyberpunk Courier" themed around the club's own neighbourhood, with the club's real landmarks, for a monthly social run.
• Walking-tour hosts, hotels and campuses. A self-guided story tour that uses the venue's real landmarks (I can add them by hand through a demo_pois.json file) and a branded voice.
• Runners directly. City or neighbourhood packs for people who are bored of the same loop.

To be straight about where this stands: I haven't run a paid pilot or validated pricing, and today a saved mission replays from my own machine. The pieces that make it sellable (compile once, replay forever, no per-listener cost, landmarks and voice swapped per client) are built. Exporting a pack as a standalone bundle a customer's phone can play without my server is the next step.

Demo

Video Link

Code

Repository: https://github.com/Nishchal-29/GhostRunner

engine/     routing, chainage math, corridor POIs, tracker, trigger engine, pacing, safety
pipeline/   LangGraph compile_graph and runtime_graph
llm/        Ollama client (structured output)
audio/      TTS providers and clip cache
client/     mobile-first PWA (vanilla JS, Web Audio API, Leaflet)
tools/      LLM benchmark, GPX replayer, demo launcher

Run it locally:

conda activate agents-env
docker compose up -d                 # Valhalla (routing) + Postgres
python tools/setup_valhalla.py
ollama serve
python tools/demo.py                 # compiles a demo loop and starts the app
# open http://localhost:8000

How I Built It

The problem: latency and trust

A story that plays late is wrong, and a story that invents landmarks is worse. A runner moves roughly 3 m/s. An LLM on a small GPU takes seconds per line, and text-to-speech adds more. If you generate reactively, the line about the clocktower arrives after you have passed it. And small models hallucinate confidently.

So I split the system by when work happens:

COMPILE (before and during the run)
Plan → Valhalla route → Overpass corridor POIs → route model (chainage)
     → Director (LLM, 1 call) → Beat planner → Beat writer (LLM, 3 variants/beat)
     → Validator → TTS pre-render (rolling window)

RUN (every tick, no LLM on the happy path)
GPS → pace classifier (TabPFN) → locate on route → trigger check → variant select → play

Two LangGraph graphs: compile_graph carries the heavy work, runtime_graph is a handful of deterministic nodes that finish in milliseconds. This split is also what makes the business model work: everything expensive happens once, at compile time, and the result is reusable.

The route is a coordinate system

I use Valhalla with pedestrian costing for routing. It returns an encoded polyline plus maneuvers with street names. I decode it (precision 6), project into the local UTM zone with pyproj, and densify to one point every ~5 m.

Everything on the route is then addressed by chainage: meters along the path. A turn is at 1,240 m. A fountain is at 1,310 m, on the left. This one choice does most of the work in the project:

• Triggers are distance-based, so a pre-compiled beat stays valid at any pace. Pace only selects which variant plays.
• Side-of-route is a cross product in metric coordinates: with tangent T and landmark vector L, the sign of Tx·Ly − Ty·Lx says left or right.
• Runtime snapping searches only [last_chainage − 30 m, last_chainage + 200 m] of the densified path, so loops and out-and-back routes can't snap the runner onto the wrong leg.

Some details I designed around, because they fail silently:

• Valhalla's begin_shape_index is relative to each leg, so multi-leg routes need a running offset or every maneuver lands in the wrong place.
• Street detours make loops overshoot (a triangle at radius D/4 is ~0.93·D before detours). I generate three candidate loops from waypoints at 60°/180°/300°, score them by distance error and self-overlap, and recalibrate the radius once if the first pass is off by more than 15%.

Real landmarks, not imagined ones

One Overpass query along a simplified (≤150 point) buffer of the route pulls historic sites, artwork, fountains, places of worship, named buildings, parks, waterways, bridges and steps. I score each by kind and distance from the path, then apply greedy non-maximum suppression so no two story landmarks sit within 150 m of each other. Results are cached per H3 cell in Postgres, so a second run makes no Overpass calls.

OSM coverage varies. When a route is landmark-poor, the story degrades gracefully into street names, turns, parks and water, and a hand-curated demo_pois.json can add landmarks. That file is also how a client's own landmarks (a club's meeting point, a hotel, a campus building) get into a pack.

The LLM narrates; it doesn't orchestrate

Small models are unreliable at tool calling, and every tool-call round trip costs latency a runner can't spare. So all decisions are plain Python: which beat goes where, which landmark, what maneuver text, how far ahead to trigger. The LLM does two jobs only:

1. Director (one structured call): produces three acts with chainage ranges, 3-5 objective landmarks (drop, checkpoint, extraction), an antagonist premise, and a story bible (callsigns, motifs, tone).
2. Beat writer: for each beat, writes three variants (cruising, accelerating, exhausted) of at most 30 words, from a structured brief. Each brief carries the story bible, a rolling summary of recent beats, and the landmarks already passed, so lines can call back to earlier ones.

A validator enforces word count, an allowlist of landmark names, and safety rules (no crossing roads, no trespassing). On failure it retries once with the violation named, then falls back to a deterministic template.

The model sits behind a small LLMClient interface, so swapping it is a one-line config change. That matters for a business: a model or a price I depend on can change without breaking the product.

Trigger timing

For each beat, the line should finish as the runner reaches the anchor. The engine dead-reckons position between 3-5 s GPS fixes and fires a beat when:

def due(beat, chainage_est, speed, variant):
    dur = beat.duration_s.get(variant) or est_duration(beat.variants[variant])
    lead_m = max(speed, 0.5) * (dur + 2.0)           # finish speaking as the runner arrives
    start_at = beat.anchor_chainage_m - lead_m
    return start_at <= chainage_est < beat.expires_chainage_m

Durations come from the measured length of the rendered audio clip, so the lead distance is accurate to the clip. A priority queue (navigation > checkpoint > landmark > challenge > tension) plays one beat at a time, enforces a minimum gap between non-navigation beats, and skips expired ones.

Pace awareness with TabPFN

I treat pace as a small tabular classification problem. Each tick builds features: rolling speed over 30 s / 2 min / 5 min, speed delta and variance, grade, time since last stop, and current speed relative to a personal baseline from the first two minutes. TabPFN classifies the runner as CRUISING, ACCELERATING, EXHAUSTED or STOPPED, with a 3-tick hysteresis filter so the story doesn't flip-flop. It runs on CPU, throttled to once every 5 s, leaving all VRAM to the LLM.

One honest caveat: there is no labeled dataset for this, so the training labels come from a transparent heuristic over simulated run profiles. TabPFN's value here is a fast, calibrated, retrain-in-seconds classifier. If it fails to load, a rules classifier takes over automatically.

Voice

ElevenLabs voices every line, rendered ahead of the runner through a rolling window of the next three triggers. Clips are content-addressed (sha256(voice + text)), so repeated lines are free. Reactive moments (off route, pick up the pace, checkpoint countdown) use a pre-rendered tier-C bank that plays from disk in under 50 ms while any dynamic line renders in the background. On the client, Web Audio adds a bandpass-and-distortion radio effect, and the ambient drones and sirens are synthesized procedurally, so there are no audio assets to ship.

Where the cost actually is

I tried to be honest with myself about the unit economics, because "free" is rarely the whole story.

• Story text: local, unmetered. One route is roughly 25 beats times 3 variants of about 30 words, so on the order of 7-8k generated tokens. That is cheap even through an API, so I won't pretend the local model is what makes the numbers work. What it buys is that I can recompile, re-benchmark and retune prompts hundreds of times with no meter running and no rate limits.
• Routing, landmarks and maps: open data, no per-request fee. Valhalla runs on my machine. Public Overpass has fair-use limits, so a real service would self-host it, and H3 caching means a route's landmarks are fetched once.
• Voice: the one metered cost. A full pack with every variant voiced is roughly 25 beats × 3 variants × ~150 characters, so about 11k characters by my estimate (check it against your own plan). Because a pack is rendered once and then replayed, voice cost is per route, not per listener. Content-hash caching also means any repeated line is paid for once.

That shape is what makes a side hustle plausible: fixed cost per route, near-zero cost per run.

Product details that make it feel real

A mobile-first PWA with a briefing screen (route preview, premise, objectives, safety acknowledgement, headphone check), a countdown, a run HUD, and a debrief with a shareable card. Every compiled mission is saved, so any mission replays with the LLM, router and TTS all switched off. Wi-Fi at a venue, or a trail with no signal, is the one thing a demo or a customer can't count on, so a saved mission needs none of it.

What I'd build next to actually sell it

• Export a mission pack as a self-contained bundle a phone can play with no server.
• Branded packs: a client supplies landmarks, a meeting point and a voice, and the pack is generated from that.
• A second narration language, which the open-model route makes possible without a new vendor.
• A pilot with one local running club to learn what they would really pay for.

Stack

Python, FastAPI, WebSockets, LangGraph (Postgres checkpointer), Valhalla, Overpass/OpenStreetMap, shapely, pyproj, H3, Ollama with an open-weight model ([model]), TabPFN, ElevenLabs, Sentry, Postgres, vanilla JS with the Web Audio API and Leaflet.

Why Does Open Innovation Matter?

For a project someone might pay for, the open pieces decide whether the business works at all, not just whether it's a nice demo.

• Customers' location stays under their control. A runner's GPS trail is sensitive, and the people I'd sell to (clubs, schools, employers, tour operators) will ask where it goes. In GhostRunner the live trail never leaves the machine running the app: the model, router and pace classifier are all local, and raw trails are deleted when a session ends. The only things that leave are the planned route corridor (to Overpass, which can be self-hosted) and the script text (to the TTS provider). "We never see your runners' location" is a sentence I can say honestly, and it would be hard to say with a hosted model in the loop.
• No lock-in on the parts that can change under me. The narrator is behind an LLMClient interface and the voice is behind a TTSProvider interface, with a mock provider so the full pipeline runs with no vendor at all. I picked the model by benchmark, and I can re-pick it. If a provider changes pricing or retires a model, I change a config value, not the product.
• The niche is a street, and open data fills it. The thing a general-purpose API handles worst is "the brick building on this corner". A model can't know that, however large it is. OpenStreetMap does, and the architecture injects it and then validates the output against it. A client's own landmarks drop in through one file. Open weights also leave fine-tuning a narrator to a client's brand voice on the table, though I haven't done that yet.
• Open map data can be cached and reused. OSM data is open (ODbL, with attribution required), so I can store routes and landmarks and rebuild packs from them. Many commercial map APIs restrict storing or reusing results, so check the terms of whichever one you would compare against.
• It works with no signal. Saved missions replay with the LLM, router and TTS all switched off, which is exactly what a park, a trail or a conference-hall demo needs.
• Constraints made the design better. A 2-4B model won't improvise a perfect story, so I built around it: deterministic structure, grounded briefs, a validator and pre-generation. That same design is what makes a pack reusable and cheap to serve.

I also want to be straight about the closed piece. Voice is ElevenLabs, because it was the best option for a convincing radio dispatcher and I wasn't going to pretend otherwise. It is isolated behind an interface and cached aggressively, so it is the one swappable, bounded cost instead of something the whole product depends on.

Prize Categories

Best Use of ElevenLabs. Every line is voiced in a per-genre radio-dispatcher voice, with an optional second voice for the antagonist. Rolling pre-rendering, content-hash caching and measured clip durations are what let the audio land at the right place, and a pre-rendered reactive bank gives sub-50 ms responses to off-route and pace events.

Best Use of TabPFN. A TabPFN classifier turns streaming telemetry features into a runner state (cruising / accelerating / exhausted / stopped) that selects the story variant in real time, with hysteresis and a rules fallback.