Most navigation apps are built for people who can see the screen. Cogniscape is built for people who cannot. It is an open-source iOS and Android application that combines OpenStreetMap pedestrian data, 3D spatial audio beacons, and on-device AI obstacle detection — enabling visually impaired users to navigate real-world environments entirely without screen interaction.
The spatial audio layer is inspired by Microsoft's SoundScape research. Audio beacons placed at destinations and waypoints shift left, right, or ahead based on the phone's compass bearing. Face your destination and the sound comes from in front. Turn away and it drifts behind you. The user always knows where they are going — by listening, not looking. On top of this, a second AI layer continuously scans the camera feed and delivers immediate directional audio warnings when obstacles — steps, poles, parked vehicles, uneven surfaces — are detected ahead. Both layers run fully on-device with no network dependency, preserving privacy and full offline capability.
The talk is structured in three parts:
Part 1 — OSM as an audio data source. Accessibility-relevant OSM tags — sidewalk, crossing, tactile_paving, kerb, surface — are inconsistently mapped globally, with South Asian cities among the least covered. I walk through how OSM data is extracted via Overpass API, transformed into a pedestrian routing graph, and enriched with POI data for beacon placement. Building Cogniscape exposed specific data quality gaps, and I describe the targeted mapathon workflows we designed to address them at a community level.
Part 2 — Offline architecture, spatial audio, and on-device AI. Routing uses embedded GraphHopper with OSM PBF extracts. Tiles are served via MapLibre Native with MBTiles offline packs. The audio engine uses Android Oboe and iOS AVAudioEngine with HRTF approximation for binaural rendering, with azimuth computed from compass and accelerometer sensor fusion. Obstacle detection runs via TensorFlow Lite on Android and Core ML on iOS — same model, two runtimes, no cloud calls.
Part 3 — Cross-platform design for non-visual interaction. Shipping on both Android and iOS required reconciling TalkBack and VoiceOver interaction models, two spatial audio APIs, and two on-device ML runtimes — while maintaining a single shared OSM data and routing layer. I cover the architectural decisions that enabled cross-platform parity and what building for blind users teaches any developer about accessible geospatial tool design.
Open source projects: OpenStreetMap, Overpass API, GraphHopper, MapLibre Native, Android Oboe, TensorFlow Lite, OSMAnd.