Featured Project

PathFindr

A mobile accessibility navigator that combines ARKit LiDAR sensing, multimodal AI agents, and adaptive spoken guidance for blind and low-vision users.

Technical Summary

  • Built an iOS-first navigation flow where ARKit streams LiDAR depth + camera context, then extracts spatial features (distance, directionality, obstacle proximity) for real-time guidance.
  • Implemented a Swift client to package structured environment snapshots into JSON and send low-latency requests to a Flask middleware API.
  • Orchestrated a multi-agent ADK backend (prompt, hazard, image, semantic, narrator) so tasks are routed by intent and merged into one speech-ready response.
  • Added multimodal feedback loops: concise audio narration for primary guidance and vibration cues for immediate hazard escalation during movement.
  • Tuned trade-offs between latency, response quality, and narration density to keep outputs actionable in motion without overwhelming users.
  • Validated the end-to-end architecture in hackathon conditions, demonstrating practical accessibility impact and resilient sensor-to-voice operation under noisy inputs.

Architecture Diagram

PathFindr architecture diagram

PathFindr Demo Video

Links

Back to all projects