DrishtiMap

DrishtiMap is an open-source Progressive Web App that provides real-time audio navigation for visually impaired Indians.

The Problem:

5 million visually impaired Indians navigate daily without knowing what is around them. Existing solutions are either expensive hardware devices, English-only Android apps that require installation, or academic prototypes that were never shipped. None of them speak Bengali or Hindi. None work offline.

What DrishtiMap Does:

The user opens the app in any browser and points their phone camera at their surroundings. An on-device AI model (TensorFlow.js + COCO-SSD) detects objects in real time and generates natural language descriptions — "Person ahead — very close", "Door on your left", "Vehicle ahead — be careful". These descriptions are spoken aloud in Bengali, Hindi, or English. The entire detection pipeline runs on the device with no server calls, making it work completely offline after the first load.

Bengali audio is powered by BanglaTTS, an open-source Bengali text-to-speech library, served via a lightweight Flask backend. English and Hindi use the browser's built-in Web Speech API.

Why It Is Different:

Unlike existing tools, DrishtiMap requires zero installation, works in any mobile browser, supports Bengali and Hindi natively, and runs the AI model entirely on-device. It is designed specifically for low-cost Android phones on slow connections — the target device for most visually impaired Indians who cannot afford dedicated assistive hardware.

Tech Stack:

- React 18 + Vite + Tailwind CSS (frontend)

- TensorFlow.js + COCO-SSD (on-device object detection)

- Web Speech API (English and Hindi audio)

- BanglaTTS — MIT licensed (Bengali audio)

- Flask backend (Bengali TTS server)

- PWA with offline support

- Fully open source — MIT licensed

No proprietary APIs are used anywhere in this application.