Skip to Main Content
Talk Intermediate GNU AGPLv3

Building TranscribeIt: Lessons from building a local multimedia accessibility platform

Proposal status is Withdrawn
Keerthana Rajesh Kumar
Keerthana Rajesh Kumar
Session Description

The talk explores into depth the need for accessibility with multimedia content such as videos for WCAG compliance and inclusion. The talk describes the lack of accessibility with online video and live streaming platform in terms of transcriptions and video description, making it harder for people with hearing, speech and visual impairments to access digital content.

It explores the journey of development of TranscribeIt, touching on aspects such as:

  • Performance optimization and timestamping with faster-whisper for working on CPU and low-spec machines.

  • Extending the platform to support generic audio and video content for transcription.

  • Combining speaker diarization with customization using pyannote.

  • Extending video content accessibility using video description, allowing correlation with transcriptions for people with visual impairments using FrameStory.

It touches on challenges encountered while developing the application in terms of:

  • WER for non-Latin languages, with emphasis on Indian languages, describing the challenges and scope of improvement with OpenAI Whisper and usage of fine-tuned models over whisperx (such as Whisper-Hindi-v2 for Indian content)

  • Striking balance between audio quality and transcription accuracy in processing real-world audio with noise reduction techniques

The talk finally touches on the future implementation and scope of improvement for improved accessibility in online platforms, making open-source accessibility software as the pioneer for digital inclusion, shaped by local LLMs for improved privacy and room for improvement with local AGI.

Key Takeaways

  • Understanding the importance of multimedia accessibility and how it is linked to WCAG compliance

  • The process of developing TranscribeIt with technical process while being local-first and privacy respecting

  • Understanding the challenges in developing multimedia accessibility systems, allowing room for scope and discussion for future development

References

https://github.com/fossiaorg/transcribeit
https://fossia.org/blog/givingtuesday-with-transcribeit
https://fossia.org/blog/building-transcribeit

Session Categories

Introducing a FOSS project or a new version of a popular project
Technology architecture
Talk License: GNU AGPLv3

Speakers

Keerthana Rajesh Kumar
Keerthana Rajesh Kumar Founder | FOSSIA

Keerthana is the founder of FOSSIA, a women-centric community for encouraging underrepresented people to contributing to the Indian free software ecosystem. She is an open source contributor and developer, focusing on improving accessibility in applications.

She is an ex-intern at Ente, where her passion for local AI grew into research for intuitive accessibility software that respects privacy and can be managed independently.

She volunteers for D&I initiatives in technological communities in terms of mentorship and guidance, especially for people with disabilities. She is a volunteer at FOSS United, CHAOSS and FOSSASIA Summit 2026.

She is also the founder of InLibre, a technological startup for developing inclusive technologies for the Indian community.

Apart from technology, she enjoys blogging, traveling and spending good time with her close ones.

https://libremusings.dev