Building Local, Secure Gen AI apps on Mobile
Session Description
As use cases for Generative AI permeates to edge and mobile devices, there are a lot of great use cases for building vector search / RAG applications running directly on users mobile devices. One of the common concerns with using cloud solutions for LLM inference is data security, privacy and latency.
This session addresses all of these concerns by introducing open source tooling capable of running fully fledged Generative AI privately, securely and on device in Android such as Onnx ML framework for embeddings, Gemma open source model for LLM, Sqlite Vector search.
So the biggest takeaway for the audience is learning tooling and how to build on device Gen AI applications for mobile with detailed code examples to build secure AI applications that run completely locally on users devices.
Key Takeaways
Benefits to the Ecosystem
The biggest use-case for the mobile and AI dev community is the fact that the same models can now run efficiently on device thanks to optimizations like WASM, Onnx, and just better open source SLM support. The speakers come with building real world use-cases for customers which shares success stories for running on Device AI applications
Session Categories
Which track are you applying for?
Speakers
Shivay Lamba
Reviews
Good topic, but another detailed proposal also touches upon local and secure on-device/mobile AI in a real-world, large-scale, user-facing FOSS app.
Agreed with the other reviewer, I prefer the on device ML proposal with a real world implementation in a consumer FOSS app.
While the topic was considered good, the reviewers noted that the conference this year has received multiple proposals focused on local and secure on-device AI, while the others provided a real-world implementation in a consumer FOSS app with more practical and tangible takeaways for the audience.