Building Local, Secure Gen AI apps on Mobile

As use cases for Generative AI permeates to edge and mobile devices, there are a lot of great use cases for building vector search / RAG applications running directly on users mobile devices. One of the common concerns with using cloud solutions for LLM inference is data security, privacy and latency.

This session addresses all of these concerns by introducing open source tooling capable of running fully fledged Generative AI privately, securely and on device in Android such as Onnx ML framework for embeddings, Gemma open source model for LLM, Sqlite Vector search.

So the biggest takeaway for the audience is learning tooling and how to build on device Gen AI applications for mobile with detailed code examples to build secure AI applications that run completely locally on users devices.

Benefits to the Ecosystem
The biggest use-case for the mobile and AI dev community is the fact that the same models can now run efficiently on device thanks to optimizations like WASM, Onnx, and just better open source SLM support. The speakers come with building real world use-cases for customers which shares success stories for running on Device AI applications