On Ratchet - a WebGPU-based browser ML framework - and the future of client-side AI

A lot of developments are happening on the client-side when it comes to AI and machine learning. In this talk, we'll explore Ratchet, a browser ML framework that leverages WebGPU. We'll dive into the technologies powering Ratchet, its underlying philosophy, and what the future holds for this exciting project. Additionally, we'll discuss the current state and future prospects of client-based AI, touching on topics like NPUs, ML compilers, and the key players in this new platform.

In this talk, I'll talk about:

1. What Ratchet is: how it works, functionality, and philosophy
2. Briefly touch WebGPU, WebAssembly, and the foundational technologies.
3. Do some code-browsing of ratchet holistically. Go through the crate-level divisions. Understand how a ML framework works and how things are compiled to WebAssembly.
4. Go through my PRs and contributions, small yet fun.
5. The state of Machine Learning on the browser:
6. Touch-base with other frameworks and runtimes in this league with the same objective.
7. Apple Intelligence, Chrome with buillt-in Gemini Nano, Firefox's new `ml` component.
8. Conclusion: the current case and future of client-based AI.
9. Talk about NPUs (neural processing units), ML-compilers, Web Neural Network API, and what the future could look like.
10. A short demo running Phi (Microsoft's LLM) using Ratchet. Look at some GPU utilization graphs. (if there's time)