COSCUP 2025

Dive into building the next generation of Mobile and Web applications featuring Offline AI Agents. This talk explores how to leverage the power of Edge AI and on-device processing using open-source tools to create intelligent agents that operate entirely without an internet connection, enhancing privacy, reducing latency, and enabling new functionalities independent of the cloud.

We'll focus on using lightweight, state-of-the-art open models, particularly Google's open Gemma family, which are designed to run efficiently directly on edge devices, potentially leveraging execution frameworks like the open-source MediaPipe (available for both Mobile and Web). Discover how these models form the "brain" of our offline agents. We'll showcase practical implementation using Flutter as a primary case study, introducing an open-source plugin (flutter_gemma) (developed by the speaker) designed to seamlessly integrate models like Gemma within this popular cross-platform framework, illustrating principles applicable across platforms.

Learn the practical steps to implement these agents: from selecting appropriate on-device model variants (like the latest Gemma models, or popular open alternatives such as Deepseek and Mistral Small) based on task requirements, to integrating them into your application architecture. Using Flutter and the plugin as our concrete example, we will explore techniques for structuring agent tasks and touch upon fine-tuning methods. Crucially, we will discuss the trade-offs involved – weighing the benefits of Edge AI versus traditional cloud-based AI solutions for Mobile and Web. Furthermore, we'll delve into advanced capabilities like on-device Retrieval-Augmented Generation (RAG), enabling agents to query and utilize local data sources entirely offline.