COSCUP 2023

What open source generative AI models and related code we can use on modern smart phones
2023-07-30, 10:45–11:15 (Asia/Taipei), TR 615
Language: English

Generative models on "modern" mobile phones have the potential to revolutionize the way we interact with our devices. They could be used to generate personalized content, create new forms of entertainment, and improve the accuracy of machine translation. However, there are still a number of challenges that need to be addressed before generative models can be used effectively on mobile phones. In this talk, I will use Stable Diffusion and llama.cpp as main examples talk about how people try to address these challenges. Despite these challenges, there is a lot of potential for generative models on mobile phones. As the technology continues to develop, we can anticipate seeing a wide range of new and innovative applications for generative models on mobile devices.

Generative models on mobile phones have the potential to revolutionize the way we interact with our devices. For example, they could be used to:

  • Generate personalized content, such as news articles or recommendations for products and services.
  • Create new forms of entertainment, such as games and virtual reality experiences.
  • Improve the accuracy of machine translation and other language processing tasks.

However, there are still a number of challenges that need to be addressed before generative models can be used effectively on mobile phones. These challenges include:

  • The need for more powerful mobile neural network accelerators / processors. Generative models are computationally expensive, and they require more powerful processors than are currently available in most mobile phones.
  • The need for more efficient algorithms. Even with more powerful processors, generative models can still be slow to run on mobile phones. Researchers are working on developing more efficient algorithms that can run on mobile devices without sacrificing accuracy.
  • The need for better user interfaces. Generative models can generate a wide variety of creative content, but it can be difficult for users to control the output. Researchers are working on developing better user interfaces that make it easier for users to interact with generative models and control the output. However, we won't touch this issue in the talk.

Despite these challenges, there is a lot of potential for generative models on mobile phones. As the technology continues to develop, we can anticipate seeing a wide range of new and innovative applications for generative models on mobile devices.

Here are some specific examples of what we can anticipate in the near future:

  • Personalized news and recommendations. Generative models can be used to generate personalized news articles and recommendations for products and services. This could help users to stay informed about the topics that are most important to them and to find the best deals on the products and services that they need.
  • New forms of entertainment. Generative models can be used to create new forms of entertainment, such as games and virtual reality experiences. This could open up a whole new world of possibilities for gaming and entertainment on mobile devices.
  • Improved machine translation. Generative models can be used to improve the accuracy of machine translation. This could make it easier for people to communicate with each other across language barriers.
    These are just a few of the many possibilities that generative models offer for mobile phones. As the technology continues to develop, we can expect to see even more innovative and exciting applications for generative models on mobile devices in the years to come.
Difficulty –

Skilled

 Target Audience –

programmers who want to know the potential challenges of making generative AI on mobile devices
Koan-Sin Tan

Koan-Sin Tan is an old programmer, who learned to use “open source” stuff on VAX-11/780 running 4.3BSD before the term “open source” was coined. He is interested in running neural networks on edge devices the past 7 years. He is a TensorFlow contributor. He converted Stable Diffusion to tflite format and wrote some glue code for it [1].

[1] https://github.com/freedomtan/keras_cv_stable_diffusion_to_tflite/