COSCUP 2026 - Conference for Open Source Coders, Users, and Promoters

Large Language Models may start in frameworks like PyTorch, but getting them to run efficiently on CPUs, GPUs, and specialized accelerators increasingly depends on compiler infrastructure.

As machine learning systems have grown more complex and hardware has become more specialized, the compiler layer has become a key bridge between model code and machine execution.

This talk introduces AI compilers from a practical perspective, covering why they matter, what kinds of problems they solve, and how open-source projects such as LLM MLIR and OpenXLA/XLA fit into the broader stack.
It will also introduce a bit about tinygrad, which is trying to take another approach rather than LLVM, to compiler models running on the hardware..

It also explains how ideas such as intermediate representations, lowering, and backend targeting connect frameworks like PyTorch to real hardware.

A small demo based on PyTorch and various compilers (e.g. LLVM and tinygrad) helps make the overall pipeline more concrete and easier to understand.