COSCUP 2026 - Conference for Open Source Coders, Users, and Promoters

要考期中考？AI 沒過也會有這天。開發 Agent Workflow 時要怎麼分辨。表現變好是 Model 的功勞，還是好的 Workflow 帶來的功勞？這時就需要一套好的驗證方法：期中考！但是道高一尺，魔高一丈。你想得到的，AI 也想得到。上網作弊、偷看答案，只有你想不到，沒有 AI 做不到。本節的故事背景，是我們在開發開源 Agent Workflow 時，設計內部 benchmark 所碰到的各種大小事。

How do you evaluate the performance of an agent workflow? When things get better, is it the model doing the work, or the workflow itself? A good benchmark is like a midterm exam. It tells you who actually studied.
But AI is smarter than you expect. It will cheat by searching the web, peek at other answers, and find every loophole you forgot to close. AI always finds a way to surprise you.
In this session, we'll share the story behind building a benchmark for an open-source agent workflow. Every twist, every workaround, and every time the AI quietly outsmarted our test design.