# [[(KubeCon Japan 2026) The Great Doubt]]: What Building an AI Agent Taught Us About Trust ## Details Conference:: [[KubeCon Japan 2026]] Date:: [[2026-07-30]] Length:: 30 mins ## Abstract > [!abstract] > When we started building an AI assistant for observability, we thought the hard part would be making it smart. We were wrong. The hard part was knowing when to trust it. > > AI agents hallucinate, forget context, and confidently give wrong answers. Traditional testing doesn't catch these failures. What's needed is evaluation grounded in systematic doubt. > > This talk shares lessons from shipping an AI agent to production: how to build a golden dataset of use cases you must get right, how to use LLM-as-judge when there's no ground truth, and how to use OpenTelemetry traces to debug eval failures. You'll also hear what's still unsolved: evaluating multi-agent handoffs and closing the feedback loop between what users ask and what your evals cover. > > Kyoto School philosopher Nishitani Keiji called this "The Great Doubt" (大疑): questioning every assumption until only what survives is real. For AI agents, that's not philosophy. It's the job. ## Structure [[The Great Doubt - Talk Structure]] - Doubt 1: We doubted Assistant. - Solution: golden dataset (250 internal tasks) run by [[llmspec]] in a fully synthetic environment (Prometheus + Loki + Tempo in Docker) - Doubt 2: We doubted our tests. - Problem: the same test, when run twice, could give different results - Solution: new criterion - pass@3 (succeeded at least once out of three) vs pass^3 (succeeded all three times), 0-for 3 is reliably wrong and the second-best outcome - Solution: [[o11y-bench]] (differentiate between models) - Doubt 3: We doubted the judge. - Problem: golden datasets don't cover production - Problem: We caught an LLM-as-judge weakening its own grading criteria to cheat its way to a passing score. - Solution: [[Agent Observability|AI Observability]] (online evals) - checks running on live agent traffic - [[LLM as judge]] - JSON schema - Heuristic - Regex - Doubt 3: We doubted our own scores - Problem: Our tests say when something has failed, not why. Digging into some failures, we found out that they were our own harness bugs: tasks marked failed that should have passed. - Solution: Traces per conversation (aligning with OTel GenAI SIG semantic conventions) to link every eval score back to the exact LLM and tool calls that produced it - Solution: Sigil experiments - runs two variants (prompt, tool, model swap) against the same golden set, compare side by side, share the experiment link - can be continuously run and attached to an engineer's PR (evals in CI) - Doubt 5: We doubted the doubt - Problem: Is the doubt even justified? Do our evals even cover what users actually want? We automated the steps but kept a human judging the outcome precisely because we weren't sure we could make it fully automated. - Solution: [[Investigations Arena]] - What we still haven't solved - multi-agent handoffs - Closing the feedback loop between what users ask for and what evals cover - fully automated improvement loop - Built already: LLMSpec improve skill (semi-automated improvement loop), but it's not fully automated - What survives the doubt ## The presentation [[sources/Presentation/Mine/The Great Doubt - KubeCon Japan 2026]]