Observability for AI Workloads: A New Paradigm for a New Era

We stand at an inflection point in software engineering. The mental models and tooling we've refined over decades of monitoring and observing systems are proving inadequate. Traditional observability was built on predictability: measure latency, count errors, track resource utilization. But what happens when your system's outputs are inherently non-deterministic? When a "successful" response might still be wrong, misleading, or subtly biased? When the distance between correct and catastrophically incorrect is measured in semantic meaning rather than HTTP status codes?

This talk makes the case for a new observability paradigm purpose-built for the AI era. We'll explore why the shift from deterministic to probabilistic systems requires us to instrument not just our infrastructure, but the quality of reasoning itself. We'll examine new observability imperatives such as model behavior tracking, semantic drift detection, and hallucination rates. We’ll discuss what it means to log or trace an agentic AI workflow, and how traditional tradeoffs of throughput, latency and payload size are disrupted.

The question isn't whether AI will transform how we build software—it already has. The question is whether our observability practices will evolve fast enough to keep these systems trustworthy, debuggable, and production-ready. If you’re facing these challenges and dilemmas, join this talk and let’s figure out together what observability means in this new era.