It's working in the demo. That's never the question. The question comes six weeks after go-live, when a contact center director is looking at a dashboard showing 40,000 calls handled this month and asking the only question that matters: what happened on any one of them?

Most CAIP vendors answer that question with a dashboard. A grid of containment rates, average handle times, CSAT scores, updated nightly. Useful for the aggregate. Useless for the specific call that just went sideways, the one a supervisor got escalated to, the one that will come up in next week's QA review whether anyone has an answer ready or not.

Forrester's Q2 2026 Wave for Conversational AI Platforms For Customer Service scored PolyAI 5.00 out of 5.00 on "AI observability and administration" — the top score on the scale, and the highest awarded to any of the 14 vendors evaluated on that criterion. Cognigy and Kore.ai, both Wave Leaders, scored 3.00 on the same criterion. Sierra scored 1.00, the bottom tier.

That's not a self-reported number. Forrester picked the criterion, ran the evaluation, and published the scorecard. Worth sitting with for a second, because it cuts against the story most CAIP vendors tell about PolyAI, and honestly, one PolyAI has sometimes told about itself.

What the score is actually measuring

Observability, in Forrester's framing, isn't whether a platform has charts. It's whether an operations team can ask an open question about what happened in production and get a real answer, fast, without filing a ticket to a data team or waiting for next quarter's BI refresh.

That's the job Smart Analyst does. It's a feature of PolyAI's analytics and operations layer, and it works less like a dashboard and more like a conversation with your own call data. An ops lead can type something like "show me calls where a customer asked about a refund and the agent didn't resolve it" and get back both the pattern and the actual transcripts behind it. No SQL. No waiting on an analyst. No static taxonomy someone built eight months ago that stopped matching how customers actually talk.

That last part matters more than it sounds. Call center categorization has always been a human problem as much as a technical one: someone builds a taxonomy, callers don't sort themselves neatly into it, and the categories drift out of date the moment call patterns shift. A query-driven tool sidesteps the taxonomy entirely. You're not selecting from a list of predefined call types. You're asking a question in plain language and letting the system find the calls that actually match it.

Supervisor Suite bundles two other pieces alongside Smart Analyst. Polyscore runs automated quality assessment against every call, not a sampled subset. Agent analysis tracks performance trends over time, surfacing drift before it becomes a pattern someone has to explain in a postmortem. Together, the three pieces cover the question a production voice AI deployment actually needs answered: not just "is this working," but "show me exactly where it isn't, right now."

This has already been used in production for exactly that purpose. In at least one enterprise deployment, a team used Smart Analyst to investigate a live quality issue by searching for good and bad calls to benchmark against, and found the problem faster than a manual review would have caught it. That's the honest version of the pitch: not "nothing ever goes wrong," but "when something does, you don't find out from a customer complaint three weeks later."

About that other Gartner number

If you've read Gartner's Critical Capabilities report for this category, you may have seen PolyAI scored 4Q, the lowest quartile, across all four use cases. That's real, and it would be dishonest to leave it out of a piece about observability.

Here's what that score actually measures, though. It's a composite across nine criteria weighted into four use cases, and "analytics and continuous improvement" is one input among nine, alongside things like multimodality, orchestration, and coding options. A single composite score built from nine inputs tells you about product breadth. It doesn't tell you much about any one input in isolation.

Forrester's observability criterion is the opposite kind of measurement: narrow, specific, scored independently. And on that specific, independent measurement, the more recent one, PolyAI comes out ahead of vendors that outscore it on the broader Gartner composite. Both numbers are true. They're measuring different things, and treating a nine-input product-breadth score as a verdict on one specific capability inside it is exactly the kind of category error a careful buyer should be watching for, on any vendor's numbers, not just PolyAI's.

The part that happens before go-live

Observability answers "what's happening now." It's not the same question as "how do we know this will work before it's live," which PolyAI answers differently: through voice-specific testing tooling, user simulators, and persona-based testing that Gartner separately flagged as a genuine differentiator (only Boost.ai offers something comparable). Worth keeping the two distinct. Pre-deployment testing catches problems before a customer ever hears them. Post-deployment observability catches the ones that only show up at scale, in production, on calls nobody scripted for. A platform needs both. Conflating them into one undifferentiated "we have quality" claim is the kind of vague positioning a specific, measured claim is supposed to replace.

Why this stops being optional at scale

SafeRide Health runs voice AI through PolyAI at roughly 1 million calls a month. At that volume, "someone will spot-check a sample of calls" isn't a quality strategy, it's a rounding error. You cannot manually review your way to confidence at a million calls a month. You need a way to ask a specific question about a specific pattern and get a specific answer, on demand, without waiting for someone to build a report.

That's the actual argument for observability tooling, and it has nothing to do with dashboards for their own sake. It has to do with what happens to quality assurance once volume outpaces the number of humans available to do it manually. Every enterprise voice AI deployment eventually hits that point. The question worth asking a vendor isn't whether they have analytics. It's what happens when your team needs an answer to a question nobody thought to build a report for in advance.

See how Supervisor Suite works on a live account. Sign up , build your own agent, and kick the tires on Smart Analyst, Polyscore, and the rest of the suite.