Who's coordinating your army of AI agents?

[00:00:00] Ian Jacobs: I've been arguing for a long time that we're not taking seriously employee agent, agent turnover or churn as a metric for success of conversational AI. If conversational AI succeeds, your rate of attrition of agents should go down because you're improving their jobs. And I don't think people are looking at it that way.

[00:00:32] Nikola Mrkšić: Hello, everyone, and welcome to another episode of deep learning with PolyAI. Today, I've got Ian Jacobs from Opus Research with me. Before we start, please like, share, subscribe. But, Ian, thank you so much for joining us on the show today.

[00:00:46] Ian Jacobs: Yeah. Thanks, Nikola, and thanks everyone for spending time with us.

[00:00:50] Nikola Mrkšić: Absolutely. And, look, I think maybe before we go into anything in a lot of detail, what's getting you excited these days in the world of agentic AI and what it can do for, like, customer experience?

[00:00:59] Ian Jacobs: It's kinda funny using the word excited because in a weird way, what's getting me excited is that people keep failing, and I feel like I have a better way that I can pitch them. And I don't wanna be, like, profiting off of people's pain sort of. But one of the things that we hear over and over again and have seen data including, like, a survey from PwC of of CEOs is that they're investing in AI, but they're not actually achieving ROI. And what's interesting, and the PwC survey in particular, is they're not seeing the ROI from cost takeout or from revenue generation. And it was like I don't remember the exact number. 55, 56% of CEOs said that in the 2026 survey. My hypothesis is that in the world of agentic and conversational AI, one of the major obstacles is that every webinar, every podcast like this, every speech you see at a conference, every vendor pitch tells brands, you've got all this high volume, low complexity work that we can automate. Let's start with the easy stuff and automate that. And I'm gonna argue that that's actually a big cause of the lack of ROI. I mean, just on, like, pure economics, Nikola, like, what what human labor are you replacing? You're replacing the lowest cost outsourced labor there, the cheapest labor you have. If you started to do things that were more complex, more multistep, automating an entire real process and then not like a password reset process, but something much more hairy problem like. Now you're starting to talk about, like, you know, l three people in the contact center who are much more expensive. And not that you're gonna replace them entirely, but even if you start to eat away 30% of their labor, that's gonna give you much better ROI than automating a 100% of the outsource person in The Philippines or Colombia or wherever your outsourcing is. Right? So that's what's getting me excited, this idea that we can work with clients to identify the more complex use cases where they'll actually get the initial ROI that's gonna make people go, wow, and open up essentially the the the imagination required to really transform the businesses the way that brands like yours are saying that, you know, these companies can with the right technology, the right processes, the right people, and the right use cases. So that that's kinda where I'm getting excited. There's a lot of nuances to that argument. I mean, I'm being simplistic for the sake of a podcast, but, like, that that's what's getting me revved up these days. And the fact that that message is resonating with people, that that's getting me excited. I don't know what I mean, you guys are talking to clients all the time. How receptive would they be to that kind of message?

[00:04:00] Nikola Mrkšić: Well, look. I think that there's a lot going on. I think that, like, right before the hype really came on, when we were still in our very early days, that messaging worked better. And I think DRI was better because whoever was crazy enough to buy into the idea that AI can automate much of their customer service was already, like, a crazy person. Right? So they were willing to be the evangelist inside their company saying, like, hey. This is coming. And they were right. Like, I'm happy to say if I started the company five years earlier, I would have been wrong for an extra five years. Right? Because I think it did take all arms to really deliver upon the full promise, but, you know, it it just meant that, like, they really had to work hard to articulate why something would work. And now I feel like a lot of vendors and, you know, AI is in everything, and everyone's an AI company and la la la. And because of all that, you kinda got to a point where the narratives have gotten good. Like, the low hanging thing. Like, hey. Think. Tell your board you bought AI so they can get off your back. Because let's be honest, you're not that evangelist. Like, you're there to tell your board and your boss that you bought AI, and they can get off your back for another quarter and another quarter and another year. Right? Uh, I think because of that, you're you're spot on. I completely agree with you. They are doing, you know, like, ITSM scenarios. They're doing, like, simple FAQs and all of that. And at the end of the day, those are really short calls. Right? They're simple. And, like but, I mean, we used to have this term called zero touch resolution. It used to be easier to start with the simple stuff because people couldn't do anything. Right? And it was a way to start without the IT buy in. Right? With start, like, deliver a bit of value, but we pushed it as, like, minimal value. It It was the opposite of that. It was like, hey. It's gonna do very little, but it's gonna get there quick. Right? And, you know, like, that's a lot easier to do still, but, like, where people you know, recently, I think we started working with a tier one global bank, and it was, like, financial health scenarios. Really hard stuff. And, you know, for, like, the the guy who who sold that is one of our best guys. And he was like, I'm just gonna go out there where, like, no one's gonna be like, well, tomorrow I'll be able to do that with OpenAI or with another vendor. Like, he it was just so crazy that no one even wanted to compete for that deal because the average handle time was thirty minutes. And, like, you know, our the chief architect of the company was in a meeting with us. He looks at if you guys wanna try, you know, it's kinda like a medieval quest. Like, if the knight wants to go and fight the dragon, like, here, have a sword. Why not? You're gonna lose anyway.

[00:06:41] Ian Jacobs: You know, if your HD is thirty minutes there, even if you automate half of the call, you've now automated, you know, five password reset calls. I mean, the equivalent, five password reset calls. Right? Or or 15 WISMO calls for a retailer. Right? Like, those are very short. Right? So right. Yeah. Like, that that's the approach. And I I like that you pointed out that in many ways, the start with the easy stuff is kind of just a modern take or a modern gloss on get some quick wins under your belt so that you can free up some investment later down the line at some point. But we are at the point where executives have seen that the simple stuff works. And, of course, it works because there are things we already knew how to automate. Like, we for LOMs, you guys could do a very good job of automating password reset. It's not. The flow is the same almost every single time. Same with WISMO. You look up in the same system every single time the order status, and you flip it back to the customer, and that's that's it. So we're automating that which was already automated and automatable. Right? And that's not any way to transform your business. That's as you're saying, it's sort of a way to say, hey. Look. We did something. We have some quick wins. But I I I'm saying we're we're past that time. Like, we need to be past that time, one, because the tools are gonna allow you to do things that you haven't done before, sell products that you hadn't sold before, sell services, which will require a new way to support those products and services and, frankly, a new way to measure the impact of that. I mean, that's the other thing we're not talking about. Right? If if you automated fifteen minutes of a thirty minute process, you still have the human involved and you have the AI involved, and you probably have the AI augmenting the human and potentially the human augmenting the AI. How do you measure that? We don't we don't have the measurement paradigm across the whole industry for that. Like, we understand how you measure AHT. We even understand, although people don't do it very well, FCR and those kind of metrics, but what what metric for this hybrid work? Like, we need to start to develop that. That's the only way that brands are really gonna start to see major transformation. So that's why I'm excited about it too.

[00:09:11] Nikola Mrkšić: So, you know, as of late, I mean, you've written quite a lot about, like, kinda, like, just the conversational experience orchestration and, like, the agent control plane and stuff. And I've read some of that work. I think it'd be really interesting if you could, like, give us a bit of an overview of that because you mentioned, like, kinda, like, humans helping AI, helping humans. Like, how should people think about, like, the relative importance of automating agent assist, the data flow between those? Like, do you see it working better than before? Or

[00:09:39] Ian Jacobs: Yeah. So let me take a step back since you mentioned CXO or conversation experience orchestration. It's not a radical concept. It's that the idea is that we used to treat conversational AI and conversation intelligence as separate worlds. They had separate buyers. They had separate user within the brands. And often, the conversation intelligence, when it was applied the service process improvement was applied to the human side. Right? So you would do automated quality and that kind of stuff with these tools. My argument, and actually, the industry is proving out that we're right, is is that that was a false dichotomy to start with. Right? Like, if you are a good conversational AI company, you've already proven that you understand how to understand the conversations. Otherwise, you wouldn't be able to have them. Right? Like, you understand the structure of conversations. You know how to analyze them. Otherwise, you couldn't have continuous improvement of your own products. Like, it wouldn't work. Is it necessarily true that if you know how to analyze the conversation, you know, if you're an analytics company, then you know how to have the conversation? Maybe not quite

[00:10:55] Nikola Mrkšić: as much

[00:10:55] Ian Jacobs: of a spin. But it's coming true.

[00:10:57] Nikola Mrkšić: Yeah. Yeah. Yeah.

[00:10:59] Ian Jacobs: Right? And we're seeing acquisitions kinda make that case. Right? NICE and Cognigy, for example, NICE, the analytics world, Cognigy in the conversational AI world, and they're trying to figure out how to bring those together so that they're not two separate products. Then you layer in AgenTek AI on top of that. So now it take action. So what we're actually seeing is that this conversation experience orchestration creates the flywheel or the loop that brands really need to have the conversation, understand the conversation in order to improve the conversation. And, yes, that does mean things like the agent augmentation. And it's interesting that we have this weird world where we're not always clear when we say agent whether we mean human or AI. And in this case, it's kind of irrelevant because it could augment either way. Right? So you could have the typical agent assist kind of version of agent augmentation, the human agent, or you could have an AI agent tackling a complex process that's following some guidance that says we always treat platinum customers well, but we have a policy that says I can't do this thing. Let me go to human and say, can I make an exception? I can't do it because my you know, the the the probabilistic part of me wants to do it, but the deterministic rules that I have to follow says I can't. So you can give me the thumbs up for the exception. And in that way, the human is augmenting the AI. And that's only possible when you have kind of the this combination of the conversational AI technology and the conversation intelligence technology because you need to understand the conversation and be able to analyze it in real time to be able to say, oh, let me reach out to a human, not necessarily for disambiguation of an utterance. That too

[00:12:58] Nikola Mrkšić: But the authority. Right? Like

[00:13:00] Ian Jacobs: yeah. More complex things, like, can you give me the the okay for this exception because we wanna treat our platinum, diamond, gold customers better than we treat Joe Schmo off the street. Right? Like, that kind of thing. So that that's where we see that loop really starting to play, and that's how that data flow really would work. And I do think that the conversational AI companies like PolyAI have seen that for a long time because you've been doing the analytics again for your own continuous improvement. It's just now doing it in real time in the customer's environment to improve their service flow and not just the agent performance so that the agent AI agent says the right thing, doesn't have hallucinations. Like, we're we're moving past that world into feeding the intelligence in to create better service process flows. That that's, again, that's exciting, isn't it? I mean, that that's pretty cool stuff.

[00:14:01] Nikola Mrkšić: No. A 100%. I mean, look. I think that when you just look at, like, the increased capabilities of these models, you know, we look at you know, I just last week looked at probably about 10 of our largest customers and the repeat callers there and what's happening afterwards, where the human journey goes that sometimes we have, sometimes we don't have access to. And, really, it is that same journey mapping. And, you know, for the longest of times, I think we've been liked by a lot of our customers for just, like, honestly telling them, like, hey. Look. We'll reach, like, a limit to what we can automate, not because we understand or don't understand, but because we'll we'll be the annoying people flushing out, like, things that you never really fully settled in your SOPs. And, you know, if you turns out that took me a few years to learn this from when we were very small. But if you say this outright, you warn them, then you show them, then you actually get credit for it versus getting blamed on your technology not working because, you know, you didn't hit this rate or that rate. And what's really interesting is, like, the tax on whether you need a human, whether you're an AI agent or, like, a tier one human agent is really that, like, spiritual, that's for, like, the digital transformation and the, like, you know, canonization of what your rules are as a business. Because, honestly, that person, even if it's a tier three agent, they're just freestyling. So, like, it's really for them the question is, like, who do you trust to freestyle? Right?

[00:15:24] Ian Jacobs: I like that idea of freestyling. Yeah. When we're looking at some of our clients' work in terms of humans augmenting AI, one of the problems that they have today is that they don't have a role for that. They don't have, like, a job rec for what that looks like. They don't know how to manage those people because who do they report to? Sent them up managers for it. For a while, I was calling them kind of coming up with a goofy name, calling them bot wranglers, because back then, we were calling everything bots but poor agents. Right? And they had to wrangle. And that was often about disambiguation of utterances and voice AI or stuff like that. But, also, as you're saying, there's, like, some canonical practices that the brand is instantiated in some policy document somewhere that's been fed into the training corpus of of the the conversational AI AI agent. And that's great, but it makes it very hard for it to freestyle if that's you know, if there's, like, a probabilistic layer over some deterministic flow, and it has to follow that flow. So in that case, clearly, the brands are trusting the humans to freestyle more. I think that that's changing. I think it will I think it will change, but this is one of those cases where you do need some kinda human backup to be there to get that trust level up to the point where you don't necessarily need the same level of human backup. Again, my example of a policy exception. Right? We we won't have human judgment there enough so that it becomes a training set that we can then use to say, okay. For this type of customer, with this type of conversation where we've done this, and in the past three weeks, they've had six different interactions with us about this thing. Sure. Go ahead. You can give an exception. But in these other cases, no. But today, that's gonna be a human kind of inserting human intelligence, human empathy, and human understanding of policy into those, you know, agentic AI flows. But I do think it's interesting that we we are getting to the point where we're actually gonna have to remake the customer service organization to create these roles. Like, what are they? Right? I we, you know, we we could come up with names. BotWrangler was me being silly, but we could have something that sounds more corporate about what that that person does, what their KPIs are, who what their management structure is, and then what their career path is. Because as I kind of described it, it's a self limiting role. You train yourself out of it and then open up the path to the next thing. I think it's one of the reasons, for example, that the BPOs are looking at that as a path for their their agents. Because if they're gonna automate away all of the l one work, what do those people do? Well, maybe we can upskill them to be this bot wrangler thing for the next three years, and then we'll have to think about what the next role would be for them and keep moving them up. And if the BPOs are doing it, brands can certainly do the same thing, maybe not at the same scale because they're managing 50,000 agents or a 100,000 agents, but same idea probably applies.

[00:18:55] Nikola Mrkšić: No. Totally. I mean, like, I've always called these guys kinda the new, like, air traffic control or really, like, you know, when the well, I mean, look. Look. I think, you know, from when we started Poly, and there was a glory days of chatbots, and no one was ever gonna call again. Today, you know, like, voice is king, and you can do no wrong in voice. Exaggerated in both cases. People just want, like, the best modality, and sometimes it's voice, sometimes it's chat, sometimes it's an app. Ideally, like, for the most part, they should just not have the problem to begin with or, you know, services that are too complex, like you said. So they just need that support because they're doing something for the first time. But what I found really interesting is, like, it's really and, you know, the latest monikers, context engineering. Like, those tier three managers, they have context. Right? And they're trusted with judgment over places where the codified word of law in the contact center has not caught up with a better judgment of, like, ten, twenty years in that organization. Right? So to me, like, they're really the ones that are just kinda, like, trusted to, like, guide the agent in those settings. And, ideally, and I think maybe, you know, people are not writing about this enough, It's really how in advance you provision up to capture that context in those situations that creates, like, a higher order canon law for the contact center to move into a higher degree of automation. That's where you get that, like, higher ROI. But people kinda just go, like, is the technology there or not? It's like, are you there or not? Like, how many times have you up leveled your SOPs to get to something that could be automated by or, you know, if you brought in, like, 5,000 new people into your command center and everyone else just did not exist, could they restart it, and how quickly could they get to the same level of performance? Because it's really the same quest.

[00:20:41] Ian Jacobs: Yeah. The flip side of that is that some things haven't changed from your chatbot days in that, for example, there are some use cases where the modern technology right now, PolyAI could automate some of these interactions, and the brands don't want to or shouldn't want to. Right? So the example that I always bring up because I spent ten years at Forrester, we had a lot of insurance clients, and talked to them if somebody had was making a claim on a life insurance policy. They didn't want automation within a thousand miles of the front end of that interaction. Right? Because somebody close enough to that person to name them as a beneficiary on a life insurance policy had died, and they just wanted a human voice. Now it's true. That was before the world of voice AI and eleven Labs coming out with Joe Sympathy or whatever the name of the the voice model is that's supposed to sound the most empathetic and sympathetic. But still, like, there are some things where it's not just that it's like that tier three stuff where we haven't figured out how to instantiate that context and expertise that those people have into the policies of the contact center yet because, right, they're being paid for contact center work anyway very well because they have the human judgment, then the brand trusts them. It's also is the we're still in that world where there are some things you probably still want human beings for and will for quite a while simply because of the nature of the type of interaction that you're having with a brand. I don't think that's changed. I mean, what's changed since the the the early chatbot days is you probably can now automate a lot of that stuff, and it would have been more difficult, you know, ten years ago to automate that. Well, look. I mean,

[00:22:36] Nikola Mrkšić: I think it's, like, too and far between that you get a chance to do it. I think we fur we had the first kinda, like, heavy bereavement workload about three and a half years ago. It's been running ever since. It processes thousands of calls every day out of a workload of a few tens of thousands daily. And we get higher CSAT scores than humans do on that thing. But you're right. People do not believe no matter and, you know, like, this is a public case study. It doesn't matter. People just they won't do this until they have to, until the service levels with humans have collapsed really badly, whatever reason that, you know, life and that content control led to, that then they have no choice but to try. So, like, I I'm fully with you. I think that people just won't take the risk if they don't have to. And then when they have to, they'll take all sorts of risks, and AI can do things, but it doesn't have to. Right? I mean, to me, it's always been like, you don't have to automate everything, and you never will automate everything. And, you know, when you automate everything, there will still be a few 100 people running those. Yeah. It's just

[00:23:38] Ian Jacobs: on your behalf. It's not exactly the flip side, but sort of a a different take on that coming at it from the other angle are the use cases where brands think that they probably need humans, but, actually, you'll see things like CSAT, but also other important metrics higher with automation. And those are the things where there's some sort of social friction that's possible, human to human interaction that's then removed when there's some AI involved. Right? So collections is an example. It's embarrassing. If you have to talk to a human and build out a payment plan, you're more likely to do it. So we've actually seen statistics from customers that promise to pay goes up when there's conversational AI. The amount, it actually is smaller because the AI isn't guilting the human the way that another human would, but amas to pay is, like, a key metric in that industry. Right? It's, like, a huge one, and we see that it's actually higher with AI. And then in the less common use case, we've seen in the, I guess, you'd call it the sexual health world, like the planned parenthoods and others of the world, like, having a conversation with a teenager is very embarrassing. And it's not that you'll get better results. You'll actually get results. Like, the the the teenagers may actually have that conversation with that they would never do with another human being. So that's another area where you you would think it would be the other way. Like, that's one where you always want a human being, but the data kinda shows something different. Like, you have to actually start to analyze the social dynamics of human interpersonal relationships to be able to sort of identify which use cases are gonna really sing for your organization. Yeah.

[00:25:30] Nikola Mrkšić: Yeah. No. No. No. I mean, I think that's completely right. I think that, like, bereavement calls are the flip side of that where, actually, people forget that if you are a contact center agent paid not that much above minimum wage and, like, one in 10 calls or someone losing a loved one talking to you about it, you're not gonna last very long in that job because no one wants to have a job where they have to hear that someone's mother or father, mother's sister died. Like, that's just not, you know honestly, that problem doesn't even have to do with how much you're paid. Like, it's just not something that you wanna do. Because it's not even support. You just hear about, like, sad things. Right? Also, really, yeah, STDs and stuff like that, that's where, like, bots reign supreme. Because how did that happen? Well, you know what? Like, the call is gonna be pretty awkward following the moment that you tell someone about, well, whatever. Right? So, yeah, I found that hilarious when we've had cases of that and more serious health issues as well, like drug addiction and stuff like that where it's just people don't feel the stigma of talking to to a bot.

[00:26:31] Ian Jacobs: And this does have real world business impacts. I mean, look at the financial results of the public BPOs over the last six months. The more traditional customer service lines of business are actually doing okay. They're stable. In some really successful ones, they're growing. The areas where their businesses are shrinking are the ones where there's a huge cost and toll on the human beings in those roles, content moderation, for example, for socials. Right? It it's not just that the AI has gotten better at that, which it has and can actually take over a lot of that functionality. It's that those people churn like crazy, and the BPOs needed on staff psychologists and psychiatrists and therapists. And, like so you can also flip the the equation and turn the mirror inward and say, like, what's the impact on your own people of having those, as you were saying, right, around sexual health? Same thing. Or not sexual health, around bereavement. Right? Same thing. It's like wearing on people. And if you can remove that, it's not just removing, like, the brain dead work. Right? It's like removing the work that that takes us cost on your psyche. Like, that's that's an area too where there's a lot of opportunity for investment in AI to help your own internal employees, like, just have a better job. I've been arguing for a long time that we're not taking seriously employee agent, agent turnover or churn as a metric for success of conversational AI. If conversational AI succeeds, your rate of attrition of agents should go down because you're improving their jobs. And I don't think people are looking at it that way. I think that's an important metric that nobody is thinking about in terms of measuring, you know, how effective conversational AI is. And, obviously, that has economic impacts. Right? Because the the cost of recruiting, hiring, training new people, especially when your attrition rate is topping 50% is very high. So anytime you lower that rate but also, frankly, it's just a more human thing to do. You're making the experience for your own employees better. You're giving them a better job, and that's why they stay. So

[00:28:45] Nikola Mrkšić: 100%. I mean, look, to me, like, one thing that's particularly exciting is, you you know, the contact center is a hard place. Right? Like, sometimes people in contact centers don't get, like, sufficient opportunities. And since we released our coding agent, Run, like, there are people in those contact centers that spend a lot of time, like, just improving the agent and doing an amount of self-service that was previously unimaginable just because, honestly, the previous tools that we and everyone else had were not usable enough by not technical people. And now we get someone going, you know, 1,500 turns back and forth with REN in a month, which is, you know, more than some of my engineers do with cloud code. And that's, like, outstanding because, really, they're the new sales ops to Salesforce are, you know, these guys to us and to kinda, like, conversational AI as a whole. But as they all build things, like, you've written quite a bit about agent control plane and agent sprawl. Like, it'd be great to hear a bit more about your views on just, like, what enterprises are doing, how they're containing this sprawl. Are they containing this sprawl? Like, many anecdotes you have. Yeah.

[00:29:48] Ian Jacobs: Yeah. So the the idea that drove this research was or the conversations that drove this research, the talking to clients who said, yeah. Well, you know, I, just in contact center, am getting pitched AI agents from the likes of Poly, but also the likes of Salesforce and the likes of ServiceNow and the likes of Verint and the like right. So all different components. It could be on the analytics side. Could be on the workforce then workforce planning side. It could be on the interaction side. It could be on the agent augmentation side. And I know that in my organization, we're also hearing from SAP about AI agents to help our back office and ERP stuff, and our HCM vendor is pitching us, and blah blah blah blah blah. So we're getting all of that while developing our own AI agents internally using Cloud Code or whatever tooling we're using to build our own AI agents. So it seemed pretty obvious that they were having the early signs of AI agent sprawl. They're gonna have an army of AI agents who are all, in theory, supposed to work to drive the outcomes that the brand wants. And many of these complex processes cross all of these different little silos that the AI agents are being pitched for. And so what we started to wonder is, what would it take for a brand to think about managing this AI agent sprawl, the AI agent workforce army holistically? What are the shared layers that any AI agent that you have should be able to tap into to drive consistent outcomes that you're looking for in your brand? We came up with five of these kinda shared layers, but I bet anybody with, you know, a good brain could come up with five other ones as well. Right? So I'm never gonna say that this was a census of everything that is required. These are just kinda the obvious ones, especially from a CX perspective, because the lens that those are the clients that we're working with. Right? So that's the lens we're looking at. The first one, it's interesting that you mentioned context engineering. We're calling it journey an intense state, but, really, it's the context. What context do you need to persist throughout an entire journey across every agent that might touch that journey to drive the outcomes that you're looking for? The next one is maybe the best understood layer when it comes to humans and one of the weaker ones when it comes to AI agents these days, and that's identity and consent. Right? The ID and V and authentication. Great. We have voice bio everywhere for humans. What about the identity and consent of the agent as it tries to do something, especially when you have multiple agents that might have the ability to do the same task? Right? Because they have access to multiple different tools that touch different systems. So, like, what is the watermarking here of I was authorized to do this thing, I did it, and you can track it back. Right? The next one is something I mentioned in passing before, which is policy and guardrails. We think about the guardrails piece in in AI. We've been wondering about that not wondering, working at that for a long time now. Yeah. But the policy piece, how do I build a consistent policy layer? Again, that any AI agent, whether it's the one that's managing the work schedule of the contact center agent or the one that's actually interacting with the human customer right now or the AI agent that goes and does some back office task that then feeds data back to the contact center agent, How do you have one policy that they're all following? Right? Again, how do we treat our platinum customers is a very simplistic example of a policy. Right? Here are the rules about what we want. It doesn't mean that if they're gonna specify every single interaction will go this way, but it's more like here is the tenor of every conversation that we're gonna have with our platinum customers, and here's where we're willing to bend and be flexible for them, and here's where we're not, all of that kind of stuff. There's knowledge as well. We're not so much focused on knowledge as a layer, meaning you've gotta aggregate all of your knowledge sources into some giant data repository. Right? Data warehouse, data lake, whatever we come up with Next. I don't know. Data ocean. Whatever. It's not that. It's more the governance of knowledge has to be consistent. Right? I mentioned that the different agents may have access to different tools, while they'll also have access to different knowledge. But what are our rules across the entire company for data freshness? Who determines that so that the AI agent is always using the right one? Or, frankly, we've got three different knowledge repositories that kind of tell them the same thing. Which one is the AI agent supposed to use at that point. Right? So it's the governance around knowledge. And then the final one is something I know you guys have worked on a lot, again, internally, but this also needs to be external facing, and that's kind of the eval. Right? How do you actually ache evaluation and testing and move it from a post hoc but predeployment one time thing, yep, it works, let's deploy it, to continuous throughout the entire process so that it's an endless loop. And, again, when we're talking about an army of agents, it has to not just be focused on a single AI agent and how that AI agent works, but the interaction between the AI agents and the interaction between the AI agents and the humans. Again, my example, augmenting AI augmenting human human augmenting AI. So you have to be able to test all of that. I know that's a lot that I said, but those are, like, the five layers that we came up with that sort of seems like prerequisite if you're really gonna go from, like, having an AI agent or two in your payable organization and one or two in customer service to be having orchestrator agents that are orchestrating 50 different kinds of agents, 50 different kinds of tools, and that's just in the service organization. Then you've got the same in the mid office and the back office and in the supply in HBM and in legal and blah blah blah blah blah. Because all of that comes to play in complex interactions with customers. Right? All of those different organizations, it's not very hard in a b to b environment, for example, to see where a service agent would need to interact with legal's AI agent. Right? That that's not an uncommon use case. But how do you evaluate that interaction between the agents and make sure that it is optimized so that it's always working? Anyway, that's the long winded answer, but, you know, it's work we did We published that a couple months ago, and it was really just a plant the flag in the ground because it's something that we're paying a lot of attention to. And I know one of the things that is the typical question when we talk about this is, well, who do I buy that from? You know, I want me one of them. Get me one of them control planes. Right? And it's a complex answer, as you guys well know, because you operate in an ecosystem and you happen to be particularly partner friendly as a company, but not all of the AI vendors are. Some of the hyperscalers come to mind. Right? Like, they want you to operate inside their own environment or big CRM providers or whatever. So for you guys, it makes sense that, like, you're gonna have to interoperate at this layer. But for some of the big guys, like, if you're a Salesforce shop, Salesforce's sales guys are gonna tell you they can do this because that's how they operate. Like, it doesn't you know, it could be vacuum cleaners. Yeah. We do that. Right? I mean, I'm being facetious, but, you know, that

[00:37:51] Nikola Mrkšić: say more and, you know, like, say anything bad about it. But, yeah, you're absolutely

[00:37:55] Ian Jacobs: I don't even mean it as a bad thing because what they end up doing is, like, they figure out where the demand is, and then they buy or build something. And it takes a couple years, but they're there. Right? And they try and right. So my my point is that this is complex. And for the most part, brands, especially today, are not gonna get this from one vendor. They're gonna be working with vendor who understands how to do the identity consent verification authentication in the human and AI world. Right? You guys will tap in to that authentication because all you need for your AI agents is, yep, we we can check that, you know, this is verified. This this agent has set to do this thing. We don't need to be the one checking it. All we need is, you know, the output of that consent process for examples.

[00:38:46] Nikola Mrkšić: Honestly, that I think that you're absolutely right. And the taxonomy you have is really good because it's kinda like, you know, for different clients, you'll see just, like, where they've made more progress for less. But

[00:38:55] Ian Jacobs: Oh, yeah. Yeah.

[00:38:56] Nikola Mrkšić: Very, very do you hear this almost cracked at all? Because, honestly, at that point, they might as well build it all on their own with a hyperscaler because they're one of the few companies in the world that can.

[00:39:05] Ian Jacobs: Yeah. Very few. That you have to be at the same level of maturity across all of these layers. Right? I that that that's clearly not gonna be the case as a brand. I mean, I know you were talking on the vendor side as well. But, I mean, also for the brand, this isn't one where, like, you get the gold check across each one at the same time. You gotta focus it on the things that are probably causing you the most pain as you start to deploy multiple AI agents first and build out from there. And your provider, whoever that is, is probably, as you're saying, stronger in some of these layers and has a technology potentially built in in a way that you don't even see. Right? It's already there. But you have to start as a brand, you have to start asking the questions about all of these different layers that we're talking about and the tech the topics that are inherent in each of those layers. Again, this is how you scale. Without that, it's gonna be very, very hard to scale to the point where people talk about replacing 90% of their contact center agents unless you've got a 10 person contact center. Right? Like, if if you've got a very large contact center, you're not gonna get there unless because you have those corner cases, the edge cases, the long haul. I mean,

[00:40:18] Nikola Mrkšić: the reason you have it is that you've done something for many years that led you to the point where you have 10,000 people doing that.

[00:40:24] Ian Jacobs: That. That. Just that. Yeah. Anyway, so we're calling that the AI agent control plane. I will point out that there are a couple of vendors out there who also use the control plane terminology. They tend to be focused in the eval and testing world, Right? Because they're saying they're gonna be able to control the agent performance by doing testing. But we're we're I looked at networking. They call it a control plane. They talk about having different layers. I'm like, alright. I'm just gonna steal that concept. So it may not be the name that this end this thing ends up being known by in the in the future, but analysts love to name things. That's, you know, one of the things that we do. So You're

[00:41:03] Nikola Mrkšić: right in this world. Right? Yeah. I mean, computer networking is always a good way to interrupt things from. I think my wife, Randlyn, was was like, well, how does the Internet work? I was like, I've taken many of our records, and I could tell you everything. She's like, go to work. Go to work. I'm like, okay.

[00:41:18] Ian Jacobs: Yeah. Network is a good it is a good thing to steal from, for sure, as a call. But, you know, the one of the keys that we didn't talk about, and this is something that's true, I think, in your more traditional part of your business, is that what's implicit in what I said is that each of those layers has access to all of the enterprise systems and data that were require to power it, the same way that you have to have the right system access for an agentic tool to do anything. Right? It has to be able to access the right thing. And then the AI agents can come from anywhere on top of that. Right? So it could be the commerce agent. It could be the legal agent or whatever it is, and they all should tap in. So I think of this as kind of the sandwich, these control plane layers between the AI agents and the enterprise systems because they, in some ways, dictate the access to the system. If the policy says you can't touch that system, don't the agent won't touch that system. Right? So it kinda sits between it. I have spent time in the tech vendor world, but I never had to create a market texture before. But, basically, that's what I you're you're I'm sure you're much better than that, much better app. I mean, this point

[00:42:33] Nikola Mrkšić: now that we have now that we have Claude, I'm very good at it. But it's used to be run to my to my cofounder for you know, and he he was always the one that that would have to bear that cross. But, yeah, I think we're we're we're running out of time. This was a real pleasure, and I hope we get to do it again. And, you know, I we'll see where this poll gets to by the next time we get to get a chance to have this conversation.

[00:42:56] Ian Jacobs: Yeah. It's possible that there will be some recognition of the need for this control plane. I think the CXO thing is a done deal. People just don't recognize it yet, but the control plane's gonna take a while. I mean, I recognize that. It's it's gonna take a while.

[00:43:10] Nikola Mrkšić: Well, we'll try try and be with everyone.

[00:43:13] Ian Jacobs: Starts you thinking about it and starts, you know, some questions in your head about where you need to go. But, Nikola, thank you so much for having me. I really appreciate the opportunity to be here.

[00:43:23] Nikola Mrkšić: No. It was a pleasure, and always always enjoy our conversation. So, Ian, thank you so much for joining us. To everyone watching, like, share, subscribe, and we'll see you in the next one.

[00:43:37] Ian Jacobs: Thank

[00:43:40] Nikola Mrkšić: you,