What happens if your AI agents get 1% better every day?
About the show
Hosted by Nikola Mrkšić, Co-founder and CEO of PolyAI, the Deep Learning with PolyAI podcast is the window into AI for CX leaders. We cut through hype in customer experience, support, and contact center AI — helping decision-makers understand what really matters.
Summary
An agent that improves every day compounds into a very different agent within months. That's the bet behind Wren. For a few years, the story in AI has been agents that do the work; Wren is the next step, an agent whose job is to improve other agents. PolyAI's new release watches how your customer-facing agents perform in real conversations, finds what's working and what isn't, and helps your team act on it, from a small refinement to a significant fix.
In this episode of Deep Learning with PolyAI, Nikola Mrkšić talks with Arkadiusz Kwapiszewski, PolyAI's Head of Agent OS and one of the people behind Wren, about why this is a real change in how enterprises build and improve their agents. The core idea is that agents should learn from real production interactions. The signal that matters is what happens on live calls with real people, and it beats any amount of extra training data.
They also get into who actually uses it. Wren has put agent development in the hands of people well beyond engineering, and inside PolyAI, 95% of production changes now ship through it. Arkadiusz makes the case for cutting out the middleman: whoever holds the context, a builder or even an executive, can make the change themselves, watch it get validated against real data, and move on.
Wren is also how PolyAI builds PolyAI now, which tells you where Arkadiusz thinks the whole industry is heading.
Watch the full episode to hear how an agent that improves other agents changes the way enterprises build.
Key takeaways
- From a “deist” coding agent to an interventionist one: The first iteration of Wren built your agent from specs and backed off; now it stays engaged — reviewing every production call, surfacing a morning list of recommendations, and implementing fixes you approve with a click. It's an adaptive loop where the agent gets roughly 1% better every day, and one customer saw booking conversions rise 30% just by working through the list.
- Real call data is the antidote to AI slop: Production conversations act as a regularization function no human could provide — you can't review enough calls yourself (about 50 a day is the human ceiling). Every recommendation links the real calls behind it, replicates the issue in simulated conversations, shows the fix working, and keeps monitoring after you approve, so trust is built incrementally.
- Whoever holds the context should make the change: Human-to-human handoffs are the real bottleneck, so Wren cuts out the middleman — builders, call center managers, even executives ship changes directly, and roles flatten into part-engineer, part-product. Around ten customers now exchange 2,000+ messages a month with Wren, 95% of PolyAI's own production pushes go through it, and cheap experiments on 1–5% of calls replace internal debate.
- Self-healing is not self-improvement: Fixing friction — bugs, gaps, frustration — is detectable and can run largely autonomously, while self-improvement is KPI-driven: redesigning flows and experiments against North Star metrics like booking rate, with humans engaged only for business rules and missing content. Like Waymo, autonomy grows as trust does — Nikola bets someone goes “Uber auto mode” within months.
-
[00:00:00] Arkadiusz Kwapiszewski: You tell okay. Build me an agent based on those specs, based on those conversations, and it will configure everything on the platform for you. You don't have to do it manually. You have that option. Everything is readable. You can go and, like, edit everything. You want, but RAN really is the main entry point, and it it handles everything for you. And that everything gets better over time. It watches all the interactions that are happening and makes the system better. Adapt the system. Right? It's an adaptive loop where your agent gets, like, 1% better every day, and that obviously compounds like that's that LinkedIn meme. Right? If you get 1% better every day, you end up being quick math.
[00:00:47] Nikola Mrkšić: Hi, everyone, and welcome to another episode of deep learning with PolyAI. Today with me, I've got, who runs our agent OS team, and we're here to talk about Run, our coding agent, and so much more. But, Harkadis, I thought we could maybe start with just, like, a bit of your background and how you ended up doing this stuff.
[00:01:06] Arkadiusz Kwapiszewski: Yeah. Yeah. So I started at Poly four years ago, which feels like a lifetime in this industry. I was originally part of the deployment team, so I was working on actually configuring and deploying dozens of agents to production, designing them, integrating them, making sure that everything went smoothly for the users. At peak time, how many reports did you have? I think it was about 40. So it was the whole agent design team, which we still obviously have, like, really brilliant people, lots of linguists, lots of people with computer so human computer interaction backgrounds. So, yeah, they're they're still doing fabulously. But now I moved to the project team to actually work on sort of automating some of those tasks that we were doing, making them work at scale so they're not just confined to human experts. But all that knowledge, all that expertise is now available, like, at scale to all of our customers at the click of a button.
[00:02:06] Nikola Mrkšić: Totally. I mean, I think that, like, there's a lot of talk of FDE. I think a lot of our competitors who previously didn't believe in forward deployed work are now rushing to talk about it, to hire people. I think we, as always, are two years ahead. I think, like, where Palantir got it right, we're building Foundry after many kinda like forward deployed deployment is, like, people forget the second part is for the platform to become something that doesn't require hugely, you know, motivated, intelligent people, problem solving ad hoc for every client individually. Like, it shouldn't be that way forever. If it's like that forever, then you're just a consulting business.
[00:02:43] Arkadiusz Kwapiszewski: Absolutely. And I I was always, like, a believer in this deployment product feedback loop. Right? Because all that expertise from the front line, you don't have to bring you have to bring it back. You have to platformize it so it then is available at scale. Right? Like, doing like, reinventing the wheel, doing things individually on each deployment that just doesn't scale.
[00:03:01] Nikola Mrkšić: We've definitely been guilty of that too. Right? I think that, well, you and I are both very motivated for less of that to happen. So okay. Tell us about RAN.
[00:03:10] Arkadiusz Kwapiszewski: So RAN is our coding agent, but it's so much more than just a coding agent. Right? So it has it has basically two ways of interacting with it. You can think of it as your platform assistant helper or or really the the main interface to the platform. Right? You bring come to a platform, agent studio. You say what you want. Like, you know, tell me about the trends for the last week. Build me a new flow for making bookings. RAN will do it for you. But RAN isn't just sort of passive waiting for you to to ask you questions. It also proactively reviews all of your calls, reviews all the interactions, and brings the recommendations so that every morning you open the platform and you see, like, a list of list of things that can be better. Right? Here's a here's a list of suggestions that that you can accept to to make your agent, like, a couple of percentage points.
[00:04:04] Nikola Mrkšić: I mean, one of our customers just posted yesterday saying that booking conversions went up 30. And not in my wildest dreams
[00:04:12] Arkadiusz Kwapiszewski: No.
[00:04:13] Nikola Mrkšić: Did I think as you said, I thought it was gonna be, like,
[00:04:16] Arkadiusz Kwapiszewski: half percent of the time. Yeah. And he just he just went through the whole list, engaged with each recommendation, thought about it, gave a bit of feedback, pushed to production, and and just saw the stats go up. And then was so excited, he posted about it on social media.
[00:04:31] Nikola Mrkšić: I think, like, you know, as I think about, like, the role of AI and who gets to implement it in an organization, I was with a very large customer yesterday talking about this and, you know, big dog dog man in the room started asking questions and then had a few really sharp observations about this, like, application that was designed by our presales team. And his insight was, like, viscerally accurate. I was like, well, no offense, but there are five people in this room from your team who did not communicate that to our team. And that's because they don't have your context. Right? So doing similarly to how you had all that context and were able to start building this, I feel like making good quality use of time by those people who are empowered to make those decisions and just agree to them is where the organization gets like I'm not saying it becomes a one person organization, but the improvements don't go through, like, five committees and debates and risk and governance, and just someone who's fully equipped to understand what they mean can just say yes.
[00:05:27] Arkadiusz Kwapiszewski: No. It is it is true that sort of, like, organizing human to human collaboration is kind of the bottleneck. Right? Because passing all of that context, making sure that the person who's executing understands the vision, all the context, it's it doesn't really scale. Whereas with AI, the the the person who has all the vision, who knows what to do, like, can just say say that and and watch the results happen.
[00:05:49] Nikola Mrkšić: Totally. Well, you know, you promised some religious references to me in the in the run up to this one, but I think one of my main core new jokes in the company now is that I refer to my cofounder Sean as the holy spirit. And that I think at this point, he will only really meaningfully engage with you if your agent can fully outline your worldview around something, and then
[00:06:08] Arkadiusz Kwapiszewski: he will, like, mix it with his and will take the next step. And that it's easier than than talking about it because good talkers tend to dominate conversations. No. Absolutely. And and to go back, I think we should talk a bit more about, like, RAN and and and what it is. Right? Because it is really sort of the next evolution of what a coding agent can be, especially for conversational AI platform. It really is the first time when we can generally call a platform, like, fully agentic, which is very exciting. So, you know, until a year ago, the platform allowed you to build agents, and those agents were interacting with your customers. Right? So you had agents for different use cases. You had hotel booking agents, a concierge. Right? Or or maybe an agent customer service agent that allowed you to check on your order. Lots of different use cases, but but it was like a single agent talking to a single customer at a time handling millions of conversations. Right? Then we have sort of first iteration of Rand, really, which was a coding agent. Right? So, again, here are the religious references that I promised. It was like a day day god of deism. Right? It builds a system for you and and backs off, like, leaves it leaves it running. So you can come to it, and this is all, like, testable. Like, our platform is open, so you can go and play with this. You tell okay. Build me an agent based on those specs, based on those conversations, and it will configure everything on the platform for you. You don't have to do it manually. You have that option. Everything is readable. You can go and, like, edit everything you want, but RAN really is the main entry point, and it it handles everything for you. So it sets up the system. The system then interacts with individual users, but it doesn't it doesn't follow-up on those interactions. Whereas now, what Wren is doing, it's like an intervention is got. Right? It's it watches it it it still cares about the world it's created, about the system it's created. It watches everything that's happening, and it basically performs miracles in order to make sure that, you know, history sort of tends towards improvements theologically and that everything gets better over time. It watches all the interactions that are happening and makes the system better. Adapt the system. Right? It's it's an adaptive loop where your agent gets, like, 1% better every day, and that obviously compounds. Like, that's there's that LinkedIn meme. Right? If you get 1% better every day, you end up being, quick math, several times better over the year. And that's basically that's basically what's happening. Right? Performing these little miracles of, like, finding finding interactions that can be improved, suggesting better design solutions to certain problems, finding knowledge gaps for you, implementing the changes for you. So all you need to do is click, like, approve, and then your agent and agent gets better. Right? And we will we'll also do auto deployments in the future. So, you know, you won't you won't even need to be there to reap the the the rewards.
[00:09:10] Nikola Mrkšić: I mean, really, you are just as team working with REN. Your job is to provide the right context and maybe direct the plumbing of that data in a slightly better way. I think one really interesting thing is the project context feature in there where it's kinda like you look at the recommendations one day and you go, like, over obsessing over, like, I don't know, handoff routes or something like that. Instead, like, I'd really like to see more ideas around how to keep people in cancel subscription flow by whatever or or, you know, recently. I've just seen some really cool stuff with, like, agent memory then being used and targeted by REN to figure out what we could do and kinda loading up your information, figuring out how to talk to you, and then running tests about it. And it's just all this stuff that was so hard to coordinate, implement, execute, and then evaluate that just runs in the background and compounds is actually miraculous.
[00:10:05] Arkadiusz Kwapiszewski: Yeah. Yeah. I mean, I love I love the fact that there's no latency cost. Right? Because the user whenever you have, like, your standard coding assistant, you ask it to do something big. You have to wait for, like, half an hour for it to complete with run. Now everything's kinda ready for you. Right? So it really is as simple as kinda swipe left, swipe right.
[00:10:23] Nikola Mrkšić: Yeah. I mean, it's like when you have an idea, you should absolutely go for it. But the truth is and I think this is where, like, the magic of what we do is maybe hardest to explain, but most important is there are people who live in Cloud Code like you do, I do, although more and more of them are in Iran as well. But do you know how many messages you send back and forth with cloud code in a month?
[00:10:44] Arkadiusz Kwapiszewski: Well, I was on cut leave last month, sir. So you send more than usual? It's not an issue. No. I think it'll be easily in the thousands. Right? Like, I I I think about it a lot, like, how how quickly we, like, adapt to this because, like, my job was completely different, like, a year ago. Whereas now now it's all several cloud code terminals running and
[00:11:07] Nikola Mrkšić: How many how many do you actually use?
[00:11:08] Arkadiusz Kwapiszewski: I think about, like I I limited myself. I found myself, like, you know, the the human attention, I was the bottleneck. Right? So I have a rule where it's like, okay, three or four things at once so I can actually actually actually follow what's going on. Yeah. I have five.
[00:11:22] Nikola Mrkšić: I renamed them every few days just to kinda keep a coherent thread of the things I'm working on.
[00:11:25] Arkadiusz Kwapiszewski: Nice. Now everybody will have these personal strategies, but the reality of work has changed a lot. Uh, and I think this is also, like, an interesting thing. Right? Because we're we're trying to kinda take that experience that we're, like, frankly, like, addicted to now. Right? It it is like, I love this new way of working of of having an idea and being able to, like, delegate it at speed and and just sort of be, you know, in charge of a team of agents. And but but it's still not, like, super accessible. Like, there's only, you know, these these most devoted people in tech companies who are really deep into this.
[00:11:59] Nikola Mrkšić: Yeah. Yeah. Actually The world is, like, a few, maybe point one, maybe 1%, probably not even that, or, like, heavy addicts. And then there are the rest of the world is just like, oh, cool. I can write an email.
[00:12:10] Arkadiusz Kwapiszewski: Yeah. Exactly. And and I think what I see as part of our mission with RAN is to make that experience, like, more accessible, right, to all these people managing custom, like, call centers. So they also get they also feel the magic, but we make it sort of tailored to them. And we you know, the things that that we find we experience as friction, like the overwhelm, the the slop. Right? The making sure that validating all the changes manually. Like, we want to build it into the harness, into the workflow so it it it feels even more magical. Right? It doesn't require much skill, and they can get hooked in the same way on an experience that is, like, even better and just tailored exactly to their needs and to their workflow.
[00:12:54] Nikola Mrkšić: Yeah. I mean, one thing I think about a lot is people talk about slop, and I think a lot of people who are not bought into the new world kinda use it as a disparaging term. It's true that it's tiring to kinda, like, work through these things. It gets like, you're holding a lot of con you know, our context when those are limited too. Right? But I think what what is wonderful about RAN and the way that it works in live deployments now is that the customer calls and the data actually ground and work as, like, a regularization function for the swap. So, like, it's actually not like, the swap is guided very heavily by enough, like, forces and input that you could never provide on your own trying to contain it. It's just too hard. You can't see enough calls. You can't think about enough things. You can't remind them about enough things. Whereas there's this flow of things coming in constantly. Yeah. And, like, it's reflected in the in the tickets it creates. It's
[00:13:45] Arkadiusz Kwapiszewski: That takes us back to, basically, this this whole idea that, you know, part of our role is to make to give brand all the data that it needs, all the constraints, and then surface some of that to the user. Right? So so we can build trust incrementally. Right? They see a recommendation. Brand suggests a change. And, you know, we link relevant calls so they can see at a glance. They can, like, just understand intuitively. Okay. I can see the issue. I see it, like, a one sentence description. I see how it affects real callers. Right? These are these are not made up. These are real calls. You can listen to them. I see that RAN actually already made the change. It used simulation convert simulated testing, right, to to actually replicate the issue. And then And calls it there for posterity so the same
[00:14:28] Nikola Mrkšić: problem doesn't happen again. Exactly.
[00:14:31] Arkadiusz Kwapiszewski: Right? So so it shows you, like, how it simulated the same conversation after the change and and how it's now working way better. Right? So you have the confidence that that the change is real. And then, you know, once you once you approve it, it keeps monitoring the calls. So then, you know, shortly after, you get, like, an okay. Like, I can see that these these type of calls is now improved. You can see the stats going up. So that's exactly the constraints you're talking about. It's it's not slop. It's just intelligence, right, within within that workflow of of improving your your call center assistant. Totally.
[00:15:06] Nikola Mrkšić: I mean, it's superhuman in a way that's just unbelievable. So I was on a bit of a customer tour over the past week and just demoing things around. And, you know, I think you and I are deep in this rabbit hole with probably your team and a few other people. And I just, like, would show people something new. Like, hey. What's coming next? And the whole meeting, we get hijacked and overrun by an hour. Because people start seeing these, like, recommendations, and then they're like, wait. Wait. What is that? Click on that. And then, like, these esoteric things that someone might have mentioned to them once or twice, and then they were forgotten. Because you they could never really quantify the impact of that change. They're like, wait. Wait. Wait. Hold on. Click on that. Click on that. Two of them enabled the post call kinda like tracking because they're like, I want human data in here as well because, like, I just want you to surface these complex, like, phenomena happening in a quote when the agent AI agent does this, and then a human does this. Why did that happen? Try this instead. Tee up this piece of information for the human. People just get so creative. And I think to me, like, the Eureka moment and, I mean, you remember kinda like how we went from having, like, a studio assistant somewhere in the corner behind the feature flag for internals to, like, this is not the home page, and that's it. Like, I think that was a big moment because it's really a gateway drug to having a lot of fun. Right? And that as I look at the data by our customers now, there are about 10 people who have 2,000 messages back and forth with REN every month. Now that's addiction of this.
[00:16:38] Arkadiusz Kwapiszewski: Yeah. And we have had that feedback. Right? They they love it. They trust it. They they see it as coworker, and they have a lot of fun because it shouldn't feel like work. Right? Like, with AI at its best, you just, like, turn ideas into reality. It doesn't feel like work. There is no friction. You're just you know, you you just see your vision happening in real time. Yeah. Yeah. So, Rann, why Rann? So we went on that name, like, back and forth for for a while, but Wren definitely had the most internal support. This is a this is a name we could imagine ourselves saying, like, every day. We could use it as a noun. We could use it as a verb. Right? Can you run that change? Can you ask Wren to do it? It's a proper name. So the original inspiration is Christopher Rand, the the polymath physicist, astronomer, architect who rebuilt London after after the fire of London. Also has some incredible buildings in Cambridge. So there's that local connection. So, I mean, Saint Paul's Cathedral, where he's buried as, like, one of his many other kind of London ones. I I spent at least, like, three years of, like, heavy exam terms in the rent library in Cambridge. And he wasn't a
[00:17:49] Nikola Mrkšić: Cambridge man. He he went to Oxford, but he designed the Rand Library at Trinity. And, like, it just had this, like, well,
[00:17:56] Arkadiusz Kwapiszewski: I guess, anglophiliac feel that polya is very much about.
[00:18:00] Nikola Mrkšić: Then as you said, he was really a polymath.
[00:18:02] Arkadiusz Kwapiszewski: Yeah. Polymath. So that's that's the sort of historical connection. And, obviously, when it comes to branding, we sort of also it it's cute. Right? I mean, it's the name of a bird. Like, there's no hiding that fact then. And we have we have that bird icon in the platform, which, again, also helps, I think, to have, like, a little mascot. Right? Uh, we we we see that with a lot of coding assistance. Yeah. I mean, we've had
[00:18:26] Nikola Mrkšić: many we've had many bird. You know? I think the previous version of our LLM was was Raven, the speech recognizer, owl, TTS, Maca. Right? I think, like, code names for our initial kinda, like, vertical agents to kinda, like, non interventionist. Ones were things like Finch and well, I'm forgetting now. We might have had a Starling and a few others. But yeah.
[00:18:53] Arkadiusz Kwapiszewski: What I kinda like about the the bird metaphor as well in this case is that, you know, you basically do get notifications. Right? Like, you get the notification chime when you come to the platform. It tells you, oh, here are some things you can do. And that also is kinda like being woken up by birdsong. Right? It's supposed to be a positive thing, so it's not alerts. It's just, you know, you get, like, that positive association of, like, a nice sound to Yeah. To come to. And that's also, like, this first experience of coming to the platform is also something I wanted to talk about. Right? Because in the past, I think most people just looked at the conversations review page. Right? So so most most of our external users, call center managers, they they look at analytics. They look at the conversation review page. That's sort of the entry point. They review some calls and then sort of you know, how many calls can you really review?
[00:19:43] Nikola Mrkšić: No more than 50 a day without Right.
[00:19:45] Arkadiusz Kwapiszewski: These these are busy people. Right? Like, they
[00:19:47] Nikola Mrkšić: It's not even just that. I think there's just a limit. Anyone who's worked on I mean, you know this, but anyone who's worked on dialogue, like, 50 calls in. Whoever can do more is, like, superhuman.
[00:19:58] Arkadiusz Kwapiszewski: That it has to be It
[00:19:59] Nikola Mrkšić: gets boring.
[00:19:59] Arkadiusz Kwapiszewski: It gets boring. And and then I think the problem with that is that when you look at you look at the performance of your agent through this, like, super narrow lens Okay. There's a sample of 50 calls out of 3,000 that happened that day. You see one example of friction, and then you sort of like you know, because humans are fallible like that. Right? It's anecdotal evidence. But you're like, well, I want to fix that one example of friction. And then it turns out that it literally happened only once, and it'll make no difference to your stats. Right? Whereas now, the the sort of homepage, as you said, is Wren. Right? That you come to the platform, and Wren is your your gateway. Into the platform, you see the recommendations. And the recommendations are based on, like, thousands and thousands of calls. Like, AI can see patterns in data that no human ever was. Right? Like, we had all that data before. We've had it for years, but it's only now that we can make it properly, like, actionable, and we can make it understandable to humans. And I think that's that's the beauty of it. Right? Is that AI can sometimes lead to overwhelm, but also it is, like, the cure to overwhelm where where suddenly suddenly you can make sense of, like
[00:21:09] Nikola Mrkšić: through the slop until you see the next fill.
[00:21:12] Arkadiusz Kwapiszewski: Yeah. And you you it suddenly allows you to make sense of all the data in the modern world. Right? And and this is also what we want to to like, the experience we want to provide. So you don't have to start with conversation review anymore. You start with the recommendations. You click approve, you know, swipe left, swipe right, and your agent gets better. You can still listen to calls as well, but it it it becomes way more targeted, way more scientific. Right? Because we can assess the impact of every change. So that I think that to me is what what feels super magical as well.
[00:21:41] Nikola Mrkšić: Yeah. I mean, like, I think it's just really interesting, the the whole kind of, like I mean, maybe for the general audience. Right? People used to kinda, like, listen to a sample of 2% of the calls. Sometimes they would listen to calls by the best agents and the worst agents, and they would have, like, these, like, very brittle algorithms that would be like, oh, when people speak to, they say nice words like happy and thank you. And when they speak with Nikola, they're frustrated because they say things like, you know, unbelievable or whatever. Right? And that was just, like, not great. Because the best thing they could do is, like, performance manage me out because I'm a worse agent than you if they had enough agents to begin with. Right? Now I think, like, with this, it's just possible to ask any more of these questions and get an answer. And I also really like the kinda, like, drift of any individual deployment where through the context, they're able to guide, like, okay. Like, for the next few weeks, we're gonna work on optimizing this and this and that. So I'd really like you to dig into the potential things that can be done here or there. And then I think, you know, there are people who would just deploy. There are people who will test. And I think that that's how do you see the future of that?
[00:22:49] Arkadiusz Kwapiszewski: Well, sir, I definitely want to make it more autonomous. Right? So there is there are some really, really interesting decisions design decisions to be made in terms of when to engage a human in this AI loop and and and when not to. When you think about what Wren Wren is, like, our customers in the past sometimes, they watched watched looked at a conversation. They they they didn't, like, anticipate. When the agent did something clever, they they hadn't realized. And they they would ask us, like, is the agent, like, self learning? Is it learning from the interactions? And and, you know, again, until Wren, the answer was was no. And now now it's it's yes. It is learning. It is changing. Every interaction Wren has changes it a little bit, and it optimizes Wren for the actual production calls. So we want to make that up optimization process better and faster. Right? So your agent is really the perfect assistant for your color base. If that color base changes for whatever reason, I mean, like, you know, we have that, uh, the upgrade. Right? The agent will change with it. Right? People start asking things about asking about things you hadn't anticipated. The relationship with your brand changes. Like, we can catch it. We can optimize for that. In terms of, like, when to engage a human in that process. Right? Because it's that loop can be, like, fully autonomous. Right? You you detect friction. You repair friction. That could happen without human supervision. But, obviously, there will always be moments where you need to engage a human to talk about, like, business rules. Right? If there is a let's say, there's the user is asking a question, we don't have any answer to. Well, Brand might be able to find the answer. It might be able to find the answer on your website. It might be able to find an answer in some of the documents that you uploaded to the platform. If it can't, it absolutely should ask you and will ask you. It won't it won't make it up. Right? Sometimes you do we just have to defer to you to help us improve your agent and and fill in the content. There is, like, a really interesting distinction as well between self healing and and self improvement, which is guiding, like, a lot of the design decisions that we're making. Self healing in a way is easier because it's all about fixing friction. And friction is easy to detect. Right? There's some frustration on the call, some gap, a bug, or something, and and and run goes in and and resolves that so everything goes smoothly in the future. Self improvement is very different, and it's it's very much KPI based. Like, you have some kind of North Star metrics, so you want to optimize your booking rate. Because, obviously, this is how you get your return on investment by by agents making bookings autonomously. You could have an agent that that has no defects. It's performing exactly as designed, but it was just designed not in the most optimal way. Right? So, suddenly, it's not about fixing defects. It's about finding better ways to engage a color base to represent your brand in order to to get those metrics to go up or maybe finding ways to, you know, integrate your agent with with your systems better to make it more useful.
[00:26:02] Nikola Mrkšić: Objectives as well. Right? I think that, like, what I found is often the difference between our deals that are, like, mid 6 figures and deals that get to look mid 7 figures is the level of sponsorship you have. But, really, what that means is you move up from just the contact center and into, like, the wider email or chief commercial officer, CMO, someone typically more in charge of revenue as well as just the bottom line. And what's interesting then is, like, the metrics they look at are different. And then if you can connect your agents, and in this case, your fully autonomous contact center, to those objectives, then, for instance, the value of cross selling when people can't book a restaurant here. Like, no. I don't have that time. About half an hour later, I've got something that's four blocks down, same time. That's, like, really valuable. And then you experiment with it, and you book, and then you break through the ceiling where previously, you know, you were trying to you can't really do much. If it's not available, it's just not available. Right? And in theory, that some some of those things were always possible, but that takes connecting it to the wider org. Right? So that's where that human contact center would have been a cross selling one. Now, again, the super humanity of our agents is that they've got time. Right? So they're not comped on how many calls did you take, why is your average handling time too long. Well, if it's longer because I convince you to show up at a different location, I'm a hero. But the metric typically would just show that Nicholas is slow and he chitchats. And, well, then I'm not really incentivized to do it, so I'm not really gonna do it. Right? And then I think when you go up through the org, then, like, the design of, like, should we incentivize people and AI agents to do that or not? And what's the propensity of that thing to succeed? Like, it's just impossible to quantify with humans. The I think the you know, that iterating at, like, the speed of dialogue where you're just gonna have this idea. You don't even have to quote it up in the way that you would with Cloud Code or REN. Right? You put it in the context, and REN comes back to you the next day with, like, five suggestions. It's got a whole team of analysts that went in, came up with these proposals, then a whole IT team that implemented it, then a whole team of, you know, like, continuous optimization people running it and evaluating it. I'm sure they're just calling that conversions up point 8% tick. Accept. Done. And, like, I can't we're not there just yet. But at the rate that this is all going, I don't think we're more than three months away from someone just going Uber auto mode, like, accept it all and and go. Because Yeah.
[00:28:38] Arkadiusz Kwapiszewski: I think I think all the decisions that can be made autonomously should be made autonomously. And then and then we we need to delegate to clients when when we just need to, like, integrate with the systems better or where we need to do something, like, unusual in terms of, like, something out of the box in order to really move the metrics in a more dramatic way. Like, what you said, right, about about Yeah. I was gonna say, yeah. Yeah. So this is, like, you know, you ask, like, what's the future? I think I think improving the self improvement part of it to make sure that we're not just picking the low hanging fruit, but we really think strategically. We really think out the box about like, how can you really, like, change the design, the flow, the conversational flow, the setup of the agent to engage engage people and, you know, convert more users and find these solutions. And and we have a system now if we can if we like, we're we're getting more and more users to our platform, and we we're getting all these decision makers as well. Right? So we now don't have to necessarily go through these loops of human approval. People people who have the full business context can interact with REN directly to make those decisions, And and they will know what's best for the business and for the brand, and then they can make it happen, and we can be aligned with these top level metrics. Right?
[00:29:56] Nikola Mrkšić: Yeah. You know, I think that people always get, like, completely petrified when the CEO has a pull request. Right? And I've done it a few times. It gets people to just reimplement it in the right way because the risk is too high. But where I think it's really interesting
[00:30:11] Arkadiusz Kwapiszewski: brand. I mean, that's kinda what we want. Right? The the CEO has the has the context, and and they have the the decision power. And
[00:30:17] Nikola Mrkšić: Yeah. And I've done like that with the number of people. I'm like, okay. You think that's a good idea? Share screen? Done? Yeah? Cool. My microphone, I just say it. Hit. And then also, when can we expect to see that thing? I was like, have five minutes. And I think that, like, at that point, people are just like their minds are blown because it's like, oh, wow. This is not, like, a transformation project where there's an invoice. And then they're like, I think the dopamine hit goes, I did this. Right? And then it's like, oh my god. Let me do another thing and another thing. And as you said, you know, 1% better every day. And, like, also, they have the context. Right? So I think that putting the future is of, like, all these roles. And, like because, you know, we make it seem like there's this one person at the top, like, you know, pulling all the strings, but I don't think that's really true.
[00:31:07] Arkadiusz Kwapiszewski: No. I mean, it definitely is easier to be, like, one person at the top again with what we talked about with AI, like, allowing you to to actually make sense of all the data, have better overview. Right? But I I think you still need people to, like, go deep and and have all the context of of specific areas. Uh, I do think that this flattening of roles is very real. I mean, I've experienced it myself where, again, my role is, like, part engineer, part product manager. And that just means you can carry the context and the expertise you have through through the entire workflow and just make things happen. Again, it's all about, yeah, having having that context in your head. I think I think I think this is almost the most valuable thing about workers now is just having all the context and knowing what the direction is, knowing what needs to happen, knowing what needs to be built. And then you have all the tools at your disposal to make that vision into a reality. Right?
[00:32:03] Nikola Mrkšić: And I think for the organizationally, the only thing you have to do is find the people willing to, you know, have their skulls collapse under the pressure of the ever expanding context window and empower them. Because they'll just if you empower them to make those decisions, like, they'll iterate, and they'll get there way faster than they can explain. Because I feel like the iteration has become a lot cheaper than the preceding debate. And the debate used to have to happen because it was like, you know, are Mike, Ike, and Peter working on this for a month, or are they working on this? And, you know, we've got five sessions each. We have 10 sessions each. And, you know
[00:32:42] Arkadiusz Kwapiszewski: I think and and that's that's that's super true. Or where you can just they make things happen. And again, with Wren, it's it's so cheap as well. And then a big part of it is experiments as well. So, you know, we talked about self improvement. You don't necessarily always know what will be, like, the best for your users. Right? Like, sometimes the way you phrase a question, like, how many questions you ask, it it makes an like, which voice your users will respond to. Right? Like, you you can't know a priori, like, how that will impact your metrics. So so you basically do have to do experiments. And when being able to conduct these experiments for you saying, okay. Like, I have a hypothesis. I think this user journey can be, like, cut in half. Just, you know, just make the questions, like, less verbose, make it more snappy. I think this will really help users engage. How about we test it out? Like, we can launch it for, like, 1% of calls, 5% of calls, whatever you want. Right? We'll watch the metrics. We'll we'll see how we we'll review the calls, see exactly what's happening. It's, like, so the friction is so low, right, for you. Like, the cost of that experiment is so low, but then you don't have to debate it anymore. Right? You don't that's it cuts through the debate.
[00:33:54] Nikola Mrkšić: I remember one of our largest customers, who shall not be named, had a few voice options and a very senior stakeholder went with one of them. I think the other two had, like, 98% of the votes between our team and theirs. And, like, for years, I had to kinda, like, prod and, like and with, like, experiments, it's just well, everyone contributes their idea, and then you let the data decide. And, honestly, by the time to throw in their idea, I feel like a lot of the egos are satisfied. And then, like, you know, the data will vote, and you kinda forget about it. And, you know, like, you feel good if you if your idea works, but, like, really, if then you just compete harder to add more ideas to it and then
[00:34:30] Arkadiusz Kwapiszewski: That's part of the optimization loop. Right? Like, I mean, this is this is what we're doing as a company. Right? Like, we're iterating on a platform faster. We're pushing to production, and we're letting like, we're putting all the new, like, UI, all the features in front of users to validate them. Right? Because we want to we want real feedback. We don't want to, like, theorize this. And we want our clients to be able to do the same. Right? Like, make changes, see how real people react, validate that against data, and then make I mean, that's the decision scientifically. Right?
[00:35:01] Nikola Mrkšić: Yeah. I mean, I think that a lot of people kinda, like, try to derisk this for companies going, we'll learn from how we're humans talk to this platform. And, like, the only thing I've been able to come up with as an analogy is, you know, we see Waymo's around London now. They've been around the Bay Area for a long time. That took time because people behave differently around Waymo's than they do around other drivers. So Waymos have to, like, collect this data and gradually adapt. And humans over time change their behavior on them too. Right? I think initially they're a bit aggressive, a bit afraid, and then later, it's just kinda comfortable because they know it won't do anything unexpected. In fact, in many ways, it's I don't maybe in all ways, it's a better driver. So I feel like the levels of autonomy there are just, like, increasing. And I think the fact that we have so many advanced workflows just allow you to go look a ride. Can we now do the highway? Can we maybe try the highway at a 120 miles an hour? Because we all know it can be done. Right? It's just a matter of, like, you know, we need to trust it to be right. And I feel like in a fully agentic world, you can do that a lot more easily. And I think we've crossed the chasm of where we now have enough data and the abilities to put it back in the hands of our customers to do it, because they're really always the ones with the context. Right?
[00:36:14] Arkadiusz Kwapiszewski: Absolutely. And and, again, cut the middleman. Like, they have the context. I have run.
[00:36:20] Nikola Mrkšić: They have the context. They have Ren, and they can just make things happen. No. I am super excited about Ren and everything that we're doing with your team. I think that our customers should just expect this thing to change twice a week going forward. And I think that's, like, one of the greatest permissions that we gave ourselves here. So, yeah, I just I'm thankful to our customers for being so excited about it as well.
[00:36:44] Arkadiusz Kwapiszewski: Yeah. I mean, it's it's like the privilege of my life already working on this, and it's also just so much fun. And, again, as I said before, like, my mission is to just share that fun with our clients so they can also, like, experience experience this these feedback loops, experience the growth, experience, like, immediate results with, like, zero friction, and just, like, love using the platform. Like, we want the platform to be to be part of people's routine. Right? They come to it every morning. They click on these recommendations. They watch the stats go up. And they just you know, it's such a good feeling, such a every like, hit of dopamine in the morning with your coffee. I look at it
[00:37:21] Nikola Mrkšić: every day, and, like, I can't believe how fast it happened. Right? I thought it would be just a gradual uptick. And then I'm like, oh, wow. Like, person from a telco in The Middle East and then someone in a software company in America and then in European bank have all done more with REN than I've done with cloud code. I'm starting to feel inadequate.
[00:37:39] Arkadiusz Kwapiszewski: I mean I mean, usually, like, I think 95% of of pushes to production are not done through brand, and it will just go up. There'll be more, and and we'll see better and better results for customers and for everybody calling customer service. Right? We are we have a mission to fix customer service, and it actually is looking really realistic. I think we finally have lightsabers. Right? I think that we were equipped for the finance or
[00:38:04] Nikola Mrkšić: our customers. So I guess we'll report back in a
[00:38:07] Arkadiusz Kwapiszewski: And, again, tackling this problem in in two ways. Right? We have the harness. Like, we we allow you to improve the harness, and we have the model layer as well with dialogue reason one. So, really, I mean, those two improvements sort of happen at the same time, but we we see them both just pushing all the stats
[00:38:25] Nikola Mrkšić: Yeah. Up. Yeah. I mean, yeah, that would be one. I think we'll have a whole different push these weeks. But, you know, it's the world's fastest reasoning model. It's the only one that's smart enough to do these tasks well while being, you know, both quick, reliable, not hallucinating. And, you know, the more people build complex use cases would run and get them into production, the more data we have to make that a reason one, two, three, I don't know what
[00:38:54] Arkadiusz Kwapiszewski: It's feedback loop? Yeah. Feedback loop. Feedback loop. All the way down.
[00:38:57] Nikola Mrkšić: Loops loops yet again. Yeah. Awesome. But, Kadoosh, thank you so much for today, and we'll check back in at, like, three months and see how how much further we we got in.
[00:39:06] Arkadiusz Kwapiszewski: Thank you.
[00:39:07] Nikola Mrkšić: Thank you. It's been a pleasure. Thank you. Thank you all. As always, like, share, subscribe, and we'll see you in the next one.