Can you build a custom AI agent in 10 minutes?
About the show
Hosted by Nikola Mrkšić, Co-founder and CEO of PolyAI, the Deep Learning with PolyAI podcast is the window into AI for CX leaders. We cut through hype in customer experience, support, and contact center AI — helping decision-makers understand what really matters.
Summary
In this episode of Deep Learning with PolyAI, Frank Ferro sits down with Ruari Phipps, Senior Forward Deployed AI Engineer at PolyAI, to show — live — what's possible when developers get direct access to a voice agent platform. The conversation is part discussion, part demo: Ruari builds a working hotel AI agent from a single PDF spec, pushes it to a branch, runs automated tests, and merges it for deployment in under ten minutes.
This episode answers the question: what happens when you don't need to choose between speed and control?
Together, Frank and Ruari cover:
- Why developers were constrained before tools like the Agent Development Kit were introduced
- What changes when you pair ADK with tools like Claude Code or Cursor for building and iteration
- How AI coding tools enable parallel testing: running ten simulated conversations at once instead of manually typing through each one
- What the live demo shows about going from zero to 80% with AI coding tools — and where human tuning still matters
- How PolyAI's years of best practices are embedded directly in the ADK's documentation and scaffolding, so developers aren't starting from nothing
- What building flows, entities, and functions looked like before — and how much faster things move now
Frank and Ruari produce an agent that handles knowledge base Q&A and a multi-step booking flow, plus is ready to deploy across voice, chat, and SMS — all from a blank project and a one-page spec. It's a clear illustration of how far we've come, and how quickly we can build.
Key takeaways
- The ADK unlocks what the platform kept sandboxed: Before, everything had to happen through the platform — custom scripts, external data sources, and bespoke integrations were impossible. The ADK is a Git-like CLI with pull, push, branching, and merge, bringing a real software development lifecycle to agent building without changes hitting live until they're tested and merged.
- AI coding tools take you from zero to 80%: Paired with Claude Code or Cursor, boilerplate like knowledge base topics spins up in minutes — work that used to take a day. Human tuning takes it the rest of the way, jumping in to fix the hiccups an initial version always has.
- Parallel simulated testing is the hidden unlock: Coding agents spin up multiple versions of themselves to run ten test conversations at once, across chat and simulated voice, using dynamic wordings that push the agent harder than static test scripts — catching edge cases a single human QA-er never would.
- Best practices are baked into the scaffolding: PolyAI's years of deployment experience live in the ADK's documentation, templates, and validation, so developers — and the coding agents they use — aren't starting from nothing. A flow that would have taken an hour of manual configuration comes together, validated, in about two minutes.
-
[00:00:00] Ruari Phipps: Before the ADK existed, you were sort of all locked into the platform, and everything you had to do was through there. So if you wanted custom scripts or be able to integrate with the other ways you want to build, it was impossible. The ADK allows you to sort of really interject and sort of build these custom scripts, these extra logic, these other tools to really improve your workflow that you weren't able to before.
[00:00:37] Frank Ferro: Welcome to deep learning with PolyAI. We're here to shed light on the latest and greatest developments in AI for CX leaders. I'm Frank Ferro, director of product management here at PolyAI, and your guest host for today's episode. Before we get started, a quick reminder to subscribe, give us a like on YouTube, and leave a five star rating on your podcast app. Today, I'm joined by Ruari Phipps, senior forward deployed AI engineer here at PolyAI. Ruari works at the intersection of engineering and deployment, helping our enterprise customers get the most out of our platform. He's been one of the engineers closest to our agent development kit or ADK since day one. Today, we're gonna dig into what ADK is, why we built it, and we're gonna do a live demo writing real code against the PolyAI platform. This all speaks to a bigger theme in agent development right now. How do you give developers the control and flexibility they want to build? The ADK is our answer. And to prove it, we're going to build a working agent live in about ten minutes. So, Ruari, thank you for joining me today.
[00:01:39] Ruari Phipps: It's great to be here.
[00:01:40] Frank Ferro: Great. Well, let's touch on some of the time savings that we get from ADK. Before this tool existed, why was it so much more difficult for developers to ship quickly?
[00:01:49] Ruari Phipps: Before the ADK existed, you were sort of all locked into the platform, and everything you had to do was through there. So if you wanted custom scripts or be able to integrate with the other ways you want to build, it was impossible. The ADK allows you to sort of really interject and sort of build these custom scripts, these extra logic, these other tools to really improve your workflow that you weren't able to before.
[00:02:16] Frank Ferro: There's a spectrum of control in enterprise AI platforms from fully managed to fully open. Where does ADK sit? And from your point of view, why do we design it this way?
[00:02:26] Ruari Phipps: The ADK was designed initially for our internal teams to sort of speed up how we're building these agents and sort of delivering it sort of as a service thing. As we sort of want to open up the platform and give more people access to the tools to build their own agents, it sort of felt intuitive to allow them to do this. The ADK gives them a lot of control in how they do this. Move fast, write functions, write topics. It really enables these developers and sort of opens it up a lot more.
[00:02:57] Frank Ferro: It's been pretty incredible to hook ADK into some of these tools like Claude Code and Cursor. What's your experience been using those?
[00:03:05] Ruari Phipps: It's been really good. I think they work really well if you can just sort of present the information as sort of stripped down as easy as possible, and the ADK allows you to do that. We've spent a lot of time writing documentation and sort of learning how to use these tools with new prompts and features to sort of be able to get them to work really well with with the ADK and really construct some new bots and sort of features a lot faster than we ever would have before.
[00:03:33] Frank Ferro: Yeah. Maybe an example of where before you were writing in code, and now we're able to
[00:03:38] Ruari Phipps: use something like Cloud Code, how quickly were you able to do that? Especially for, like, the simple sort of boilerplate stuff, it's excellent. You know, you wanna add sort of the knowledge based stuff, and it's, like, oh, what's some example questions? Ping his 10 straight away. Even for more complicated features, it works really well. A lot of the time, stuff that you just you have to spend time sort of making sure you've configured properly, it can spin up in the matter of of minutes and have everything linked together much faster than we would have before.
[00:04:08] Frank Ferro: Yeah. I've been using it myself. I've been able to spin up multilingual agents in languages that I don't even speak, running it through iterations, tests, and simulating conversations, which have led to really great experiences when, um, sharing those agents with others.
[00:04:26] Ruari Phipps: Yeah. I think one of the best things about a lot of these tools is they're able to sort of spin up multiple versions of themselves. So if you wanted to do 10 different tests instead of you having to manually sit and and call or sort of type away in in a chat widget like you used before, it could just go out and do them all at once and and get back to you really quickly. It's sort of a real real time saver and can get you really good insights much faster than you would have before.
[00:04:54] Frank Ferro: True. True. Not only on building the agents, but the testing piece, I think, is one of the major unlocks here as well. The simulated tests from an omnichannel perspective. So now being able to both test in text through chat or through actual simulated voice has really unlocked some speed and given us the ability to test more edge cases as well.
[00:05:18] Ruari Phipps: Yeah. Absolutely. If you sort of can give these bots when it's testing just some license to just, you know, have some fun as much as they can, just play around, come up with new wordings rather than static wording for your tests, it could really make these testing much more dynamic and sort of, especially when you're putting these voice agents in the world, nothing's ever static. And so it's like having that flexibility where it can really sort of push your your agent and make sure it's up to sort of production ready is really useful.
[00:05:49] Frank Ferro: Yeah. And I think the ADk really enables that with these tools. I've been able to create multiple personas and to test different cases that I, as an individual QA er, would not have been able to catch myself. I think that's been really exciting. You feel like when you're going live, there's less of a gap between your first launch and a really great agent.
[00:06:13] Ruari Phipps: Yeah. Being able to go, like, just create from nothing to 80% with these coding tools is really, really useful, and you can then sort of use it to tune, or you can sort of get into the weeds yourself and start working on it. But there's sometimes magic.
[00:06:29] Frank Ferro: Yeah. I think that's a perfect segue into doing the demo where we can show people what ten minutes really does because I do think we can get from zero to 80% very quickly and then give users the ability to fine tune to get themselves to production.
[00:06:46] Ruari Phipps: Yeah. Absolutely.
[00:06:48] Frank Ferro: So let's do it. Let's pull it up. For those of you listening to the audio, we'll narrate this as we go, but head over to YouTube to see the code and agent in action. And if you're as excited as we are and wanna share with your development team, send them to poly dot a I slash developers for everything they need to know. Alright. Perfect. Ruari, wanna give a little bit of background to our listeners and watchers out there on what it is we're actually showing today.
[00:07:13] Ruari Phipps: So we're gonna show how you can sort of use the ADK and sort of a coding agent like Claude to sort of spin up an agent from scratch, really. So I've got here sort of a pretty much empty workspace. I've set myself up with a little sort of demo hotel that we're gonna sort of try and build out. This is just sort of a spec that you sort of may just get from sort of a client or your own business, and it's just, like, just covering the basics. With the ADK, an agent can use this and really sort of build something sort of surprisingly fleshed out from sort of very, very little detail initially.
[00:07:50] Frank Ferro: Yeah. And this is something that the platform allows you to pull from a website, from different documents. So it's pretty simple to get the information in there, and I think it only really needs what every business already has created.
[00:08:03] Ruari Phipps: Absolutely. So I'm gonna quickly set up the ADK. It's incredibly simple to install. It runs on the latest version of Python. So we're just gonna set myself up here with a quick Python environment and sort of activate that. It uses Python three fourteen as this is sort of how we run our agents and the functionality we use. So, and that should be that, and we can sort of get it there. Once the ADK is installed and all set up, we can easily view all of the commands it can do. The ADK is just a CLI tool, pretty much, and it works quite similar to Git. You can pull. You can push. We have branching, so you can work on different features at once. And there's lots of different ways we can interact and sort of edit locally to make sure everything's going well.
[00:08:56] Frank Ferro: Yeah. And that's quite important, I think, as an enterprise ready offering. The software development life cycle and what we provide when it comes to branching versions, the ability to push to sandbox prerelease and live really makes users feel comfortable with the changes that they're making. None of this is actually gonna hit live until you're comfortable, you've done tests, and feel like it's ready to push forward.
[00:09:17] Ruari Phipps: Yeah. Absolutely. I've already set myself up with, uh, an API key that I can generate on the platform. And once that's there, I can just initialize the project and sort of pull in this blank empty projects that I sort of created for this hotel. And it is just all of the default parts of the projects. There's sort of nothing filled in. We don't have any functions or tools. Knowledge base is empty. The sort of main agent prompt is just the default thing. It's all sort of unspecified. So it's sort of really a blank canvas for us to sort of get in and make some changes.
[00:09:56] Frank Ferro: For the users out there, just so you know, nothing here is set up beforehand. It's all from scratch, so it's just a brand new project.
[00:10:03] Ruari Phipps: Yeah. We'll open the the project, and we're just gonna open Claude and get it to have a look. We sort of set ourselves up with a small Claude skill just to sort of teach it how to use the platform. The CLI has sort of tools and information that sort of can really teach these agents how to do it. The Claude skill is quite simple. It's just sort of showing, oh, use these commands, use this to learn, and sort of how to do the workflow. But once it knows that, it can easily use the ADK and the documentation provided to build. So we can sort of get this set up here. We'll say, use Poly Hotel PDF, uh, set up my agent. And I think what we want here is to use the PDF example, and we'll just set up initially some knowledge based topics because that's fairly simple. And create knowledge based topics. Now it's you start thinking and sort of exploring everything we have here with the ADK. We've sort of given it permission to run the commands and have a look. And look, it's just read the PDF, and now it will start looking through everything. Here we go. It is starting to read the documentation and sort of understanding what it needs to write. Now within our internal teams, we often are working on several projects at once, and we find Claude and other agents, especially when you have a good knowledge base of information, how to set it up. It's referencing other projects. It's looking at other things. So this is something that the more you tune and the better you build your agent when you're adding new things, it can really sort of, like, learn the best practices based on what you're doing. Coming from scratch, we the documentation, we do have sort of some of our best practices and some of our ideas on how to set up a good agent, which we hope would really help getting you set up initially.
[00:12:04] Frank Ferro: Yeah. I think that's a really important part here, Ruari, is that a lot of this is possible because of the work that we've done here at PolyAI over all the years to get here. So the best practices, the frameworks, the foundation, the architecture that allows for this to be so simple.
[00:12:21] Ruari Phipps: Yeah. Absolutely. And then here it is. It's starting to set up some topics here, and we can start seeing what it's creating. So it's starting to build out these knowledge based topics, say, oh, this is sort of what people might ask to trigger it. This is the information and sort of telling how when the agent is running, how it should answer these questions. And it's just looking through the PDF we gave it, and it's working this out themselves using its own knowledge of, like, of what a customer agent should be, and it's really able to produce this a lot quicker than I would have been able to type it out.
[00:12:54] Frank Ferro: Yeah. Before this, Ruari, it would have taken you definitely a day to have done something like this.
[00:12:59] Ruari Phipps: Yeah. It's sort of all in the platform. You're sort of manually typing it out. Maybe you can sort of come up with prewritten stuff, so you're copy pasting it in. Even before all of the ADK all of the Claude tools, and we just had the ADK, this was much such a bigger thing because you can start doing automations without these AI tools. Maybe you wanna pull from a spreadsheet or different sources of information and sort of inject it directly here. You don't have to use these Claude tools, but they're really good at sort of setting this up.
[00:13:31] Frank Ferro: That's a great point. It's really up to the user on what they like to use, if they wanna use the ADK directly or if they wanna use one of these AI coding tools.
[00:13:38] Ruari Phipps: Yeah. So here, it should have set this up. It's run some validation to make sure everything is configured properly. It sort of checked everything and made sure it's all okay, and I think we're gonna ask it to push it up to agent studio because currently everything's locally. Say, uh, create a branch and push this, please, and get it to do that. You can push, That should we've done correctly. And now so once it's there, we can start to chat with it. Uh, chess and test, please. So it's set up as about 10 or so different topics, and then we're just gonna get it to start running some test conversations.
[00:14:14] Frank Ferro: So really you've gotten that control too. So not only do you have the speed, but you also have the control and comfortability that what you've done, you can test right away.
[00:14:22] Ruari Phipps: Yeah. And that's that's really important. You can test it on your branch, so you're not affecting anyone else's changes if someone else is working on the same projects. You can easily just push onto your branch, test there, iterate, and make sure everything is working as expected. Here we go. It sort of outputted all the tests. Everything is working correctly. If we wanna sort of check this ourselves, we can easily do this. We'll go into the project, and we can start a chat here, like, oh, look. It's got the correct assurance at the beginning. Can we cancel booking? And, yeah, it's all there, so it's, you know, Claude isn't making that up. It does actually all work. It is able to tell them about the about the cancellation process policies. So it's all good, and you can interact with it here as well.
[00:15:08] Frank Ferro: That's great, Ruari. And we're not gonna get into it today, but it is possible to just very easily set up integrations here as well. So to actually create a booking in a system is quite possible with the ADK as well.
[00:15:19] Ruari Phipps: Yeah. It's definitely it's using all its knowledge, especially Claude and and stuff is trained to make code and building out these API integrations and these complex sort of ways of interacting with sort of your systems, it can do that. You just tell it. You go, okay. I'll build it like this, and it's able to do that itself. I was also gonna quickly show building something more complicated flow for this. This would be a booking flow. We're not gonna build it into integration, but it's able to do something more complicated than just, like, one turn features for the agent.
[00:15:53] Frank Ferro: We're using it to build live production agents, so it goes much deeper. But, obviously, in ten minutes, we wanna just show you the scaffolding here of what a flow would look.
[00:16:02] Ruari Phipps: We're telling it to build a booking flow. We'll say, collect all the information, confirm with the user, then a hand off. Probably here in a booking flow, you'd want it to call the API to do it itself. But if this is the situation you want and hand off, that's sort of what we'll do here. Flows and functions are a lot more complicated, and you have to stitch together a few things. But this is something that the ADK has been created to easily validate, make sure everything is set up is correct, and sort of teach these sort of coding agents how to do it well and sort of present everything in a way that it can easily see.
[00:16:40] Frank Ferro: Yeah. And flows are our way of introducing control to give you that fine grained ability turn by turn to do exactly what you want the agent to
[00:16:51] Ruari Phipps: do. Yeah. Absolutely. Tell it it can it can make these changes. So it's sort of here, it's quickly produced this entities file. So this is everything that it it wants to collect. Sort of these are used throughout the flow. So at the different points, it'd be like, oh, at this point, we need to make sure we collect all these things, and having these entities allows the agents to sort of analyze the user speech and extract the information that we need to use for these API calls and sort of process this information correctly throughout the call. So we go. It started writing functions as well. So this is a function that will start the booking process and trigger that. It's also starting to build this flow.
[00:17:34] Frank Ferro: So, Ruari, before this, you would have written this by hand in code?
[00:17:38] Ruari Phipps: Yeah. With the ADK and without the call, we would have to manually create all of these step by step, or we would have been on the platform dragging out these boxes, typing out these prompts ourselves, and sort of doing it there. Internally, now we have, like, a whole library of how these work, and we've sort of tried to get billed the ADK so it can represent that the best and add on to our docs how it should construct these flows. And we're making that library available
[00:18:07] Frank Ferro: to users of the ADK so that they don't have to write their own code. They don't even have to go drag and drop, grab templates. It's all accessible through the ADK and easily implemented through a coding tool like Claude here.
[00:18:20] Ruari Phipps: Uh, definitely. So here, it's produced this flow. It sort of correctly fixed some of the issues that it sort of did on the first thing, and now we're gonna get it to push and do some tests. Oh, it's not even asking me. It's just starting to test itself in the background. And, yeah, this is, what, it's two minutes, and then we've got a booking flow, which it doesn't look like it's got a huge amount of steps, but it's just making sure everything is sort of set up and correct and filled in would have definitely taken an hour or so beforehand. And on the scale that we're building agents where the flows can be super complicated or it's just the number we're doing, having this speed up is really good. One of the best things about some of these agents is just the way it can input data. So here, we've come from a PDF, but especially for flows, which are a lot like if you've seen, like, flow diagrams and sort of decision trees, those sort of stuff, you can just show it a diagram, and this is what we wanted to do, and it's able to sort of produce it. So if there's other places that you want to have this Claude Code and these other tools able to really read this as well. Here we go. It's sort of telling the end to end things and talk about everything's done and all pushed. And, yeah, it says everything's pushed on the branch and it's ready to merge. Let's do a test of ourselves and sort of just verify it there as well. Um, I wanted to book. What date is this weekend? What other room would you prefer? Classic room. And my guests just be me. And there we go. I mean, I think it may submit, for example, Friday eleventh, Saturday twelfth. What day is it? Yep. We'll say that. Yes. It's once a specific day is Friday eleventh and twelfth. So there is some hiccups with the initial version, but this is where you can easily jump in and sort of go, oh, you know, this is not right, and manually tweak it or get Claude to tweak it, and it is really good and sort of being able to interact with it that way. That's all all done. We could say, yep. All good. We're ready to merge. And the ADK should merge this all in. It's gonna show me a quick diff before it happens just to make sure everything's cracked. You can merge, and it will just quickly run the merge. And now once it's done, it's ready to deploy straight away, and that's all ready to be used by you know, if we want to start pointing agents to all customers, they can go straight away now.
[00:20:53] Frank Ferro: Yeah. It's incredible, Ruari. And so this agent would actually power our omnichannel experience. So you'd be able to call into this agent, chat with this agent, receive SMSes from this agent as well if we set that up on one of these flows.
[00:21:06] Ruari Phipps: Yeah. Absolutely. At the moment, this is just configured for voice. But if you wanted to have it, yep, via web chat, the configuration is there. If you want it to be able to do both, it also gives you that flexibility, and it can make those edits through the ADK.
[00:21:21] Frank Ferro: Yeah. It's incredible. So from here, you could also build in multilingual functionality as well right off the bat.
[00:21:28] Ruari Phipps: Yeah. And as you said, you can build it out for, I don't speak Japanese. But if we wanted the agent to speak Japanese, I'm sure we could get Claude to do some translations and make sure it's all set up correctly.
[00:21:40] Frank Ferro: Yeah. Absolutely. And those are some of the great tools that PolyAI and Agent Studio provide to users. All that's inside of the platform and already there for you.
[00:21:50] Ruari Phipps: Yes. Absolutely.
[00:21:51] Frank Ferro: Ruari, this has been incredible. Seeing an agent go from zero to actually demo ready and not too far from getting ourselves to production with a little bit more tweaking, adding some integrations, etcetera. I just wanted to thank you. Is there anything else you wanna share with our listeners and viewers out there?
[00:22:08] Ruari Phipps: Not too much, really. The ADK is sort of constantly growing and expanding to the whole of sort of the PolyAI's toolkit and sort of really evolving as the products evolves. It sort of really helped us integrate with our systems and build bigger testing frameworks and our own sort of methods that weren't possible without this sort of being able to interact directly with the platform and sort of add the sort of tools and facilities that just having it all sandboxed away. It was just it wasn't possible, but but now it is, which is is really good to have.
[00:22:44] Frank Ferro: Yeah. It's really exciting. And I'm looking forward to seeing what everyone's going to be able to build with this new tool. So reminder to everyone out there, please review and subscribe to podcast, and go out and check out the ADKk. It's at poly.ai/developers.