Towards Self Improving Autonomous Agents

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:10

   So two days ago, I'd ordered food from a food delivery app and they kind of messed up the order. So I raised a complaint and I was asked to upload an image or a video of what went wrong with my order so that they could fix it. And this was a chatbot that I was talking to, right? And I tried uploading a video, but it kept failing. It kept asking me to retry multiple times. And then I gave up and I said, "You know what? Let me just up upload a photo

2. 0:43

   instead." So then I uploaded a photo and within seconds my issue was resolved by the AI agent. And then that got me thinking. It was able to do it so quickly with the photo, but why did it struggle as much when I uploaded a video of, you know, the exact same product, right? And that is where I think autonomy plays a big role. This AI agent, this chatbot that this food delivery app was using was able to

3. 1:18

   process the the the pixels in the photo, but it couldn't understand it couldn't figure out how to operate and analyze the video, right? Imagine if it could imagine if it actually had the autonomy to be able to build its own tools, if it could build its own agents, and more importantly, if it could even update its own source code, right? And all of this in production, it wouldn't fail as easily. it could be more resilient and we would have agents that we can trust and with very little overlooking.

4. 2:00

   But is that really possible? So today I'm here to answer the primary question of how we can a can it and b how can it how can we build agents that can create their own tools in runtime and deploy them and work with them. My name is Sandra Subramani and I work at AWS and we have this product called Strand's agent which is open source fully open source and it's an SDK which you can use all of your

5. 2:38

   different models with which can do this level of autonomy. Now I want to talk to you more about it and give you a quick teaser of how this would work in production. So I'm going to jump right into it. This is just the simplest way of executing it. Just pip installs trans agent and we want to give it an access to one of the pre-built tools called shell. Right? How does this work? I'm going to quickly run it and show you how this works.

6. 3:10

   Right? So here I'm just asking it what can you do and we see that it's got access to shell so it's just running all of this like how it's supposed to be from the directory right so next step what we want to do is because we give it access to shell and the directory we can go on to building self-extending agents right now these are agents which can use tools and build tools that are in the directory

7. 3:40

   and then deploy them at create them and deploy them at runtime. Right? So what I what I do here is I give it an example of what a tool looks like. set up all of the arguments and show it as an example and then create just give it the system prompt which is this and then give it access to shell which is the only tool it has access to and then we also say load tools from directory equals to right and let's watch a quick video of how this works now in this video

8. 4:14

   all I'm giving is like I said just go create a tool and then in uh we set up the tool definition and then we create the system prompt. We give it access and we just say create five new tools. That's all I'm giving it, right? That's all the instruction that it has. We run it and as you can see it will directly go to the empty directory which has no tools right now. And you can see it creating these tools on the fly within

9. 4:42

   seconds. It's creating your calculator tool and your password generator tool, your text analyzer tool. It's creating all of these tools in seconds and after it's creating them is also executing them. And I did not redeploy this or have to reload load all of this. It's doing this on the fly because it's a because the only thing I said was hey create new file new tools and it's also

10. 5:07

    convert and it's also doing all of these calculations on the fly. So today if in that app that I was talking about we built it with strands it could have directly identified that the person that was talking to the app is sending a video as opposed to a photo. It would have gone and created that video analyzer on the fly and it would have used it on the fly. It would have processed my video on the fly and it would have given me the result. So I

11. 5:40

    wouldn't have had to do multiple retries and have to finally upload an image. This is the power of strands agent with just the meta tooling capability. Right? Just give it access to do these tools. But this is just a sneak peek. This is just the teaser. What it can also do which I don't have time to talk through today because I've only been given 10 minutes is that these agents can also create a multi- aentic system. It can orchestrate it. If it's comp it's more

12. 6:09

    complex than just hey is just the tool missing or is the capability that's required is it more complex right? So it can create multi- aentic systems and it can also update the system prompt by itself and update the source code and so in that way there's you don't have to perfectly define the scope of your agents in production. you know that it has the ability to evolve, to become more automated and to learn from its own mistakes, right? And in production, give

13. 6:43

    it access to memory, right? Um because at AWS, we use bedrock agent core for most of our production, we can also give it that additional memory capability. We can have it learn from its previous interactions and deployed into production quickly. This is amazing. But the one thing that I want to talk to you about also is talk about guard rails because it's really easy to say hey

14. 7:10

    let's just go create an agent and give it complete access to your directories but before you give it access to your shell we need to ensure that all of our guard rails are in place. So I want to also just talk about and put it out there so that your developers just don't go into production and say hey do whatever you want build whatever you want. We want to make sure that the environment is constrained. We want to

15. 7:33

    make sure that it's in a sandbox execution. We want to give the right AM access. We want to make sure that we limit the type of tools that it creates, the version controlling of the tools. We don't want it to go back and delete any tools. So, we want to give it the right set of constraints and also increase observability. We want to be able to track what is happening and how and how well we can actually control the working

16. 8:04

    of our agentic systems in production. Um this was just a teaser. I want to also show give you resources where you can read more about the tools and the different capabilities of Strand agent and I'm available offline if you want to have a deeper discussion on how can we build not just meta tooling with strands but meta agents themselves and move closer towards actual AGI and autonomy where agents can really and truly think for themselves.