Unbelievable! AI commands Minecraft bots using GPT-4!

I’m going to show you what it looks like to integrate GPT 4 into a Minecraft Bot Minecraft is a great sandbox environment for testing out AI so let’s jump right into the demo so you see I’ve created this little artificial Grove that we can use to test out our bot’s ability to harvest trees I’m going to start by starting up the client and spawning in our first bot there you go you can see Alpha spawned in and teleported to the starting space now I’m going to test out the First Command which is come to player so contrl T to open the terminal then type at and the bot’s name which is Alpha at Alpha slome now Alpha comes to the player now I’m going to demonstrate the GPT 4 integration so same thing contrl T at Alpha but now instead of using a slash command we’re just going to talk to the bot in natural language so come over here now Alpha is thinking GPT 4 has to take some time to process you can see then Alpha came to the player bring him back over here TR T at Alpha SLC come all right now I’m going to ask Alpha to chop down that tree contrl T at Alpha chop down that tree Alpha is thinking and there it goes it’s breaking the blocks of the tree dynamically figures out which blocks are included in the tree and once it’s done breaking all the blocks it’s going to go around and collect all of the dropped wood so there you go it’s collected all the dropped wood and it responded tree harvested I’m going to ask it how many wood it collected TR T at Alpha how much wood did you collect thinking I collected four oak logs which is correct all right now I want access to those logs so I’m going to ask him to put the logs in the chest T add Alpha put the wood logs in the chest there he goes he just dropped the wood in the chest we’ll take a look you see there’s four logs in there now pull those out and we’ll ask him how much wood he has in his inventory now contrl T and Alpha how much wood do you have left I have no wood left nothing left in his inventory all right let’s ask him to chop down another tree at Alpha chop down another tree there he goes finds the closest tree and starts breaking blocks and there he collected the whole tree now let’s test his memory of what he’s done how many trees have you cut down I’ve cut down two trees correct how much wood did you harvest from the second tree harvested four oak logs from the second tree I think that’s correct but we’ll check first I’m going to ask how many logs he harvested total how many logs have you harvested total harvested a total of eight oak logs all right let’s store the remaining logs in the bot’s inventory just going to use the/ store command which does not use Chachi BT detected this chest over here this time and stored four Oak log logs into that chest so that’s perfect that works now we’re going to spawn in two more Bots and have them all Harvest at the same time I’m going to do contrl T at Alpha /spawn and the name of the next bot that we want to create beta beta join the game see she teleported to the starting position I’m going to tell beta to come over here beta come over here she comes now I’m going to put Alpha on the opposite side and lastly I’m going to spawn a bot in the middle at Alpha SL spawn camama and there gamma has teleported to the center I’m going to go up here for a better vantage point now we can see all three set the time back to day so we can see what’s happening and now I’m quickly going to tell all three Bots to harvest wood at the same time Alpha Harvest a tree Beta start chopping down a tree gamma start harvesting there you can see now all three Bots are harvesting at the same time Alpha’s just finished and collected his wood beta just finished and collected her wood and Gamma is finishing chopping down his tree and there he’s collected all of his wood of course this would be faster if they actually had axes but baby steps now we can ask gamma how much wood he collected gamma how much wood you got gamma collected five oak logs beta how much did you harvest harvested five oak logs as well and we’ll mix it up for alpha alpha how many trees have you chopped down total Alpha says I’ve chopped down three trees in total total so that’s correct now I’m going to instruct them all to place their wood in the chests just use the store command now each of these Bots has an internal GPT instance that they’re able to communicate with each has their own thread which means they each have their own memory but as that thread memory that context becomes longer and longer the cost of your API calls is going to become greater and greater per API call so I’ve also built in a function to reset that g GPT context basically creating a new instance so contrl T at Alpha SL reset now behind the scenes we have a new GPT controlling Alpha and now if we ask the GPT how many trees they’ve chopped down they shouldn’t remember chopping down any trees contr T at Alpha how many trees have you chopped down total Alpha thinking I don’t have the capability to track past actions or provide a history of actions taken so Alpha doesn’t have any prior context about trees being chopped down so he just tells us that he doesn’t have any information about past actions so that’s it for the demo you can probably imagine this is just scratching the surface of what these Integrations are capable of for instance you could ask the Bots to harvest a particular type of wood or keep harvesting trees until it’s collected a certain amount of wood and obviously there’s more that we can do to help the Bots communicate with each other and coordinate for instance you might want Alpha to be the commander bot and have him Direct other bots so that I don’t have to type as many commands eventually the goal would be to create a team of bots that can go out and execute a task and just bring back the resources and materials that I need subscribe for more videos like this I’ll catch you in the next one