Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér. After an incredible day of Apple showcasing their culmination of many-many amazing research papers, and also light transport algorithms applied to soundwaves. Today we are going to have a look At NVIDIA’s new invention that can play Minecraft, but not in the way you think. It truly is a sight to behold as it is able to gather resources for itself, go and mine, catch a fish, these things I expected. However, what I didn’t expect is that can even engage in more complex tasks like building a base, or hunting this adorable little piggy that kind of breaks my heart. Now, it can do 3 more complex things, including fighting, which I will show to you in a moment. But let’s have a look at the paper, and immediately when looking at the article, I was like “what?”. Are you sure? You see, this paper claims to use large language models to play Minecraft. How is that even possible? Large language models are typically adept at Answering text-based questions with text-based answers. So how does it play a computer game? It doesn’t just play the game, but it plays it incredibly well. How? Well, hold on to your papers Fellow Scholars, because it is able to build a curriculum by itself, for itself. You see, It gets some text information about the world it is in, for instance, the items that it has or what time it is and who is nearby. And based on this information, it starts reasoning by itself. That is incredible. Look, for instance, if we have a wooden pickaxe and some stones, then it Is a good time to upgrade to a stone pickaxe. Or, if it’s nighttime and there is a zombie nearby, it is time to grab a sword and shield. Now, these are simple statements, but the key is that this Little AI makes these deductions by itself. We don’t need to program these by hand, and thus, if it were dropped into a completely different game, it would likely do quite well there too. But wait a minute, this is still just text. How does it become gameplay? Well, After creating this curriculum, the AI writes computer code to control the game and achieve its goals. If this piece of code works, it observes what it achieves, and then stores it away in a skill library. These will be the building blocks, for new, more complex skills later. So a text-based chat assistant that can play a video game. That is very impressive. So, how well can it play the game? With a little help in the form of human feedback, it can do three amazing things. One, it can build a house. And I have to say, that is some proper craftsmanship there, or proper craftsAIship. Lovely house. Good job, little AI! Two, it can also build a nether portal. This allows it to travel to a different dimension with unique terrain and resources. And, get this, it can even fight an Enderman. This creature makes short work of an unsuspecting player, however, the AI came well prepared with high-quality equipment, and bam, we are done. Now, these all sound good, but we are Fellow Scholars here, so we are looking for a detailed Comparison of this technique against previous AIs. Is this any better? Oh my goodness. Are you seeing what I am seeing? It can explore so much more than its predecessors, for instance, much more than the fan favorite AutoGPT, a version of ChatGPT that can prompt itself and work on its own. And it Gets better! Wow! Look at that. I can’t believe it! It is more than 15 times faster than AutoGPT. AutoGPT has barely come out a few weeks ago, and it has already been improved 15x already. Wow. Just look at that! What took AutoGPT 75 iterations likely only takes maybe 5 iterations, Likely even less. AutoGPT plateaud at iron tools where the new technique is barely getting warmed up. Incredible improvement in just a couple of months. These text-based large language models can not only write code to play video games, but they can even plan their journeys. And just Imagine what they will be capable of just two more papers down the line. What a time to be alive! Thanks for watching and for your generous support, and I’ll see you next time! Video Information
This video, titled ‘NVIDIA’s New AI Mastered Minecraft 15X Faster!’, was uploaded by Two Minute Papers on 2023-06-09 15:16:35. It has garnered 194880 views and 8692 likes. The duration of the video is 00:06:13 or 373 seconds.
❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.com/papers
📝 The paper “Voyager: An Open-Ended Embodied Agent with Large Language Models” is available here: https://voyager.minedojo.org/
My latest paper on simulations that look almost like reality is available for free here: https://rdcu.be/cWPfD
Or this is the orig. Nature Physics link with clickable citations: https://www.nature.com/articles/s41567-022-01788-5
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Benji Rabhan, Bret Brizzee, Bryan Learn, B Shang, Christian Ahlin, Eric Martel, Geronimo Moralez, Gordon Child, Jace O’Brien, Jack Lukic, John Le, Kenneth Davis, Klaus Busse, Kyle Davis, Lorin Atzberger, Lukas Biewald, Martin, Matthew Valle, Michael Albrecht, Michael Tedder, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Richard Sundvall, Steef, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi. If you wish to appear here or pick up other perks, click here: https://www.patreon.com/TwoMinutePapers
Thumbnail background design: Felícia Zsolnai-Fehér – http://felicia.hu
Károly Zsolnai-Fehér’s links: Twitter: https://twitter.com/twominutepapers Web: https://cg.tuwien.ac.at/~zsolnai/ #nvidia #minecraft