AI and Games – Minecraft Villages Built by AI – The Generative Design in Minecraft Competition | AI and Games #46

Minecraft: one of the most successful games of all time and continues to be a fun and exciting space for gamers of all sizes all around the world. It’s home to thousands of creators who work together to build massive structures and landscapes or just chill out

And focus on building their own wee home away from home. But what if instead of humans coming together to build these structures both big and small, we challenged AI systems to do it themselves? I’m Tommy Thompson and in this episode of AI and Games, we’re going behind

The scenes of the GDMC – the Generative Design in Minecraft Competition – to learn more about how academic researchers are building AI systems that build entire Minecraft villages by themselves. So grab your Pickaxe and watch out for Creepers it’s time to start start digging…

So before we start exploring the AI systems in earnest, let’s cover the basics of the competition itself. The GDMC has been operating since the summer of 2018 and at the time of this video has just announced the results of the 2019 competition at the Foundation

Of Digital Games conference held this year at California Polytechnic State University. The competition is organised by Michael Green, Rodrigo Canaan and Julian Togelius of New York University, alongside Christoph Salge who is currently a research fellow at the University of Hertfordshire in the UK.

The challenge is straight forward: write an algorithm that can create a settlement in Minecraft. But the challenge comes in the game itself, given this is what we would call Holistic Procedural Content Generation. The algorithms being built and the content that

It creates almost fit together in a way that makes sense. Naturally this includes being flexible enough to adapt to be able the terrain of a specific Minecraft map, but there are a lot of other considerations as well. The challenge of building these settlements is an abstraction of very real problems in

City planning and working around an existing geometric space. Real world settlements ranging from villages to cities are built to respect their local geography, building up the sides of mountains and along rivers, they also typically respect their local climate plus by virtue

Of them being made by humans, for humans, each settlement will features that enable for mobility and access. Pathways, stairs and ramps for pedestrians, roads for vehicle traffic, bridges, tunnels, tram systems and cable cars. But beyond all that there is culture

And a sense of identity. No two cities in the world are alike and even if we break it down to specific regions of the world you can still see how each city carries its own distinct architecture and cultural resonance. If you consider my home country of Scotland,

The two biggest cities – Glasgow and Edinburgh – are completely distinct from one another in the size, composition and architecture despite being only 40 miles apart. So while this challenge originally sounds straightforward, there are a lot elements that need to be considered when we attempt recreate human-like settlements in algorithms

– then we need to adapt those ideas to fit around the rules of the world of Minecraft. Naturally you have some functional requirements: in that the settlement you’re creating will have one or more buildings in a particular space, each of them with walls, roofing, doors

And windows. But then there is consideration for the composition of the settlement: are buildings placed close to one another with paths, is there lighting or any other signposting to communicate the main areas of the settlement. Is your settlement a collection of smaller

Buildings in a region, or a handful of larger structures? Does your settlement achieve some evocative storytelling? While not immediately obvious, this too is something that is derived from real-world behaviour as cities continue to grow and change. Do we visit these locations

And a get a sense of their history? Of how this settlement was formed and how it has grown over time? As areas are built from scratch, destroyed and rebuilt once more to either better fit the needs of the community or in response to conflict. But all of it still

Comes back to the Minecraft map that’s generated upon: does that settlement makes sense in the context of the world it’s been built within? Does it respect local geography? Does it build structures that carve through or work around it?

In order for me to better understand how the underlying API works as well as get an idea of the quality of settlements being built by AI competitors, I was given behind-the-scenes access to the competition by becoming one of the judges for the 2019 competition. So

Earlier this summer, I spent a couple of laid back days exploring the worlds created by these procedural generation systems plus had to subsequently grade each generator per the rules of the competition. So let’s explore how the competition helps participants to build settlements and take a look at some of the settlements themselves.

So I’m going to start explaining the rules of the competition and how it all works, but during that time, I’m going to show you footage I recorded during my time as a judge, as I wandered around each of the settlements as part of the assessment process.

The competition operates by providing a framework through which entrants can take a new or existing minecraft map and then modify it using whatever algorithm you’ve designed. To do this the GDMC relies on MCEdit – a popular save game editor for the Java version of Minecraft that’s

Been around since 2010. As a result, if you want to try this yourself, you need to own the older Java version of the game, given the MCEdit and in-turn the competition don’t support the Windows 10 or console versions. MCEdit allows for the use of filters – commands

That apply a specific behaviour or process over a specified subsection of a MineCraft map. So for the GDMC, your goal is to build a filter that creates a settlement in a defined region of the map. To pull this off, the competition adopts a plugin called MCEdit2 that enables

For one or more Python classes to write custom filters for MCEdit. So a submission for one person, is the complete python code for the MCEdit filter. To help new users get started, there is a sample bot provided that uses Binary Space Paritioning to break up the region into

Lots, followed by Cellular Automata to build the actual structures like you see here. This creates a settlement, but as you can see while it is functional it lacks in variety or any real creativity in its aesthetic. However, it provides a great starting point through which to start building your own generator systems.

Once a filter is submitted, it is ran on three test maps selected by the organisers that the competitors are not privvy to in advance – with the caveat that the filter needs to run ideally no more than 10 minutes to build its structure on a 256 x 256 map. The test

Maps are all pre-generated and ensures a level playing field for showcasing how each generator will handle their unique geometric properties. Once submitted, each settlement for that user is graded in four categories scoring between 1 and 10: – First up there’s Adaptation: does the settlement respect and if necessary deform the local

Environment. Does it look like it has respected the environment but also reshaped it in parts to better reflect the settlements needs. Do the settlements appear to use the materials of the region? Do they reflect their local environment? – Next up is Functionality: does the settlement provide for non-player characters and even

Players. Are there paths that allow for easier movement around the space? Are there bridges across rivers and other dangerous geography? Does it protect its citizens from danger? Such as keeping mobs out of the region or other environmental hazards? Is it easy for

Citizens to grab food? Even something as simple as providing street lamps or other lighting to help guide players is a functional consideration. – What evocative narrative exists within the settlement? Upon visiting this settlement do we get a sense of story, of the people who live there, the history that has shaped

Their lives and how the settlement has reacted and grown in conjunction with that. Do you understand upon seeing the settlement what its function is? Or it’s culture? Plus is there evidenece of how these things might have evolved over time?

– Lastly, there is aesthetics: where we consider how the design of the village evokes a particular set of design rules: be it the scale and proportion of buildings, the specific types of structures that exist and how they all relate to one another. Is there consistency to these rules?

And does it all look quite appealing to look at? One all grades are complete, the final grade overall is the average of these four categories, averaged out across all judges. Now the thing you’ll notice is that each of these metrics

Is rather subjective in nature and that’s part of the challenge: the basic elements of the buildings are rather straightforward to assess, but how it all comes together into an interesting settlement is a larger issue and that is not something that humans can

Readily agree on, much less writing an algorithm to do it for us. If you recall way back in episode 4 of this series, where I looked at the Mario AI competition, the same rules applied to the level generation track in 2010, where the winning levels were deemed those that

Were the most interesting to human testers. In total there were 11 judges involved in scoring the maps for this years competition from a variety of corners of games development and academia. In its first year of 2018, the competition had four submissions, with another five being

Submitted for the 2019 run. Congratulations are in order for this years winner – Filip Skwarski – whose generator has successfully won the competition for the last two years with a 2019 tally of 5.5. However, it was a very tight competition, given Julos14 scored

Not far behind at a grand total of 5.39, with 3rd place going to ArtCodeOutdoors a generator built by Lisa and Bryan Carter, scoring 4.95. All of the source code and resulting maps from the competition are now available to download from the GDMC website for you to try out and play with yourself!

In a recent round-up by the competition organisers at the FDG conference, they were happy to report that the work had seen significant improvement over the last year, with more use of pathfinding algorithms such as A* to build paths between buildings, while the buildings

Themselves are typically built from templates. Templates enable a level of consistency to the buildings being created, as well as allow for variation in each case, with some generators placing household items like chairs, tables and beds in each of the rooms. These templates

Are also increasingly more diverse, with the likes of windmills, watchtowers and market squares that are all being placed based on specific rules being employed within the generation systems. In addition the efforts taken to accomodate for terrain has improved significantly over the first year.

Despite that, there is still plenty of room to grow, which is healthy for the competition. The GDMC now seeks to grow upon the success its had already and is currently gathering feedback from its participants on how to proceed with the competition through into 2020. That

Said, it’s in a great position right now for both university undergraduate university students as well as grad school students seeking a project to sink their teeth into. Plus even for hobbyists there’s plenty of scope to build and explore some weird and wonderful ideas.

For anyone out there with a passing interest in building mods for Minecraft or programming in Python, this is a great opportunity to explore. Congratulations to everyone who submitted to the 2019 competition, the work is making real strides and is pretty fascinating to explore. While the 2019 competition has just

Wrapped up, the competition is still ongoing and the tools and documentation are available for anyone to try their hand at making their own Minecraft settlement generator. I hope by watching this video, some of you are keen to get your hands on the competition tools

And try building your own Minecraft settlements. Be sure to visit the GDMC website, where there are links on how to setup your Minecraft installation and get started writing your own generators. In addition, there is a wiki for the tools being used as well as a Discord channel for

Anyone keen to try it out. I’ve included all the relevant and other materials in the video description and encourage you to go and check it out. It would be great to see even more submissions for the competition in 2020.

Thanks for watching this episode on the Generative Design in Minecraft Competition, be sure to like and subscribe to help spread the word as I release new videos on the world of AI and Games each and every month. Special thanks to Christoph Salge for giving me a behind-the-scenes

Look at the competition and helping make this video a reality. Plus a special thank you to my brother Darren Thompson – who is an avid Minecraft fan – who helped me record some footage of the game for this episode. And of course thank you to my crowdfunding

Supporters on Patreon who support me to make these videos as well as my YouTube members who sponsor the show too, with a special shout out to RetroReversing.com, Nick Alves and Aaron Martin. To join our community, watch episodes early and get to vote on the topics

On the show, check out the links on screen now and in the description. Stay safe, have fun, I’ll be back.