Microsoft Unveils Gen AI Model

Microsoft Corp. has introduced Muse, an AI tool capable of generating video-game scenes that traditionally require human programming and animation.

Developed using data gathered from Xbox gamers and their controllers, Muse is the first AI of its kind, according to the company.

To create this model, Microsoft’s machine learning research team, led by Katja Hofmann, consulted with game developers to understand how generative AI could best support their needs.

They then trained Muse using seven years of gameplay data from Bleeding Edge, a 2020 multiplayer title from Xbox’s Ninja Theory studio.

Hofmann said in a demonstration for reporters:

“If you haven’t seen any of this generated gameplay footage before, you might think these are just clips from a video game.”

The model was officially unveiled in Nature on Wednesday.

Satya Nadella, Chairmand and CEO of Microsoft, also posted on X (formerly known as Twitter).

Potentially Bringing Old-School Xbox Games to Modern Hardware

Microsoft's Xbox team, along with other industry players, is exploring AI to help curb the escalating costs of developing blockbuster games, which have ballooned in recent years despite slowing sales growth.

However, the move has sparked controversy, with concerns from players about AI potentially compromising game quality, while studio employees, already facing layoffs, fear further job reductions due to automation.

Bleeding Edge, a 2020 online multiplayer game, served as a data source for the AI model, with over a billion game images and corresponding controller data gathered from recorded gameplay sessions.

This data trained what Microsoft calls the "World and Human Action Model," which generates game visuals based on human-created scenes.

Hofmann noted:

“The model is able to generate something that is really consistent, that demonstrates a very accurate understanding of the actual game world.”

The AI can produce in-game interactions, and external researchers will have the opportunity to build upon Microsoft's work with prototype software set for release.

She concluded in the blog:

“Today, our team is excited to be publishing our work in Nature and simultaneously releasing Muse open weights, the WHAM Demonstrator, and sample data to the community. I look forward to seeing the many ways in which the community will explore these models and build on our research. I cannot wait to see all the ways that these models and subsequent research will help shape and increase our understanding of how generative AI models of human gameplay may support gameplay ideation and pave the way for future, novel, AI-based game experiences, including the use cases that our colleagues at Xbox(opens in new tab) have already started to explore.”