Google trains a GenAI model to simulate Doom's game engine in real-ish time A team from Google and Tel Aviv University have developed a generative AI game engine capable of simulating the cult classic Doom at more than 20 frames per second because research. The work, detailed in a paper published [PDF] yesterday, demonstrates how reinforcement and diffusion models can be used to simulate game engines …
Google says each TPU v5p pod composes together 8,960 chips over our highest-bandwidth inter-chip interconnect (ICI) at 4,800 Gbps/chip in a 3D torus topology. If it took that to make something which looks like Doom running at 20fps on a 486, maybe it could run Crysis on all the data centres in Ireland...?
Or just run Doom in your VR headset and you not only don't need all those TPUs of "AI model" to generate low-res JPEG quality, replacing them by 120 plus FPS of stonking high quality visuals, but you also get to reach all of the game map.
Using Stable Diffusion, this approach is just recreating, badly, the visuals it has been trained on. It isn't creating new 3D maps of exciting new levels for you to explore, it isn't coming up with new in-game AIs for you to interact with, or new plot lines to follow. Those are the features that would make for an interesting holo-deck experience.
Plus all the physical interactions via improved haptics (pressor fields and tractor beams!), full surround sound with thousands of identifiable points of origin, a scent organ with more than a dozen whiffs at its disposal...
"really a proof of concept at this point"
What concept are they trying to prove? That "AI" can be used rip off games as well as books, photos, music, art, speech etc? What next? A proof of concept that an "AI" can simulate the entire process of creating a TV show? (Ok, that last is probably an easy ask, especially if modelled on daytime TV or History Channel type fare :-))
On a slightly more serious note, I'm seeing a lot of research in AI where the real questions asked before starting should be "is this useful? Is it somewhere we really want to go?" rather than "will this get us attention and funding and make us rich even if it's pointless drivel?"
"will this get us attention and funding and make us rich even if it's pointless drivel?"
Seems fairly useful to me, at least as far as the researchers are concerned!
Game engines are just software, though. If the ai can learn to simulate DOOM, does that mean it can learn to simulate other software, too? Or perhaps learn to amalgamate features from different applications into one simulation? Or even acquire new capabilities by watching third party software at work? I could see how that might be useful, if AI with everything is your thing; why bother trying to integrate AI "assistants" into every application ever written, when you can just teach an AI to replicate the software instead? Could it be that we've been barking up the wrong tree by worrying that Ai-written code will make programmers superfluous, when in fact it won't even need to write code, just simulate existing code on demand?
> does that mean it can learn to simulate other software, too? Or perhaps learn to amalgamate features from different applications into one simulation
Following on from the logic of this example, you'll get a result that produces the visuals from the other software without any of the internals. For some cases (playing Doom) that may even be enough.
But I'd question the utility of a simulation of the display from Inkscape as it is used to create a new design: you won't actually get a usable SVG file out at the end, just a simulation of what you get when the SVG was rendered to a bitmap: time to break out the autotrace[1].
Could be fun explaining why the newly simulated Office App To End All Office Apps can't even copy text to the clipboard, as the Stable Diffusion-created output resolutely stays as a bitmap.
There *may* be some mileage in the concepts you describe, but the system described in this article doesn't come anywhere near their execution.
[1] now, training up a neural net to be a good autotrace, turning a bitmap into a *good*, minimalist, SVG, that is a decent goal. Tried a few websites which claim to have done that and they are junk!)
"Could be fun explaining why the newly simulated Office App To End All Office Apps can't even copy text to the clipboard, as the Stable Diffusion-created output resolutely stays as a bitmap."
Given the number of PDFs I've read where each page *was* just an embedded bitmap rather than an on-the-fly rendering of the page content, I suspect such a limitation would be entirely acceptable to some document pushers...
The AI isn't replicating the software, it's replicating the imagery that the software generates*. It's presumably aided by the fact that Doom is conceptually very simple; the game world is essentially a 2D map and the player's viewpoint doesn't change at all in these demos. I don't think it would even handle mouselook in its current state. Try it with Quake...
* I noticed some random bullet hole decals appearing and disappearing at one point.
Dishwashers washed tedious dishes for you, thus saving you the bother of washing them yourself, video recorders watched tedious television for you, thus saving you the bother of looking at it yourself; AI plays tedious games for you, thus saving you the bother of playing them yourself
It's interesting to explore how far this model is effectively and accurately recreating the data it's been trained on. It looks like it is - in which case the various copyright lawsuits have even more solid grounds to suggest that these sorts of 'generative' models are not making unique new output, but re-packaging stuff they have seen with a little additional noise.
The big limitations here seem to be that this isn't really a "controlled" output - no-one is going to be able to ask a machine to creatively come up with a game for them any time soon. It's taken a slice of Doom and predicted what happens next, for a very short time frame. Even when limited to Doom, it's not clear how internally consistent that prediction will be. Wil it render "well known scenes from Doom" that have robust spatial relationships between areas, demonstrate retained state and all the other hallmarks of an actual game? At this stage I'm guessing not.
Still... it looks pretty. It won't take long before someone does the same but for a more modern game engine graphic style and people will claim that "great strides" are being made. We're easily fooled by the shiny stuff...
> Dubbed GameNGen, pronounced "game engine,"
If you have to tell people how to pronounce the name, you should probably have thought of a different name. Although I guess that guidance is because "NGen" sounds nothing like "engine" with UK pronunciation? (At least to my inner voice)
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
