Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: Applying AI Graphics Generators to DF  (Read 2925 times)

alan8325

  • Bay Watcher
    • View Profile
Applying AI Graphics Generators to DF
« on: July 10, 2021, 07:11:31 pm »

Saw this video about NVIDIA's GANCraft AI generating realistic graphics for a Minecraft-like scene and immediately thought of how it could generate scenes from Dwarf Fortress maps first rendered using something like Armok Vision or Isoworld.

https://www.youtube.com/watch?v=jl0XCslxwB0

There are also various AI algorithms that generate images from short strings of text. Would love to see them tried on DF descriptions!

https://www.youtube.com/watch?v=HeUFHX964d4

I wonder if some of these could make it into LNP utilities one day, though the utilities would maybe require access to some cloud compute datacenter where these algorithms can be run in reasonable time.
Logged

Dr. Melon

  • Bay Watcher
    • View Profile
Re: Applying AI Graphics Generators to DF
« Reply #1 on: July 20, 2021, 04:41:40 am »

Out of curiosity, I used this Google Colab notebook to generate an image from a forgotten beast description using VQGAN+CLIP:
https://colab.research.google.com/drive/1_4Jl0a7WIJeqy5LTjPJfZOwMZopG5C-W

Description:
Quote
The Forgotten Beast Tithleth Shodukadag Lurstibmer Angzak has come! An enormous feathered crocodile. It has wings and it belches and croaks. Its yellow feathers are downy. Beware its noxious secretions!

Result (800 iterations):
Logged

Thisfox

  • Bay Watcher
  • Vixen.
    • View Profile
Re: Applying AI Graphics Generators to DF
« Reply #2 on: July 20, 2021, 04:56:09 am »

That's... wow. That's a nightmare. In a good way! Surprisingly good, in fact.
Logged
Mules gotta spleen. Dwarfs gotta eat.
Thisfox likes aquifers, olivine, Forgotten Beasts for their imagination, & dorfs for their stupidity. She prefers to consume gin & tonic. She absolutely detests Facebook.
"Urist McMason died out of pure spite to make you wonder why he was suddenly dead"
Oh god... Plump Helmet Man Mimes!

alan8325

  • Bay Watcher
    • View Profile
Re: Applying AI Graphics Generators to DF
« Reply #3 on: July 20, 2021, 01:51:31 pm »

Out of curiosity, I used this Google Colab notebook to generate an image from a forgotten beast description using VQGAN+CLIP:
https://colab.research.google.com/drive/1_4Jl0a7WIJeqy5LTjPJfZOwMZopG5C-W

Description:
Quote
The Forgotten Beast Tithleth Shodukadag Lurstibmer Angzak has come! An enormous feathered crocodile. It has wings and it belches and croaks. Its yellow feathers are downy. Beware its noxious secretions!

Result (800 iterations):


Nice! Dwarf Fortress is a great source of descriptions for these algorithms. I imagine one day you'll be able to feed one an entire log of combat messages and it will generate a video of the combat scene.
Logged

HexaBlu

  • Bay Watcher
  • Hi! I'm Hexa :D
    • View Profile
Re: Applying AI Graphics Generators to DF
« Reply #4 on: July 21, 2021, 11:41:42 am »

I've been thinking about this exact concept over the past few days, I was literally just about to make a post almost exactly verbatim of this, talking about how cool something like this could potentially be, but it seems like I'm not the only one thinking about how it would be a great idea! (I even had the GANCraft Example already pulled up) It just makes so much sense to me, ascii based games by nature kind of let you use your imagination to visualize different things and how they appear to you, sort of like reading a book and picturing the scenes play out in your head like a movie. the thought that we could have an AI do that on its own for us in high definition -especially with such an in-depth game like this, just hypes me up so much haha

I bet it might be more plausible at this point in time to make some kind of tool that would use this technology to gradually render out a still cinematic screenshot of your fort, but I really hope sometime sooner or later there will be some real-time AI Visualization  :o
Logged

alan8325

  • Bay Watcher
    • View Profile
Re: Applying AI Graphics Generators to DF
« Reply #5 on: July 24, 2021, 01:01:41 pm »

In the GANCraft video I posted it is mentioned that one of the main features of the algorithm is the ability to do pretty much that.

Where other algorithms interpret each input image individually, resulting in a slideshow of widely varying output images, GANCraft can "remember" the previous images as it interprets the current image.
Logged

Nopenope

  • Bay Watcher
    • View Profile
Re: Applying AI Graphics Generators to DF
« Reply #6 on: August 02, 2021, 07:18:28 pm »

A bit tangential but it'd be neat to see GPT-3 generated texts using in-game book descriptions as prompts. Hell, with an API key (or using EleutherAI) you could even make a plugin that displays this text in-game
Logged

alan8325

  • Bay Watcher
    • View Profile
Re: Applying AI Graphics Generators to DF
« Reply #7 on: August 05, 2021, 09:00:52 pm »

A bit tangential but it'd be neat to see GPT-3 generated texts using in-game book descriptions as prompts. Hell, with an API key (or using EleutherAI) you could even make a plugin that displays this text in-game

It would probably need to also consider the events that happened in the time that the subject of the book existed. For example a book about a fortress would need the list of events that happened in the time of the fortress's existence. A book about a dwarf would take the events of the dwarf's life, etc.

A book that shares the secrets of life and death would be a fun read!  :D
Logged

Mohreb el Yasim

  • Bay Watcher
  • ♫♪♫♫♪♫♪♪♫♪
    • View Profile
Re: Applying AI Graphics Generators to DF
« Reply #8 on: August 10, 2021, 10:41:29 am »

In the context of dwarf fortress, the books about the secret of life and death might all be just :
"
The raw files.
"
I mean in their world this is a trough...
Logged
Mohreb el Yasim


GENERATION 24:The first time you see this, copy it into your sig on any forum and add 1 to the generation. Social experime