The first DND campaign I ever played started in a floating supermax prison nestled in the northern mountains. This was such a cool introduction to a fantasy world that I decided to recreate that prison for the purposes of this guide. Here is an example of the outcome achieved:
This looks pretty ominous and cool, but how good was the art to begin with? Here you go:
So how is it done?
I used stable-diffusion to build these images. The installation can be OS specific so I won't go in to details on the install here. Instead, check out the wiki guide on the subreddit r/StableDiffusion.
The AI is good, but it's not perfect. I've had the most luck giving it one thing to focus on at a time. In this case, I want to create the backdrop for the prison first and then later I will go in and add the floating building.
For this part, I just went into Photoshop (Paint, Gimp, etc. work just as well) and created a rough draft of what I wanted the outcome to look like.
With the amateur sketch done, I took my image and loaded it into stable-diffusion's img2img generator. This takes an image as the input and combines it with a prompt to produce a new output.
You can play with the settings a bit. I turned the Denoising Strength up high and kept the sampling steps reasonably low. It's better to give it more freedom in these early stages so that it can take your prompt and run with it rather than sticking too closely to the sketch.
Settings used:
ddim_steps: 50
denoising_strength: 0.9
sampler_name: DDIM
target: img2img
prompt: Digital fantasy painting, mountains, cloudy, dark night, highly detailed, snowcapped mountains, evil
This looks much better than my stupid sketch, but it's not there yet. I took this output and fed it back in to the model to give it a better starting point.
It's time to enhance.
I dropped the strength now that there is a better starting point. It may be necessary to run the model a few times until a desirable output is achieved.
Settings used:
ddim_steps: 50
denoising_strength: 0.6
sampler_name: DDIM
target: img2img
prompt: Digital fantasy painting, mountains, cloudy, dark night, highly detailed, snowcapped mountains
For this section, I used the txt2img feature to generate a prison. With no source image, I just played with the prompt and strength until a suitable prison was generated. You can adjust how closely the model will follow your prompt if the model is getting a little too creative.
Prompt: Digital fantasy painting, floating fantasy prison, levitating castle, highly detailed, dark
I just pasted this prison on top of the generated landscape to use as the source image for this next step.
I fed this back in to the img2img model. Depending on how close you are to your desired output, you can start turning the strength down and the sampling steps up. You can also dial up the resolution at this point if your GPU has the memory for it.
Settings used:
ddim_steps: 50
denoising_strength: 0.8
sampler_name: DDIM
target: img2img
prompt: Digital fantasy painting, floating fantasy prison, highly detailed, mountains, dark night, by greg rutkowski
Note the by greg rutkowski in the prompts. This is an important step. For some reason, assigning an artist in the prompt really improves the quality of the output.
The great thing about this approach is if you don't like the output, you can always just run it again and get something new.