Recipe for Gameplay Assets with AI
Dernière mise à jour : 3 nov.
In March 2023, we conducted an early test to see if we could speed up the process of creating gameplay assets from an existing art style and integrate them into our game engine swiftly. The outcome was promising, and now it's hard to tell which characters in this battle scene were crafted by our artists and which were generated from their work.
The generated assets were good enough to be used in gameplay scenes.
The results matched our handmade visual style.
Our recipes allowed for a wide variety of assets.
ControlNet effectively controlled the sprite shape while fine tuning handled the style.
Background removal features are effective but not flawless.
Fine Tuning with Original Concepts
We began by creating a fine tuning file (LORA with Khoya_ss GitHub link) using 20 of our handmade creatures. This file helped maintain style consistency across generations. The time-consuming part was ensuring the captioning of each original image matched the prompt format for later image generation.
Iteration with Various Settings
In the stable diffusion WebUI (GitHub link), we tested different settings using various samplers, models, VAE, negative and positive prompts, adjusting the number of steps, and tweaking CFG scales until we could reliably generate good-looking creatures.
The X,y,z plots extension in Automatic 1111SD webui helped visualize variations in settings.
Prompt Variation with Dynamic Prompts
With the right inference and implementation recipe, we could quickly generate new creatures by changing a few words in the description using a dynamic prompt dictionary (GitHub link), which adapted to the character elements defined in the game design.
The percentage of quality assets generated depends on your recipes.
Specific Shapes with ControlNet
Using ControlNet (GitHub link), it was fairly easy to apply our creature style to existing reference shapes to bring more variation to the sprite anatomy and align with specific animation skeletons we planned to create.
ControlNet has an option to provide reference image in batch
After a background removal (Adobe link) and a basic clean up in Photoshop , we managed to get a decent sprite for our game’s battlefield. We tried automatic background removal with Automatic 1111, but the remaining artifacts were too many.
Several removing background AI features have been added to photoshop recently
Adding the sprites and turning them into real characters with their own stats was straightforward thanks to our existing RPG engine. For a final form of the sprite in battle, we plan to improve the cleanup and colorimetry, although the animations are intentionally simple.
Pet Trainer is 6 vs 6 idle RPG