HD Assets from Simple Sprites with GenAI

In June 2023, we embarked on a mission to add HD characters to our character progression menu, building upon our gameplay sprites. Our approach hinged on utilizing AI generative tools to elevate the quality of assets and expedite their implementation into our game engine.

Key Takeaways:

• The AI-generated assets were visually impressive and stayed true to the original concept.

• The technique requires multiple iterations for each asset, making batch creation less feasible.

• Production time per asset averaged 4 hours, a significant reduction compared to the 2 days required for manual creation.

• A blend of expertise in generative techniques and strong drawing skills is crucial for artists involved in this process.

From Concept to HD Asset

Stable Diffusion's ControlNet Canny Model

In the realm of Stable Diffusion, ControlNet Canny (github link) is a model that preserves the overall shape outline of a canny edge image derived from any given image. We employed this model with our creature gameplay assets to maintain consistency between gameplay visuals and menu representations.

For this type of image, we tested and selected the canny method over several others

(line art, depth, scribble, etc.).

Selecting Reference Images

ControlNet's Reference_only model ensures the art style of the reference image is carried through. Elements like hair rendering and color palette are retained. At times, crafting a custom reference image with some quick Photoshop editing comes in handy.

During the iteration process, we reused the result image as the reference_only image with good success.

Recipe Settings

We explored different samplers, positive and negative prompts, adjusted the number of steps, and tweaked CFG scales. We also experimented with varying weights and starting/stopping steps for the ControlNet models to gauge their impact on the final image.

Once an initial recipe is established, we maintain it for subsequent images.

Reworking and Upscaling

This phase unfolds in a step-by-step process ensuring each asset is refined and prepped for game integration:

1. Batch Creation and Iteration: Initially, a batch of a dozen images is created. Through iterations, the best results are reused as new references for ControlNet, refining the quality and consistency of generated images.

2. In-Painting: The next step involves in-painting work in Stable Diffusion to address any missing or incomplete areas in the images, enhancing their completeness and coherence.

3. Upscaling: Following in-painting, images are upscaled from 512x512 to 2048x2048 using upscaling models to achieve the desired resolution for HD display.

4. Manual Refinement in Photoshop: The upscaled images then move to Photoshop for manual work which includes colorimetry adjustments, removing any remaining artifacts, and photobashing to ensure the final assets align with our artistic vision and game aesthetics.

This structured approach to reworking and upscaling allows for a meticulous refinement of assets, ensuring they meet our quality standards while aligning with the game's visual narrative.

Background Enhancement

We leveraged Adobe Photoshop's generative fill feature to enrich the background surrounding each character.

Game Implementation

In RPG games, the Unit Screen is a crucial interface where character progression and player attachment are developed. It's the area where the skins, evolutions, and gears of the pets are displayed. The implementation of HD assets here is essential for providing a clear and engaging view of character progression and customization options.

The implementation process is quite direct:

1. File Addition: The first step involves adding the generated HD asset files into our project directory.

2. Data Linking: Post file addition, these files are linked in the character's data file within our game engine. This linking ensures the correct display of the HD assets on the Unit Screen, reflecting the character's current state and any customizations made.

3. Asset Display: With the files properly linked, the HD assets are then displayed on the Unit Screen, showcasing the skins, evolutions, and gears of the pets to the players.

The asset would also be used in other parts of the game and for marketing assets.

Production Pipeline Efficiency

With the established pipeline, our artist could produce two visuals of this caliber daily, a significant acceleration compared to the two-day timeframe per visual without generative AI. This efficiency has allowed for a richer exploration of visual styles for our HD Pets. It's an encouraging start, and we are in the continuous process of optimizing our workflow to further enhance production efficiency.