Art and Designs

I also posted this guide to the reddit sub “/r/StableDiffusion” under the username “spiritusastrum”.

Generating the same character in multiple poses is one of the most common questions for beginners in AI generated art.

Most solutions involve LORA’s, Dreamboth, etc, etc, and these are preferred, however they are complex, and require good quality training sets.

It is possible to generate enough training images using just SD, but this is difficult.

I have, after some research and trial and error, discovered a very simple way to create a unique character using entirely Stable Diffusion and then change that characters pose, while keeping most of their likeness (Clothing, hair, face, etc, intact).

I am using “DivineAnimeMix” to generate the images used in this guide.

The guide is aimed mainly at simple artwork, such as the kind you would see in Visual Novels, etc. With complex art or close ups of a characters face, etc, this technique may not work as well.

First, (using txt2img) create an extremely specific prompt, and use it to produce a single image of a character.

This image should be the “main” character image, used for the characters default image in the visual novel, etc.

It is important to make sure that this image matches the prompt as much as possible.

For example, if the colour of the characters jacket is different from the prompt, it should be fixed now, otherwise the jacket will be the wrong colour when the pose changes. This can be fixed, but it is easier to fix it now.

Also, it can be very difficult (Or almost impossible) to get things like tattoos, makeup, etc, to match properly when the pose changes, so it can be desirable to avoid creating characters with tattoos, etc. If a character does have tattoos, makeup, etc, it is important to specify the location of the tattoo and what it is.

So, instead of just “With tattoos” say “With a tattoo on their left arm”, etc.

The final point with prompt generation is to add a pose or stance with a “weight” modifier, such as:

(Standing: 5.5).

This is used to change the pose, without changing the prompt.

This is the image that I generated at this stage of the process:

With the default image and prompt generated, it is now possible to change the pose.

This is done by just changing the weighed prompt, so:

(Standing: 5.5) could become (Riding a Motorcycle: 5.5), or anything else.

Of course, cfg scale, restore faces, etc, can all be used to improve the quality of the images, however the prompt should not be changed, apart from the pose.

What will now happen is that many similar, not identical, characters will be generated.

This is the image that I generated for this step:

Notice that the character is similar, but not identical, to the first image.

Once an image is produced that firstly, matches the desired pose, and secondly, has relatively few differences to the “main” image, it is time for the next step.

Download the second image and open it in photoshop, or any other basic image editor (Even MS paint would work fine for this).

Now, with the main image as a guide, roughly mark any areas which the AI has gotten “wrong”.

Is the jacket the wrong colour? Use the colour dropper tool to paint over the jacket on the second image with the colour from the first image.

Is the AI wearing long pants in the second image, and shorts in the first?

Again, use the eye dropper tool to paint a skin-colour texture over the long pants texture.

Is the hair too long, or too short? Are there tattoos where there shouldn’t be?

Do the same thing.

If something is missing from the second image, simply select and copy it from the first image.

Tattoos, a belt, a style of glove, even an entire face, can be very crudely copied, pasted, and scaled, onto the second image.

This will result in something that looks awful, but this is perfectly fine. The goal is simply to add visual cues to tell the AI which parts of the image to regenerate, the AI will do the rest.

This is the image that I created here:

Notice that I have painted over her right arm (Her sleeves should be short) and her right hand (She should be wearing gloves). I have also copied and pasted the face, right sleeve, and the fur collar from the first image.

When this is done, upload the modified image to stable diffusion, this time to img2img.

Use the same prompt that was used to generate the second image (not the main image!), and set the “denoising strength” as low as possible. The idea is to JUST regenerate the parts that you painted over in photoshop, not the rest of the image.

You can use inpainting for this (painting over only the parts of the image that you want to regenerate, leaving the rest), but I found that img2img works as good or better (I seemed to end up with bad-quality faces more often with inpainting).

You may need to generate several images, but you should end up with a character that looks MUCH more like the main character that you initially generated, but with the pose of the second character.

If there are any minor issues remaining, simply take the best result, download it to photoshop, and go through the process again, you can repeat this as many times as necessary.

This is one of my final images:

Notice that the right arm is still wrong (She has veins instead of tattoos, and she has a bracelet instead of gloves) but these issues could be fixed in photoshop. The major details match the original character, while the pose is different.

Here is another image:

Again, the likeness is not perfect, but it is close enough that I think most people would regard this character as the “same” as the first one.

Here is a side by side comparision between the initial character and the final image:

I think that, for a single pass, this is a very good result.

If I did another pass in photoshop, I could fix the issues with the right glove, and the red stitching on her pants, as well as modify the tattoos to help them match up more.

I have tried this with “realistic” and anime style checkpoints and it works very well with both. I suspect it would work better with illustrated or manga specific checkpoints, because there would generally be less detail involved, and so it would be harder to detect differences between the images.

This solution, of course, does NOT solve the problem of creating the “same character in a different pose”. You are generating a new character, that just happens to look similar, however, this process seems to work well enough for my purposes at least, and it may work for others as well.