I have now tried the Prodigy Lora training with another parameter that is also recommended for SDXL. In general, in both cases, epoch 10 (the last one) gives (for me) absolutely satisfactory and stable results.
- This is the optimizer args that I am using now: "use_bias_correction=True" "weight_decay=0.01" "decouple=True" "betas=(0.9,0.999)" "d_coef=2" "safeguard_warmup=False"
It makes the model more flexible and works well on Clip Skip 1. The way I look at it, 3-6 epochs already give good results, at higher epochs you can see one of the characteristics of overdrawing, the freckling.
- this is what I used yesterday: "use_bias_correction=True" "weight_decay=0.5" "decouple=True" "betas=(0.9, 0.99)" "d_coef=2" "safeguard_warmup=False" (here the weight_decay value is stronger)
This is also very good from 3, and even 10 didn't burn. But this gives an absolutely strong, realistic model even at 512x512 resolution, without Hires Fix. This is worth training on Clip Skip 2 to make it flexible. I got such a perfect result that it not only restored the tattoo to its position, but even its shape. However, it does not like getting naked because of this.
So if you want very realistic images, the latter is better, if you want more flexible, the former.
Update1: Epoch 4 seems to be the most flexible out of the 10, after that you need stronger prompts if you want to see not only the face but also the body.
