Software Engineering Courses (SECourses)•3y ago

@Dr. Furkan Gözükara Hi Dr.Furkan & everyone, is there any guide for training ControlNet models (for

@Dr. Furkan Gözükara Hi Dr.Furkan & everyone, is there any guide for training ControlNet models (for non-coders)? I found a ControlLoraSave node but not sure how to use it. Is ControlNet training much harder than Lora training, so only a few folks/companies can do it?
https://github.com/Stability-AI/stability-ComfyUI-nodes & https://github.com/HighCWu/ControlLoRA

akellio•10/24/23, 3:04 PM

Hi, who has tried running on nvidia A100? What is the fastest video card for sdxl lora training on runpod?

iamannakogan•10/24/23, 5:01 PM

Thank you!

Mmikemenders Your photos will be 1024x1024 resolutions, use minimum 20-25 with different plac...

iamannakogan•10/24/23, 5:02 PM

Thanks!

rafraf•10/24/23, 5:55 PM

@Dr. Furkan Gözükara is there a tool to find duplicates within a large dataset?

mikemenders•10/24/23, 5:56 PM

Speaking of Adafactor: why can't I select the Adafactor scheduler if the optimizer is Adafactor? I get an error and it suggests to use constant with warmup optimizer.

Mmikemenders Here is my config for style. I used 10 images with 10 repeats, captions and 10 e...

Jas•10/24/23, 5:58 PM

Hey, I see you are very knowledgable about creating 1.5 LORAs.

I am trying to create some LORAs based on art styles/artworks.

Would this .json be appropriate?

I have got 50 images I want to train with for each style, or do you recommend less?

Is the base 1.5 model best to use? I have seen others recommending RealVision.

I am testing with training images 768x768, although I see you recommended 512x512, will that be ok?

Cheers for your help

mikemenders•10/24/23, 6:12 PM

@Jas The large file is also good for styling, and can be reduced to 512 by lora.
Which is why I train on SD 1.5:
- 512 resolution will take over the style, Hires can scale up the result, plus it adds exciting new detail, so it's okay if it burns in a bit.
- If you don't have powerful hardware, 512 will do. 768 will take longer to train, but you can also train on SDXL.
- I got one of the best training in style on Prodigy. Plus with 10 images, so 50 images will be perfect, even with 10 epochs.
- Label your pictures, but not with danbooru, but with a one sentence description of the picture in natural text, but not too long. If you use WD14 tagging, you will still have style, but SD 1.5 interprets it a bit differently and doesn't know everything from the words, so you may slip a bit on style. It won't be bad, but it will be a bit different. SDXL is supposed to understand the text better now.
- Don't use reg images, I recommend 10x repetition.

mikemenders•10/24/23, 6:14 PM

Here is my json for styles

Mykee_-_Prodigy_-_SD15_-_style.json2.97KB

mikemenders•10/24/23, 6:15 PM

One more thing in favour of SD 1.5: if you train your style on it, you can mix the lora with most models, and you can get very good results with cartoon models.

vKorby•10/24/23, 6:16 PM

Hey guys, i need some help here does anyone have any videos or advice if my SD is running literally for like 30 mins lmfao

mikemenders•10/24/23, 6:22 PM

Sometimes Auto freezes for me too, so restart the backend cold (close command prompt and restart it).
Use the --medvram-sdxl option. The preview image will be poor, but you will get a good image at the very end of the generation.

Mmikemenders Sometimes Auto freezes for me too, so restart the backend cold (close command pr...

vKorby•10/24/23, 6:37 PM

ight bet, will i still get the same type of images? because im looking for an anime/realstic type of image

VvKorby Yeah, heres another vid

vKorby•10/24/23, 6:37 PM

like this

mikemenders•10/24/23, 6:39 PM

Of course. The --medvram-sdxl option is there to generate the same image faster and in a different way. And it really will generate faster, and you'll see the end result at the very end.

Mmikemenders Of course. The --medvram-sdxl option is there to generate the same image faster ...

vKorby•10/24/23, 6:42 PM

sorry kinda new to this, where is that option under?

mikemenders•10/24/23, 6:55 PM

Type it to webui-user.bat file after set COMMANDLINE_ARGS=set COMMANDLINE_ARGS=

mikemenders•10/24/23, 6:56 PM

like this: set COMMANDLINE_ARGS=--medvram-sdxlset COMMANDLINE_ARGS=--medvram-sdxl

mikemenders•10/24/23, 6:56 PM

and start SD with webui-user.bat

Mmikemenders and start SD with webui-user.bat

vKorby•10/24/23, 7:00 PM

got it trying it rn thank you, btw my "webui-user.bat" file doesn't have ".bat" its just "webui-user" is that normal?

BarbaraM•10/24/23, 7:00 PM

idk why but control net doesn't seem to work for me on sdxl models, (even with control nets sdxl models), tried with canny, open pose ad depth. nothing seems to work

Capture_decran_2023-10-24_a_20.59.36.png

VvKorby got it trying it rn thank you, btw my "webui-user.bat" file doesn't have ".bat" ...

mikemenders•10/24/23, 7:01 PM

yes, but you don't have the file extension display turned on in windows

Mmikemenders yes, but you don't have the file extension display turned on in windows

vKorby•10/24/23, 7:02 PM

Oh ok thanks

mikemenders•10/24/23, 7:15 PM

I just did a one-on-one Lora training. I only kept the images that were identity 1 based on CompreFace, even discarding the 0.99 ones. I trained with three different optimizers, Adafactor, Prodigy and AdamW. Adafactor and Prodigy trained on the RealisticVision 5.1 model, AdamW trained on SD 1.5.
All three models bring out the person and their body well, and it's very interesting to see how each model brings out different details in the person's face and body. Since the images are high resolution, I'll try SDXL training on them soon. I am very curious!