Software Engineering Courses (SECourses)•17mo ago

Dr Furkan, i hope evertyting is ok, im traying to make my firt lora model in flux, i have a 4070 sup

Dr Furkan, i hope evertyting is ok, im traying to make my firt lora model in flux, i have a 4070 super ti 16 gigas vram this speed is good? if i install the torcho 2.5 could increase speed? im use the rank 5 setting

Ssergio_gonzo Dr Furkan, i hope evertyting is ok, im traying to make my firt lora model in flu...

Furkan Gözükara SECourses•8/31/24, 2:51 AM

speed is good . after this try torch 2.5 and see if improves. i think may improve

sergio_gonzoOP•8/31/24, 2:54 AM

thank you Dr Furkan Im using 44 images maybe i will try with fewer the next time

Phuong Bui•8/31/24, 6:29 AM

Can I ask if a 1080 8gb card can train lora? If not, is the 3060 capable?

PPhuong Bui Can I ask if a 1080 8gb card can train lora? If not, is the 3060 capable?

Furkan Gözükara SECourses•8/31/24, 10:28 AM

it cant since 1080 doesnt have bf16

Furkan Gözükara SECourses•8/31/24, 10:28 AM

but 3060 perfectly can

FFurkan Gözükara SECourses it cant since 1080 doesnt have bf16

Phuong Bui•8/31/24, 11:10 AM

Thank you for letting me know. With 3060 how long does it take to train on average?

PPhuong Bui Thank you for letting me know. With 3060 how long does it take to train on avera...

Furkan Gözükara SECourses•8/31/24, 12:18 PM

it would take like 10 hours to get a perfect quality

Furkan Gözükara SECourses•8/31/24, 12:18 PM

because we have to use huge optimization

Furkan Gözükara SECourses•8/31/24, 12:18 PM

however there may come a new technique

Furkan Gözükara SECourses•8/31/24, 12:18 PM

training a single layer

Furkan Gözükara SECourses•8/31/24, 12:18 PM

waiting kohya on that

daroth•8/31/24, 12:31 PM

Seems like vast majority of character Loras work even without including their supposed trigger words. Interesting

Ddaroth Seems like vast majority of character Loras work even without including their su...

Furkan Gözükara SECourses•8/31/24, 12:39 PM

flux has internal encoding

Furkan Gözükara SECourses•8/31/24, 12:39 PM

so every image fully captioned

dxqb•8/31/24, 1:06 PM

not sure what you two mean. I have used a character lora with flux recently. trigger word (which was the name of the person) was necessary. using the lora without it already did something, but not everything

Ddxqb not sure what you two mean. I have used a character lora with flux recently. tri...

Furkan Gözükara SECourses•8/31/24, 1:29 PM

if you dont use trigger word and if it is not overfit you wont get exactly that person

Furkan Gözükara SECourses•8/31/24, 1:30 PM

becuase you cant know exact internal encoding / caption

FFurkan Gözükara SECourses if you dont use trigger word and if it is not overfit you wont get exactly that ...

sergio_gonzoOP•8/31/24, 3:23 PM

Dr Furkan i hope evrything is ok, wht do you think could be the best, cost/benefit amount fo dataset image for training?

Ssergio_gonzo Dr Furkan i hope evrything is ok, wht do you think could be the best, cost/benef...

Furkan Gözükara SECourses•8/31/24, 3:43 PM

i plan to test with 100 images

Furkan Gözükara SECourses•8/31/24, 3:43 PM

currently works as low as 15 very good

daroth•8/31/24, 4:29 PM

hm, does Lora merging work for flux?

Ddaroth hm, does Lora merging work for flux?

Furkan Gözükara SECourses•8/31/24, 4:30 PM

they added some tools but didnt test yet

daroth•8/31/24, 4:30 PM

I remember results weren't always good previously, especially if one wanted to merge loras of different sizes

macba•8/31/24, 5:53 PM

When using Dreambooth training, what is the formula for total steps when using multiple GPUs? In the video about Flux training it seemed like you would multiply everything by the number of GPUs, so 100 images, 1 repeat, batch size 1, 10 epochs would be 1000 steps on a 1 GPU, 4000 steps on a x4 GPU machine. However, when running a Dreambooth finetune, the system insists it's going to be 1000 steps unless I actually increase the number of epochs to 10 x the number of GPUs…

Mmacba When using Dreambooth training, what is the formula for total steps when using m...

Furkan Gözükara SECourses•8/31/24, 6:10 PM

whatever you calculate divide to number of gpus

Furkan Gözükara SECourses•8/31/24, 6:10 PM

so single gpu 1000 steps or 100 epochs becomes 250 steps and 25 epochs with 4 gpus

Furkan Gözükara SECourses•8/31/24, 6:11 PM

also use number of processes qual to number of gpus

macba•8/31/24, 6:11 PM

That's what I thought, but: I had 175 images, so I set 12 epochs hoping I'd get 8400 steps

macba•8/31/24, 6:12 PM

But the command line logs only showed 1200 steps to go

macba•8/31/24, 6:13 PM

So I stopped and changed the settings--should I have let it run with 1200? There's no indication that it's running those 1200 separately on 4 GPU

Mmacba So I stopped and changed the settings--should I have let it run with 1200? There...

Furkan Gözükara SECourses•8/31/24, 6:15 PM

1200 steps becomes 4800

Furkan Gözükara SECourses•8/31/24, 6:15 PM

since process 4x at once

macba•8/31/24, 6:16 PM

Sorry, mistyped here: it was supposed to say '2100' of course

macba•8/31/24, 6:16 PM

175x12 → 2100

macba•8/31/24, 6:17 PM

So you're saying that the stdout logs show 2100, but in the end the real count will be 8400?

FFurkan Gözükara SECourses they added some tools but didnt test yet

sergio_gonzoOP•8/31/24, 8:45 PM

dr Furkan, exist controlnet canny or lineart for flux version that can be used in forgue?

Mmacba So you're saying that the stdout logs show 2100, but in the end the real count w...

Furkan Gözükara SECourses•8/31/24, 11:44 PM

yes batch size will be 4

Furkan Gözükara SECourses•8/31/24, 11:44 PM

effective batch size

Ssergio_gonzo dr Furkan, exist controlnet canny or lineart for flux version that can be used i...

Furkan Gözükara SECourses•8/31/24, 11:45 PM

i think i saw some early ones but probably not good. also i dont know if works with forge

ECD•9/1/24, 3:52 AM

@Furkan Gözükara SECourses Hello Doctor, I am having problems with the presets tier 1 V2 slow and fast.
I seem to have a memory leak. The VRAM fills up, and then it eats almost all the RAM, the training becomes extremely slow and my PC loses stability so I have to urgently abort the process.
Do you know if Onetrainer updated any parameters? (I just executed the update.bat today) With the other presets or older versions of your own presets don't give any problems. Thanks.

daroth•9/1/24, 6:31 AM

tier 1 is too much for any consumer GPU I think, tier 3 is the best that fits into ANY

macba•9/1/24, 9:42 AM

Is there a @Furkan Gözükara SECourses in the house? I couldn't find any material (post or video) about the importance of captioning and regularization images for SDXL Lora/Dreambooth training. Can you point me in the right direction?

macba•9/1/24, 9:43 AM

I got very good results with Flux (where you mentioned reg images are not useful, and a single trigger word is good for single persons), but my SDXL results where horrible…