Software Engineering Courses (SECourses)•17mo ago

Who can tell me if this is normal at this rate of training?

LLW24Ai Who can tell me if this is normal at this rate of training?

it•9/2/24, 4:56 AM

You're training way too many steps. you can get a perfectly good lora in 1200-2000 steps.

it•9/2/24, 4:57 AM

unless you're training like 1000s of images

it•9/2/24, 4:57 AM

if your dataset is 15-30 images then 1200-2000 steps is enough

Iit You're training way too many steps. you can get a perfectly good lora in 1200-20...

LW24AiOP•9/2/24, 4:58 AM

Except I've been practicing for 24 hours, where do I adjust the number of steps?

it•9/2/24, 4:59 AM

here

LLW24Ai Except I've been practicing for 24 hours, where do I adjust the number of steps?

it•9/2/24, 5:00 AM

too late now, you'd have to stop your training

Iit here

LW24AiOP•9/2/24, 5:01 AM

it•9/2/24, 5:01 AM

how many repeats are in your img folder

it•9/2/24, 5:01 AM

Iit how many repeats are in your img folder

LW24AiOP•9/2/24, 5:01 AM

it•9/2/24, 5:01 AM

just do 1

Iit here

LW24AiOP•9/2/24, 5:05 AM

What happens if I shrink Epoch?

LLW24Ai What happens if I shrink Epoch?

it•9/2/24, 5:05 AM

less training steps

it•9/2/24, 5:06 AM

if you do

1 Repeat
20 Images
100 Epochs

= 2000 steps

1 Repeat
20 Images
100 Epochs

= 2000 steps

LW24AiOP•9/2/24, 5:06 AM

And where do the files go after training?

it•9/2/24, 5:06 AM

Wherever you put your output folder

FFurkan Gözükara SECourses https://www.patreon.com/posts/110879657

druhl•9/2/24, 5:29 AM

Doc, the Lora turned out so good, even with the initial settings, that I'm going to wait till the next weekend just so you're able to release a significant update to the settings, before training again.

druhl•9/2/24, 5:30 AM

At this point, I believe further improvement in the input dataset will bring vast gains.

druhl•9/2/24, 5:32 AM

However, what I'd like to know - does Rank 1 bring more quality in the final results than say, Rank 5?

druhl•9/2/24, 5:33 AM

(my current understanding is that Ranks are just means to be able to train on any GPU, time varies but quality does not - am I wrong?).

druhl•9/2/24, 5:34 AM

@Dr. Furkan Gözükara

Iit if you do ```1 Repeat 20 Images 100 Epochs = 2000 steps ```

LW24AiOP•9/2/24, 5:51 AM

Thanks

LW24AiOP•9/2/24, 5:53 AM

Will Lora work with fp8 or is it only on fp16?

LLW24Ai Thanks❤️

it•9/2/24, 6:30 AM

are you using @Dr. Furkan Gözükara configs?

LLW24Ai Will Lora work with fp8 or is it only on fp16?

Neo•9/2/24, 6:43 AM

I just used his (2nd to latest) config with 20 images to ~4000 steps, with no issues. Make sure in the dataset preparation you are updating the folder repeats from 40 to 1 (saying because I missed that step). If you've already started teh training you can stop it and just rename the img sub-folder(s) to 1_ohwx man (or whatever concept it is)

Neo•9/2/24, 6:44 AM

@Laura Wolf If it has saved outputs you technically can resume the LoRA training from the latest output...

daryhead•9/2/24, 8:51 AM

I'm on runpod using a 3090 training flux with kohya and using the rank 4 file.
It's going for 3800 steps and says it's going to take 6 hours.
I'm going to stop it after about 2,000 steps, but even that is going to take a very long time.

What can I do to train in a shorter amount of time?
And is this normal for flux?
(When I used to do trainings for SDXL it was about 30 minutes to an hour)

Iit are you using @Dr. Furkan Gözükara configs?

LW24AiOP•9/2/24, 10:09 AM

Uporabljam config Rank_5_11498mb_počasi

LW24AiOP•9/2/24, 10:10 AM

LLW24Ai Who can tell me if this is normal at this rate of training?

Furkan Gözükara SECourses•9/2/24, 10:28 AM

you made the repeat 40

Furkan Gözükara SECourses•9/2/24, 10:28 AM

repeat has to be 1

Furkan Gözükara SECourses•9/2/24, 10:28 AM

speed is normal for rtx 3060

Ddruhl Doc, the Lora turned out so good, even with the initial settings, that I'm going...

Furkan Gözükara SECourses•9/2/24, 10:29 AM

rank 1 and rank 5 are very similar

Furkan Gözükara SECourses•9/2/24, 10:29 AM

since uses lora 128 t5 attention and 1024 px t5 text encoder

LLW24Ai Will Lora work with fp8 or is it only on fp16?

Furkan Gözükara SECourses•9/2/24, 10:30 AM

it works both after training

Ddaryhead I'm on runpod using a 3090 training flux with kohya and using the rank 4 file. ...

Furkan Gözükara SECourses•9/2/24, 10:30 AM

tell me step speed

Furkan Gözükara SECourses•9/2/24, 10:30 AM

but sounding accurate

Furkan Gözükara SECourses•9/2/24, 10:30 AM

you can also use multi gpu training

Furkan Gözükara SECourses•9/2/24, 10:31 AM

i am editing video sadly it is delayed to today

LLW24Ai Uporabljam config Rank_5_11498mb_počasi

Furkan Gözükara SECourses•9/2/24, 10:31 AM

ye this one is slow config expected

FFurkan Gözükara SECourses ye this one is slow config expected

LW24AiOP•9/2/24, 10:40 AM

I am working on the flux1CompactCLIPAnd_Flux1DevFp8 model If I train lora on your config. Will it work on my flux1 Compact model? I tried third-party lora, they didn't work on it.

LLW24Ai I am working on the flux1CompactCLIPAnd_Flux1DevFp8 model If I train lora on you...

Furkan Gözükara SECourses•9/2/24, 10:49 AM

dont use it

Furkan Gözükara SECourses•9/2/24, 10:49 AM

use fp8 or fp16

Furkan Gözükara SECourses•9/2/24, 10:50 AM

and it doesnt matter what you use but use accurate model to not have any error

LW24AiOP•9/2/24, 10:50 AM

Thanks

ghstoic•9/2/24, 11:13 AM

Ever since updating to the latest patreon version, I keep getting "training has ended." without any error messages before a single step has even run. I've tried doing clean reinstalls and downgrading to an earlier patreon script version, but it keeps happening. I've tried it with the new configs, as well as old configs & training datasets that worked perfectly yesterday on an older script version (I can't recall which version, though), but the same issue keeps occurring. Any idea's how to fix it as tried googling it without much success and doing clean installs as well?

WindowsTerminal_02-September-2024_02.12.02PM.png

Gghstoic Ever since updating to the latest patreon version, I keep getting "training has ...

Furkan Gözükara SECourses•9/2/24, 11:21 AM

looks to me like out of RAM

Furkan Gözükara SECourses•9/2/24, 11:21 AM

can you set virtual RAM

Furkan Gözükara SECourses•9/2/24, 11:21 AM

restart PC and let me know?

Furkan Gözükara SECourses•9/2/24, 11:21 AM

start from minute 39 to learn how to set virtual ram

Who can tell me if this is normal at this rate of training?

Similar Threads

Similar Threads

Similar Threads