Software Engineering Courses (SECourses)•17mo ago

@Dr. Furkan Gözükara would it be possible to add torch 2.5.1 for supir? Also I noticed I got better

@Dr. Furkan Gözükara would it be possible to add torch 2.5.1 for supir? Also I noticed I got better results with an old version of supir with v37 and using stage one and stage two

AAIGambino How long will 102 images take to train on flux for a lora at 1k res? Seems like ...

nicearoni•8/30/24, 12:03 AM

it's really more about steps\epochs than 'how long' since the speed of cards varies so much...

Nnicearoni it's really more about steps\epochs than 'how long' since the speed of cards var...

AIGambino•8/30/24, 12:58 AM

3000 steps bs 1

Ddsienra I'm training same parameters no class and regularization images, in the first sa...

dsienra•8/30/24, 1:35 AM

20 epochs 0 resemblance to the training characters using regularization lets see if with more epochs improves.

dsienra•8/30/24, 1:39 AM

TE and unet lr 1e04

Jehex•8/30/24, 3:08 AM

Hi, someone know an expert about transformers, diffusers files etc...

uisato•8/30/24, 3:39 AM

Hey! I'm trying torch 2.5 to see if I can speed things up. Think I set it up properly. But I'm not seeing much improvement [some, very little].

25000 steps / 66 hours on a RTX 3090 | 25 images / 5 repeats / 200 epochs

Is that normal for Flux?

Uuisato Hey! I'm trying torch 2.5 to see if I can speed things up. Think I set it up pro...

dsienra•8/30/24, 4:00 AM

how many it/s?

dsienra•8/30/24, 4:08 AM

why are you using 5 repeats? 5 repeats 200 epochs = 1000 epochs is excessive

Jackie•8/30/24, 4:10 AM

The real photos that I used for training

Ddsienra how many it/s?

uisato•8/30/24, 4:14 AM

9.47

Ddsienra why are you using 5 repeats? 5 repeats 200 epochs = 1000 epochs is excessive

uisato•8/30/24, 4:14 AM

how much'd you suggest? for a style LORA, first test

Uuisato 9.47

dsienra•8/30/24, 4:18 AM

slow

dsienra•8/30/24, 4:19 AM

what graphics card?

Ddsienra what graphics card?

uisato•8/30/24, 4:20 AM

3090

dsienra•8/30/24, 4:22 AM

with T5 attention mask off I get 4.5 it/s in my 3090 ti

dsienra•8/30/24, 4:23 AM

with T5 attention mask enabled 6.5 it/s

Uuisato how much'd you suggest? for a style LORA, first test

dsienra•8/30/24, 4:24 AM

I suggest 1 repetition 200 epochs

uisato•8/30/24, 4:24 AM

would you suggest 1 repeat / 200 epochs? 2 repeats / 100 epochs?

uisato•8/30/24, 4:24 AM

great.

uisato•8/30/24, 4:25 AM

thanks!

dsienra•8/30/24, 4:25 AM

5000 steps

Ddsienra with T5 attention mask off I get 4.5 it/s in my 3090 ti

uisato•8/30/24, 4:25 AM

have u seen quality differences with / without?

dsienra•8/30/24, 4:27 AM

no in ma case but is better if you have captions

dsienra•8/30/24, 4:27 AM

theoretically

uisato•8/30/24, 4:27 AM

so if i'm using captions, i should aswell use t5 attention

dsienra•8/30/24, 4:28 AM

yes, you can try with and without

dsienra•8/30/24, 4:30 AM

for a firts run you can try the fast way 1 repeat 150 or 200 epochs and see the results

dsienra•8/30/24, 4:30 AM

then try with attention mask and compare

dsienra•8/30/24, 4:31 AM

just a suggestion, there is no one solution for every case

dsienra•8/30/24, 4:33 AM

at this point is trial an error

AIGambino•8/30/24, 4:38 AM

Any way to improve these settings for ai toolkit?

message.txt2.23KB

AIGambino•8/30/24, 4:39 AM

102 images so far has been going 12 hours

AAIGambino Any way to improve these settings for ai toolkit?

dsienra•8/30/24, 4:58 AM

I only used kohya

dsienra•8/30/24, 4:58 AM

but you are using adamw8bit

dsienra•8/30/24, 4:59 AM

it consumes more vram

dsienra•8/30/24, 4:59 AM

I use adafactor

dsienra•8/30/24, 5:00 AM

what graphic card do you have?

mikemenders•8/30/24, 5:04 AM

Anyone use Prodigy for Flux traning?

Mmikemenders Anyone use Prodigy for Flux traning?

dsienra•8/30/24, 5:05 AM

not me, it too memory hungry

SeatownSin•8/30/24, 5:28 AM

are there any differences i should be aware of when making a gguf of a flux.1-dev fp32 safetensor model, compared to the way i would do it for an llm? just looking to make a q8 for people with less vram.

Tacite•8/30/24, 6:14 AM

https://huggingface.co/mo137/FLUX.1-dev_Q8-fp16-fp32-mix_8-to-32-bpw_gguf/tree/main

mo137/FLUX.1-dev_Q8-fp16-fp32-mix_8-to-32-bpw_gguf at main

TTacite https://huggingface.co/mo137/FLUX.1-dev_Q8-fp16-fp32-mix_8-to-32-bpw_gguf/tree/m...

J2_Nedo•8/30/24, 6:38 AM

using only original ones from Black forrest labs (Flux1-dev),
there are too many versions out for me to keep track of and whether they are all
compatible with my LoRAs or not.

Tacite•8/30/24, 6:57 AM

Yeah they are original, you can read the repo page to have the info