Software Engineering Courses (SECourses)•16mo ago

I got caption files with 300 tokens, so it wont train?

XXcorpion I got caption files with 300 tokens, so it wont train?

Furkan Gözükara SECourses•9/13/24, 10:38 AM

it will

Furkan Gözükara SECourses•9/13/24, 10:38 AM

but only 225

XcorpionOP•9/13/24, 10:57 AM

is there any way to make it 300?

XcorpionOP•9/13/24, 10:57 AM

sad

XXcorpion is there any way to make it 300?

Furkan Gözükara SECourses•9/13/24, 11:21 AM

what do you train sdxl?

papanton•9/13/24, 12:22 PM

@Furkan Gözükara SECourses for flux what is the best trainer currently? Konya, AI toolki, onetrainer, something - something else? Do you see OneTrainer catching up?

Ppapanton <@205854764540362752> for flux what is the best trainer currently? Konya, AI too...

Furkan Gözükara SECourses•9/13/24, 12:47 PM

onetrainer not good unless you have 40 gb gpu :d

Furkan Gözükara SECourses•9/13/24, 12:47 PM

it doesnt have fp8 yet

Furkan Gözükara SECourses•9/13/24, 12:47 PM

kohya works great

Furkan Gözükara SECourses•9/13/24, 12:47 PM

i didnt compare ai toolkit

papanton•9/13/24, 12:55 PM

how about training time on a A6000 - what should one expect for a 15 person picture Lora with your configs?

Ppapanton how about training time on a A6000 - what should one expect for a 15 person pict...

Furkan Gözükara SECourses•9/13/24, 1:11 PM

easy to calculate

Furkan Gözükara SECourses•9/13/24, 1:11 PM

lets say you used Rank_2_27700MB_4_85_Second_IT.json

Furkan Gözükara SECourses•9/13/24, 1:11 PM

it is 4.85 second / it

Furkan Gözükara SECourses•9/13/24, 1:11 PM

you need 3000 steps - assuming 200 epoch

Furkan Gözükara SECourses•9/13/24, 1:11 PM

3000 x 4.85 second

papanton•9/13/24, 2:04 PM

I see - thank you. By what factor does speed increase on mass compute multi GPU training ?

Ppapanton I see - thank you. By what factor does speed increase on mass compute multi GPU ...

Furkan Gözükara SECourses•9/13/24, 3:23 PM

almost linear

Furkan Gözükara SECourses•9/13/24, 3:23 PM

if you rent 8 gpu you get like 7x speed up maybe more

macba•9/13/24, 3:34 PM

@Furkan Gözükara SECourses congrats on the latest training article. Can you add something about captioning (if you used it, how you made it…)?

macba•9/13/24, 3:35 PM

I mean, how did the model learn what each expression was? What expression does it inherently know, as far as your dataset goes?

Mmacba <@205854764540362752> congrats on the latest training article. Can you add somet...

Furkan Gözükara SECourses•9/13/24, 3:35 PM

i added it actually :d

Furkan Gözükara SECourses•9/13/24, 3:35 PM

I didn't caption them only ohwx man since captioning reduces likeliness and doesn't bring improvements.

Furkan Gözükara SECourses•9/13/24, 3:35 PM

i updated article added more details

macba•9/13/24, 3:36 PM

The Patreon one?

Furkan Gözükara SECourses•9/13/24, 3:36 PM

yep

Furkan Gözükara SECourses•9/13/24, 3:36 PM

https://www.patreon.com/posts/111891669

Patreon

Training a FLUX LoRA with 256 Images Experiments So Far | SECourses...

Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon

macba•9/13/24, 3:37 PM

I haven't looked at the training dataset--as far as you can tell, does it look like Flux understand 100% of facial expressions? 50%?

Mmacba I haven't looked at the training dataset--as far as you can tell, does it look l...

Furkan Gözükara SECourses•9/13/24, 3:37 PM

it understand alls

Furkan Gözükara SECourses•9/13/24, 3:37 PM

i shared dataset too

Furkan Gözükara SECourses•9/13/24, 3:37 PM

but lower res

macba•9/13/24, 3:39 PM

Everything like "anxious"? Like, if you put in a picture of yourself looking worried about something (dunno if you're that good of an actor

), do you think the model would pick it up and be able to reproduce it when the prompt asks for it?

Mmacba Everything like "anxious"? Like, if you put in a picture of yourself looking wor...

Furkan Gözükara SECourses•9/13/24, 3:59 PM

yep

Furkan Gözükara SECourses•9/13/24, 3:59 PM

model has great internal captioning system

FFurkan Gözükara SECourses what do you train sdxl?

XcorpionOP•9/13/24, 7:05 PM

yeah yeah, anime style lora

XXcorpion yeah yeah, anime style lora

Furkan Gözükara SECourses•9/13/24, 9:05 PM

i dont have a very good config for sdxl lora atm

Furkan Gözükara SECourses•9/13/24, 9:05 PM

but training sdxl dreambooth / fine tuning and extracting lora works great

dxqb•9/14/24, 7:47 AM

what are they offering here?

Ddxqb what are they offering here?

Furkan Gözükara SECourses•9/14/24, 12:32 PM

something very inferior i guess

Furkan Gözükara SECourses•9/14/24, 12:32 PM

probably high batch size, high LR low step

dxqb•9/14/24, 12:34 PM

even if all that and even if they train only some layers of the model, 5 min and only 1 epoch seems extreme. They must have found something new?

Ddxqb even if all that and even if they train only some layers of the model, 5 min and...

Furkan Gözükara SECourses•9/14/24, 12:34 PM

i doubt that

dxqb•9/14/24, 12:35 PM

we'll see if civitai gets flooded with sh** flux loras soon

Furkan Gözükara SECourses•9/14/24, 12:35 PM

high batch, high LR, single layer makes sense

Ddxqb we'll see if civitai gets flooded with sh** flux loras soon

Furkan Gözükara SECourses•9/14/24, 12:35 PM

yep

dxqb•9/14/24, 12:36 PM

the fine print says 1 epoch. so it isn't high batch or massive parallelization, it's just 1 epoch.

dxqb•9/14/24, 12:36 PM

I don't have any of their tokens to try, but would be interested if someone her has

dxqb•9/14/24, 12:36 PM

*here

Ddxqb the fine print says 1 epoch. so it isn't high batch or massive parallelization, ...

Furkan Gözükara SECourses•9/14/24, 12:45 PM

1 epoch but how many repeats

dxqb•9/14/24, 12:49 PM

repeats are a kohya thing, 1 epoch in the general context of ML usually means going once through your dataset. And look here, 2,5 minutes. Even if you used multiple GPUs, it cannot mean much more than going through your dataset once with the s/it I have seen so far on flux

I got caption files with 300 tokens, so it wont train?

Similar Threads

Similar Threads

Similar Threads