Software Engineering Courses (SECourses)•15mo ago

@Dr. Furkan Gözükara I had wanted to ask, I read some documentation that training captions requires

@Dr. Furkan Gözükara I had wanted to ask, I read some documentation that training captions requires commas to separate the prompt, does that still hold true for flux? I have my prompts structured as sentences and there are no commas only peroids at the end of the sentence.

VVigilence @Dr. Furkan Gözükara I had wanted to ask, I read some documentation that trainin...

Furkan Gözükara SECourses•10/9/24, 8:48 PM

i think not

Furkan Gözükara SECourses•10/9/24, 8:48 PM

but you dont need captions for flux

Furkan Gözükara SECourses•10/9/24, 8:48 PM

at least for distilled dev

VigilenceOP•10/9/24, 8:49 PM

too late

VigilenceOP•10/9/24, 8:49 PM

VigilenceOP•10/9/24, 8:50 PM

It just finished and I am happy with the results.

VVigilence Click to see attachment

Furkan Gözükara SECourses•10/9/24, 8:50 PM

nice

VigilenceOP•10/9/24, 8:50 PM

Fully captioned, including colors, with activation sentence and word.

VigilenceOP•10/9/24, 8:51 PM

"painted of the style of "word"".

VVigilence Fully captioned, including colors, with activation sentence and word.

Furkan Gözükara SECourses•10/9/24, 8:54 PM

yes it works

Furkan Gözükara SECourses•10/9/24, 8:54 PM

but a little bit subpar

Furkan Gözükara SECourses•10/9/24, 8:54 PM

i compared :d

Furkan Gözükara SECourses•10/9/24, 8:54 PM

but depends on your dataset

Furkan Gözükara SECourses•10/9/24, 8:54 PM

it must be consisten

VigilenceOP•10/9/24, 8:55 PM

I compare caption with activation vs caption without when making the image in comfyui. I prefer caption with activation word as quality is slightly better.

VigilenceOP•10/9/24, 8:55 PM

I did caption by hand with gpt4

Furkan Gözükara SECourses•10/9/24, 8:55 PM

i need to start 4 lora training tonight

Furkan Gözükara SECourses•10/9/24, 8:55 PM

so i can compare best lora vs best fine tune :d

VigilenceOP•10/9/24, 8:56 PM

try dora as well and compare all 3

VVigilence try dora as well and compare all 3 😄

Furkan Gözükara SECourses•10/9/24, 8:56 PM

it will be only later time

Dazzastrous [4090]•10/9/24, 8:56 PM

u reckon you will be finished with flux training by next christmas doc?

VigilenceOP•10/9/24, 8:57 PM

The only problem with this lora is that it took a long time to train - 25 hr at 2.61 it/s

Dazzastrous [4090]•10/9/24, 8:58 PM

i think i could now recognise you blindfolded

VVigilence I did caption by hand with gpt4

hamuz•10/9/24, 8:58 PM

I had previously tried manual captioning with GPT Vision too, but it was really tedious and torturous because my internet was very limited and I couldn't download a BLIP model, especially with large datasets

VigilenceOP•10/9/24, 9:02 PM

It doesn't do so well vs certain art styles like kawaii if it has lots of busy things happening, but for paintings its 95% great.

DDazzastrous [4090]i think i could now recognise you blindfolded

Furkan Gözükara SECourses•10/9/24, 9:05 PM

Furkan Gözükara SECourses•10/9/24, 9:07 PM

batch size 7 fails

Furkan Gözükara SECourses•10/9/24, 9:07 PM

meanwhile fine tuning works :d

FFurkan Gözükara SECourses put into diffusion_models

Papst•10/9/24, 9:10 PM

Thanks a lot, I am so stupid!

PPapst Thanks a lot, I am so stupid! 😄

Furkan Gözükara SECourses•10/9/24, 9:10 PM

no issues

Furkan Gözükara SECourses•10/9/24, 9:11 PM

testing gradient accumulation steps 7

Furkan Gözükara SECourses•10/9/24, 9:11 PM

instead of batch size 7

Furkan Gözükara SECourses•10/9/24, 9:11 PM

to make comparison more fair

Furkan Gözükara SECourses•10/9/24, 9:14 PM

gradient accumulation steps 7 works

Furkan Gözükara SECourses•10/9/24, 9:14 PM

but sloww

DDazzastrous [4090]i think i could now recognise you blindfolded

OinkSauce•10/10/24, 1:03 AM

What, exactly, are you doing with these images of Dr. Gözükara that you can feel his likeness?

wardensc2•10/10/24, 1:08 AM

hi @Dr. Furkan Gözükara I wonder if I train lora using flux dev but gen image with Flux Snell with trained lora, the quality is still good enough ?

gawn•10/10/24, 1:55 AM

hi! I am following the lora tutorial to do a dreambooth flux tune on massed compute - I am loading the 48gb config for my A6000, but when the training starts it says it will be something like 880,000 steps and will take 2500 hours. I have 125 images and am using the default of 40 repeats with everything else left as-is in the config (best configs v5) - is that accurate or am I missing something?

Ggawn hi! I am following the lora tutorial to do a dreambooth flux tune on massed comp...

wardensc2•10/10/24, 2:11 AM

change repeat to 1, the config file does not change this that why you need 880,000 steps

wardensc2•10/10/24, 2:11 AM

a few thousands step is enough

gawn•10/10/24, 2:13 AM

thank you! so 22000 steps and ~40 hours is normal?

gawn•10/10/24, 2:13 AM

it still has 200 epochs maybe that's also unnecessary?

wardensc2•10/10/24, 2:18 AM

yes due to step we only use 1, 200 epochs is a must

gawn•10/10/24, 2:18 AM

alright, guess I just set an alarm on my phone to check in 2 days lol. thank you!

Ggawn thank you! so 22000 steps and ~40 hours is normal?

wardensc2•10/10/24, 2:19 AM

depend on how do you want to train: checkpoint or lora. style lora or person lora

gawn•10/10/24, 2:19 AM

person lora but I read the post about how finetunes are much better at keeping it flexible

wardensc2•10/10/24, 2:20 AM

for a person lora more than 6000 steps I think a bit much

gawn•10/10/24, 2:21 AM

got it - even going the dreambooth route?

wardensc2•10/10/24, 2:28 AM

I haven't trained checkpoint yet but it will require more hardware power and times since many people have already done it.

gawn•10/10/24, 2:28 AM

I guess I will see how it pans out in ~12ish hours!

@Dr. Furkan Gözükara I had wanted to ask, I read some documentation that training captions requires

Similar Threads