Software Engineering Courses (SECourses)•8mo ago

<@205854764540362752> is there a tutorial or explaination on how to determine if the model is overfi

@Furkan Gözükara SECourses is there a tutorial or explaination on how to determine if the model is overfit or not? I know it was mentioned somewhere, but I cant find a specific clear instructions. Thank you!

NN7-Blackbird <@205854764540362752> is there a tutorial or explaination on how to determine if...

Furkan Gözükara SECourses•5/7/25, 12:04 PM

only way is generating samples

Furkan Gözükara SECourses•5/7/25, 12:04 PM

sadly loss rate is meaningless

TrumpYouAll•5/7/25, 1:48 PM

@Furkan Gözükara SECourses Will you make a video to tell us how to train HiDream by using Dreambooth in MassedCompute Cloud?

TTrumpYouAll <@205854764540362752> Will you make a video to tell us how to train HiDream by u...

Furkan Gözükara SECourses•5/7/25, 1:51 PM

yes i plan to. the things that comes never ending :d

FFurkan Gözükara SECourses yes i plan to. the things that comes never ending :d

TrumpYouAll•5/7/25, 1:52 PM

Thanks very much, I'd like to use HiDream but without Dreambooth, not much use for me for now.

I know you are gonna make it ! Thanks @Furkan Gözükara SECourses

skyrrr•5/7/25, 1:55 PM

Do you think it worth it compared to flux dev finetune ?

Sskyrrr Do you think it worth it compared to flux dev finetune ?

TrumpYouAll•5/7/25, 2:12 PM

Flux has too many internal censorships

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭•5/7/25, 2:58 PM

Has anyone fine tuned on pixelwave checkpoint?

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭•5/7/25, 2:58 PM

Or made lora

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭Has anyone fine tuned on pixelwave checkpoint?

Furkan Gözükara SECourses•5/7/25, 5:10 PM

i did

Furkan Gözükara SECourses•5/7/25, 5:10 PM

https://medium.com/@furkangozukara/thoroughly-experimented-with-fine-tuning-dreambooth-training-of-flux-dev-de-distill-pixelwave-13fab8c625bf

Medium

Thoroughly experimented with Fine-Tuning / DreamBooth training of F...

Thoroughly experimented with Fine-Tuning / DreamBooth training of Flux-dev-de-distill, PixelWave v03, Verus Vision and base FLUX Dev model…

Sskyrrr Do you think it worth it compared to flux dev finetune ?

Furkan Gözükara SECourses•5/7/25, 5:11 PM

what i care is multi concept

Furkan Gözükara SECourses•5/7/25, 5:11 PM

if works or not

Furkan Gözükara SECourses•5/7/25, 5:11 PM

otherwise for single concept flux pretty much amazingly working

Escanor•5/8/25, 9:38 PM

Hello, I want to train sdxl koyha_ss on massed compute with A100 graphic card, I use the Tier1_48_GB_Faster_v2 config but I don't know what epoch I should set. I only have 8 images and I use all 5000 reg images. From what I understand, I should aim for 3000-5000 total steps. So I'm currently trying with the default settings of the configuration but setting 400 repeats so I have about 3200 steps...

Is 400 repeats with 1 epoch and 8 pictures good?
I set a batch size of 4 since the A100 gpu has 80gb of memory. Is this right?
Could I optimize the config better to be faster or more efficient for A100 gpu?

Thank you very much for all the tutorials and the help

Escanor•5/8/25, 9:43 PM

Also, on massed compute, I see Could not find TensorRTCould not find TensorRT when starting training with kohya_ss, is this normal? Does it matter to install TensorRT for training only?

Ah and if it's only 8 portrait images, do I want captions? From what I understand no

Thanks

EEscanor Hello, I want to train sdxl koyha_ss on massed compute with A100 graphic card, I...

Furkan Gözükara SECourses•5/8/25, 10:33 PM

welcome

EEscanor Also, on massed compute, I see `Could not find TensorRT` when starting training ...

Furkan Gözükara SECourses•5/8/25, 10:33 PM

not important

Furkan Gözükara SECourses•5/8/25, 10:33 PM

8 images really low

Furkan Gözükara SECourses•5/8/25, 10:33 PM

so go up to 300 epochs

Furkan Gözükara SECourses•5/8/25, 10:33 PM

since it is sdxl

Furkan Gözükara SECourses•5/8/25, 10:33 PM

make sure to use reg images

Furkan Gözükara SECourses•5/8/25, 10:34 PM

for kohya

Furkan Gözükara SECourses•5/8/25, 10:34 PM

you need to test repeating to be able to use class images

Furkan Gözükara SECourses•5/8/25, 10:34 PM

400 repeat 1 epoch sounding good

Furkan Gözükara SECourses•5/8/25, 10:34 PM

make sure to save once every n steps

FFurkan Gözükara SECourses make sure to save once every n steps

Escanor•5/8/25, 10:51 PM

Ok, and is the tier1 48gb config optimal for A100 graphic card with 80gb or I should change something to make it better and use more available memory? Batch size around 4-7? Do you recommend onetrainer finetune over kohya_ss dreambooth?

Thank you for the quick answer

EEscanor Ok, and is the tier1 48gb config optimal for A100 graphic card with 80gb or I sh...

Furkan Gözükara SECourses•5/8/25, 10:53 PM

i dont recommend change anything it is best quality

Escanor•5/8/25, 11:01 PM

I'll try to give results here if I find the best sweet spot for my number of training images!

I do think it could be useful to have somer kind of pinned message with recommended setups of epoch #, repeat # and such specific to the # of images, or at least a ballpark close to it

Like 8 images: 400 repeats, 1 epoch

EEscanor Ok 🙂 I'll try to give results here if I find the best sweet spot for my number...

Furkan Gözükara SECourses•5/8/25, 11:13 PM

sadly there isnt much but i usually recommend 200 epoch

Furkan Gözükara SECourses•5/8/25, 11:13 PM

however your train image count too low

Furkan Gözükara SECourses•5/8/25, 11:14 PM

more is better

Escanor•5/8/25, 11:39 PM

Yeah I only have 8 images, I'm hoping I can finetune a model and generate decent enough images to be able to train with more images... Not sure how that will go lol

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭•5/9/25, 9:08 AM

hi, for quick trainings I often use civitai. I remember you talked about rank of lora in kohya, here it is both rank and alpha, and not sure about these. The default seems to be the same for both flux and sdxl trainings: Default is also not adafactor, but the Adam...any recommendations here? realistic person training

EEscanor Yeah I only have 8 images, I'm hoping I can finetune a model and generate decent...

Furkan Gözükara SECourses•5/9/25, 1:52 PM

you can also do this

Furkan Gözükara SECourses•5/9/25, 1:52 PM

1st fine tune generate pick good ones

Furkan Gözükara SECourses•5/9/25, 1:52 PM

and fine tune again

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭hi, for quick trainings I often use civitai. I remember you talked about rank of...

Furkan Gözükara SECourses•5/9/25, 1:52 PM

well the combination of all parameters make a good workflow

Furkan Gözükara SECourses•5/9/25, 1:53 PM

so you should research good params there with testing all combinations

FFurkan Gözükara SECourses and fine tune again

Escanor•5/9/25, 1:55 PM

Yeah, I'll first try to finetune with only 8 images, will try to find the best # of steps sweet spot until not too much overfit and then generate new images and finetune again

TrumpYouAll•5/10/25, 4:11 PM

@Furkan Gözükara SECourses I am not 'urging' u to make a video now but I would like to know if Kohya can be used to train HiDream model (with my personal pics) like Flux ? Can I do it in Massed Compute ?

Escanor•5/10/25, 5:50 PM

If I train model A for 3200 steps and model B for 5000 steps, both having the same base model, would the result of generating an image be the same for both if for model B I use a checkpoint that's near 3200 steps like model A (save every x epoch/steps)?

And if I train a model for 3200 steps and then retrain it for another 1800 steps for a total of 5000 steps, will that model be as efficient as if I trained a model to 5000 steps directly?

Thank you

EEscanor Yeah, I'll first try to finetune with only 8 images, will try to find the best #...

Furkan Gözükara SECourses•5/11/25, 11:30 PM

great

TTrumpYouAll <@205854764540362752> I am not 'urging' u to make a video now but I would like t...

Furkan Gözükara SECourses•5/11/25, 11:30 PM

that is also on my list but so many stuff

EEscanor If I train model A for 3200 steps and model B for 5000 steps, both having the sa...

Furkan Gözükara SECourses•5/11/25, 11:31 PM

yes it should be same 3200 + 1800 = 5000 directly

Furkan Gözükara SECourses•5/11/25, 11:31 PM

since our learning rates are all constant

Escanor•5/13/25, 3:38 PM

@Furkan Gözükara SECourses Is the kohya dreambooth sdxl 48gb config usable with the kohya sdxl lora extract configuration that you have in one of your patreon posts (not sure if I can post image here)?

I am asking because if I leave the default settings of Clamp Quantile at 0.99 and Minimum difference to 0.01, I think it says the text encoder does not need to be extracted or something similar and when I try to use the lora, it simply does not show anything of my trained model compared to if I use the settings in your image of 0.1 for Clamp Quantile and 0.00001 for Minimum difference where now it works fine.

It just made me wonder if that screenshot of the kohya sdxl lora extract settings was up to date with the latest 48gb config of kohya dreambooth settings!

I did see a recent post for lora extract, but it was for flux so I wasn't sure

Thank you for the great support!

EEscanor <@205854764540362752> Is the kohya dreambooth sdxl 48gb config usable with the k...

Furkan Gözükara SECourses•5/13/25, 4:37 PM

make it 0.000001 to extract

Furkan Gözükara SECourses•5/13/25, 4:37 PM

also flux post is updated too but flux extract not very good

JAV_Pomoe•5/14/25, 1:54 AM

Doctor, could you please tell me the best settings for LoRA training for SDXL? I saw your previous post about fine tuning... Have you already posted about LoRA training?

<@205854764540362752> is there a tutorial or explaination on how to determine if the model is overfi

Similar Threads