Software Engineering Courses (SECourses)•11mo ago

I did imagine that. But I watched <@205854764540362752> tutorial and saw that he had some incredibl

I did imagine that. But I watched @Furkan Gözükara SECourses tutorial and saw that he had some incredibly stylized images... I think he also only trained with realistic images. That's where my doubt comes from

EElli_Oaken Hi! Don't know which channel to write this on. I'm training a lora and I'm wonde...

Furkan Gözükara SECourses•2/20/25, 10:48 AM

yes buyt i never tried

Furkan Gözükara SECourses•2/20/25, 10:48 AM

its option is here

Furkan Gözükara SECourses•2/20/25, 10:49 AM

MMetaKan I have finetuned flux-dev with 180 images of a character and 180 photos from var...

Furkan Gözükara SECourses•2/20/25, 10:49 AM

yes flux is very realistic

Furkan Gözükara SECourses•2/20/25, 10:49 AM

therefore i recommend 2 step training for stylized images

Furkan Gözükara SECourses•2/20/25, 10:49 AM

first train, generate some good stylized images, and retrain on them

MMetaKan I did imagine that. But I watched <@205854764540362752> tutorial and saw that h...

Furkan Gözükara SECourses•2/20/25, 10:50 AM

i had 256 images and dataset was variety . also still i had hard time and needed to find prompts

MetaKanOP•2/20/25, 10:52 AM

thank you sir!

Furkan Gözükara SECourses•2/20/25, 10:56 AM

The grid test I am analyzing to find newer FLUX DEV training. This is not even including the first 10 different trainings i made

1176 images. I think we gonna have a better workflow than what we have with using more VRAM and more training time.

Renzo•2/21/25, 1:46 PM

In case someone is interested in creating LoRAs from Lumina2: https://huggingface.co/sayakpaul/trained-lumina2-lora-yarn, https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_lumina2.md (the code).

GitHub

diffusers/examples/dreambooth/README_lumina2.md at main · huggingfa...

Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch and FLAX. - huggingface/diffusers

FFurkan Gözükara SECourses The grid test I am analyzing to find newer FLUX DEV training. This is not even i...

Mathew•2/21/25, 6:49 PM

Its Lora train or checkpoint ?

MMathew Its Lora train or checkpoint ?

Furkan Gözükara SECourses•2/21/25, 11:46 PM

fine tuning atm

Furkan Gözükara SECourses•2/21/25, 11:46 PM

after that i plan lora

ghstoic•2/22/25, 10:42 AM

For multi-GPU training, does the set-up need 80gb vram TOTAL, or does EACH gpu need 80gb vram?

Gghstoic For multi-GPU training, does the set-up need 80gb vram TOTAL, or does EACH gpu n...

Furkan Gözükara SECourses•2/22/25, 11:23 AM

for fine tuning

Furkan Gözükara SECourses•2/22/25, 11:23 AM

each gpu 80 gb

Furkan Gözükara SECourses•2/22/25, 11:23 AM

for lora not necessary 48 gb gpus works great

FFurkan Gözükara SECourses each gpu 80 gb

ghstoic•2/22/25, 12:25 PM

Gottcha, thank you!

ghstoic•2/22/25, 12:27 PM

Out of interest, what causes the VRAM per GPU requirements to jump up so much? Just curious on what's going on in the background processes that requires VRAM requirement to go from 24gb VRAM per GPU from single training to 80gb VRAM per GPU for muli-traning

Gghstoic Out of interest, what causes the VRAM per GPU requirements to jump up so much? J...

Furkan Gözükara SECourses•2/22/25, 12:45 PM

extremely good question

Furkan Gözükara SECourses•2/22/25, 12:45 PM

i asked this to kohya

Furkan Gözükara SECourses•2/22/25, 12:46 PM

he was also not sure

Timson•2/22/25, 11:51 PM

How does multy GPU trainign work anyways?
Each GPU has a copy of a trained model and perform traning steps independantly, so how in the end we are not gettign two separate models?

Timson•2/22/25, 11:52 PM

they cannot possibly synchronize model state after each step, that woudl take ages (minutes)

Timson•2/22/25, 11:55 PM

And if the models are just merged as last step - would that mean the final model can have questionable quality (missed local minimas and such)?

TTimson How does multy GPU trainign work anyways? Each GPU has a copy of a trained model...

Furkan Gözükara SECourses•2/23/25, 10:01 AM

with diffuser models

Furkan Gözükara SECourses•2/23/25, 10:01 AM

each models loaded fully into vram

Furkan Gözükara SECourses•2/23/25, 10:02 AM

like flux sd etc

Furkan Gözükara SECourses•2/23/25, 10:02 AM

so we have exact replicate

FFurkan Gözükara SECourses each models loaded fully into vram

Timson•2/23/25, 10:41 AM

yeah, but how the state of each model loaded into GPU sync up?

Timson•2/23/25, 10:41 AM

So the training result in single model, not multiple models

TTimson yeah, but how the state of each model loaded into GPU sync up?

Furkan Gözükara SECourses•2/23/25, 10:45 AM

they are snyched at steps

Furkan Gözükara SECourses•2/23/25, 10:45 AM

so it is being increased batch size

novsky•2/23/25, 4:31 PM

long time no see. I had trained few flux loras earlier, and had to reinstall my windows recently. After that I had to reinstall kohya, and now can't train anymore. First it didn't create venv (idk, maybe it's because latest installer or something else). I've installed kohya from original repo, and it started to work. But now I have another problem. Training speed reduced. Much. ~500s/it. I guess it's because it's using ram or something. Earlier I noticed it's caching t5 and clip-l to ram (to free vram), now there is no such notice. Any suggestions?

novsky•2/23/25, 4:32 PM

gpu 4060ti 16gb, ram 64 gb

novsky•2/23/25, 4:32 PM