Software Engineering Courses (SECourses)•11mo ago

The grid test I am analyzing to find newer FLUX DEV training. This is not even including the first 1

The grid test I am analyzing to find newer FLUX DEV training. This is not even including the first 10 different trainings i made

1176 images. I think we gonna have a better workflow than what we have with using more VRAM and more training time.

Renzo•2/21/25, 1:46 PM

In case someone is interested in creating LoRAs from Lumina2: https://huggingface.co/sayakpaul/trained-lumina2-lora-yarn, https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_lumina2.md (the code).

GitHub

diffusers/examples/dreambooth/README_lumina2.md at main · huggingfa...

Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch and FLAX. - huggingface/diffusers

FFurkan Gözükara SECourses The grid test I am analyzing to find newer FLUX DEV training. This is not even i...

Mathew•2/21/25, 6:49 PM

Its Lora train or checkpoint ?

MMathew Its Lora train or checkpoint ?

Furkan Gözükara SECoursesOP•2/21/25, 11:46 PM

fine tuning atm

Furkan Gözükara SECoursesOP•2/21/25, 11:46 PM

after that i plan lora

ghstoic•2/22/25, 10:42 AM

For multi-GPU training, does the set-up need 80gb vram TOTAL, or does EACH gpu need 80gb vram?

Gghstoic For multi-GPU training, does the set-up need 80gb vram TOTAL, or does EACH gpu n...

Furkan Gözükara SECoursesOP•2/22/25, 11:23 AM

for fine tuning

Furkan Gözükara SECoursesOP•2/22/25, 11:23 AM

each gpu 80 gb

Furkan Gözükara SECoursesOP•2/22/25, 11:23 AM

for lora not necessary 48 gb gpus works great

FFurkan Gözükara SECourses each gpu 80 gb

ghstoic•2/22/25, 12:25 PM

Gottcha, thank you!

ghstoic•2/22/25, 12:27 PM

Out of interest, what causes the VRAM per GPU requirements to jump up so much? Just curious on what's going on in the background processes that requires VRAM requirement to go from 24gb VRAM per GPU from single training to 80gb VRAM per GPU for muli-traning

Gghstoic Out of interest, what causes the VRAM per GPU requirements to jump up so much? J...

Furkan Gözükara SECoursesOP•2/22/25, 12:45 PM

extremely good question

Furkan Gözükara SECoursesOP•2/22/25, 12:45 PM

i asked this to kohya

Furkan Gözükara SECoursesOP•2/22/25, 12:46 PM

he was also not sure

Timson•2/22/25, 11:51 PM

How does multy GPU trainign work anyways?
Each GPU has a copy of a trained model and perform traning steps independantly, so how in the end we are not gettign two separate models?

Timson•2/22/25, 11:52 PM

they cannot possibly synchronize model state after each step, that woudl take ages (minutes)

Timson•2/22/25, 11:55 PM

And if the models are just merged as last step - would that mean the final model can have questionable quality (missed local minimas and such)?

TTimson How does multy GPU trainign work anyways? Each GPU has a copy of a trained model...

Furkan Gözükara SECoursesOP•2/23/25, 10:01 AM

with diffuser models

Furkan Gözükara SECoursesOP•2/23/25, 10:01 AM

each models loaded fully into vram

Furkan Gözükara SECoursesOP•2/23/25, 10:02 AM

like flux sd etc

Furkan Gözükara SECoursesOP•2/23/25, 10:02 AM

so we have exact replicate

FFurkan Gözükara SECourses each models loaded fully into vram

Timson•2/23/25, 10:41 AM

yeah, but how the state of each model loaded into GPU sync up?

Timson•2/23/25, 10:41 AM

So the training result in single model, not multiple models

TTimson yeah, but how the state of each model loaded into GPU sync up?

Furkan Gözükara SECoursesOP•2/23/25, 10:45 AM

they are snyched at steps

Furkan Gözükara SECoursesOP•2/23/25, 10:45 AM

so it is being increased batch size

novsky•2/23/25, 4:31 PM

long time no see. I had trained few flux loras earlier, and had to reinstall my windows recently. After that I had to reinstall kohya, and now can't train anymore. First it didn't create venv (idk, maybe it's because latest installer or something else). I've installed kohya from original repo, and it started to work. But now I have another problem. Training speed reduced. Much. ~500s/it. I guess it's because it's using ram or something. Earlier I noticed it's caching t5 and clip-l to ram (to free vram), now there is no such notice. Any suggestions?

novsky•2/23/25, 4:32 PM

gpu 4060ti 16gb, ram 64 gb

novsky•2/23/25, 4:32 PM