Software Engineering Courses (SECourses)•10mo ago

<@205854764540362752> I am trying out dreambooth training on runpod using your guide. I got it to ru

@Furkan Gözükara SECourses I am trying out dreambooth training on runpod using your guide. I got it to run but out of memory on a 4090.

Any guidance would be helpful.

------------------------------------
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 48.00 MiB. GPU 0 has a total capacity of 23.64 GiB of which 20.81 MiB is free. Process 3037847 has 23.62 GiB memory in use. Of the allocated memory 23.08 GiB is allocated by PyTorch, and 86.61 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)"

Uumrfarooq Hi <@205854764540362752> , I need some advice as I have been struggling with Lo...

Furkan Gözükara SECourses•3/22/25, 2:07 AM

hey your trigger word is wrong. it is many multiple words combination

Furkan Gözükara SECourses•3/22/25, 2:07 AM

i recommend watch this video no skip

Furkan Gözükara SECourses•3/22/25, 2:07 AM

https://youtu.be/FvpWy1x5etM

YouTubeSECourses

FLUX Full Fine-Tuning / DreamBooth Training Master Tutorial for Win...

If you want to train FLUX with maximum possible quality, this is the tutorial looking for. In this comprehensive tutorial, you will learn how to install Kohya GUI and use it to fully Fine-Tune / DreamBooth FLUX model. After that how to use SwarmUI to compare generated checkpoints / models and find the very best one to generate most amazing image...

Hhau <@205854764540362752> hey, really enjoyed your videos. would appreciate a deep-...

Furkan Gözükara SECourses•3/22/25, 2:08 AM

i already did and full dreambooth on flux works as low as 6 GB GPUs

Furkan Gözükara SECourses•3/22/25, 2:08 AM

with 64 GB RAM : https://youtu.be/FvpWy1x5etM

YouTubeSECourses

FLUX Full Fine-Tuning / DreamBooth Training Master Tutorial for Win...

JJoshua A <@205854764540362752> I am trying out dreambooth training on runpod using your g...

Furkan Gözükara SECourses•3/22/25, 2:08 AM

yes which config you used?

Furkan Gözükara SECourses•3/22/25, 2:08 AM

and are your images all 1024x1024?

FFurkan Gözükara SECourses yes which config you used?

Joshua AOP•3/22/25, 2:14 AM

Yes. All images are 1024. The config is 24GB_GPU_22900MB_7.3_second_it_Tier_1.

Mentioning this because I think it's strange. I am noticing when I add the preset to kohya preset nothing changes. I have to go through and manually change everything. and when I save the config. It doesn't save. refreshing the browser will erase all data.

I have 200gb storage, 30 gb of local storage like you say in the video.

JJoshua A Yes. All images are 1024. The config is 24GB_GPU_22900MB_7.3_second_it_Tier_1. ...

Furkan Gözükara SECourses•3/22/25, 2:14 AM

that means you didnt load config accurately

Furkan Gözükara SECourses•3/22/25, 2:14 AM

it has to load everything

Furkan Gözükara SECourses•3/22/25, 2:15 AM

tell me your config path you tried to load

FFurkan Gözükara SECourses that means you didnt load config accurately

Joshua AOP•3/22/25, 2:17 AM

Let me check. this is also what I am running it on. pytorch:2.2.0-py3.10-cuda12.1.

Joshua AOP•3/22/25, 2:17 AM

Should I make sure the folders don't have spaces in the names? I keep remembering runpod doesn't like spaces.

JJoshua A Should I make sure the folders don't have spaces in the names? I keep rememberin...

Furkan Gözükara SECourses•3/22/25, 2:19 AM

yes never use space anywhere

Furkan Gözükara SECourses•3/22/25, 2:19 AM

runpod starts with

Furkan Gözükara SECourses•3/22/25, 2:19 AM

/workspace/ etc

Joshua AOP•3/22/25, 2:21 AM

/workspace/dreambooth/DreamBooth_Tab_Fine_Tuning_Best_FLUX_Configs/24GB_GPU_22900MB_7.3_second_it_Tier_1.json

FFurkan Gözükara SECourses yes never use space anywhere

Joshua AOP•3/22/25, 2:21 AM

It looks like the config file path is ok.

JJoshua A It looks like the config file path is ok.

Furkan Gözükara SECourses•3/22/25, 2:24 AM

yes you loaded into dreambooth tab

Furkan Gözükara SECourses•3/22/25, 2:24 AM

or lora tab

FFurkan Gözükara SECourses yes never use space anywhere

Joshua AOP•3/22/25, 2:24 AM

Can I use flux1-dev-fp8.safetensors for training or do I need to use the original?

JJoshua A Can I use flux1-dev-fp8.safetensors for training or do I need to use the origina...

Furkan Gözükara SECourses•3/22/25, 2:24 AM

use original

Joshua AOP•3/22/25, 2:24 AM

It was the dreambooth tab. I didn't use the original so I can try that out now.

JJoshua A It was the dreambooth tab. I didn't use the original so I can try that out now.

Furkan Gözükara SECourses•3/22/25, 2:25 AM

ye try it

FFurkan Gözükara SECourses ye try it

Joshua AOP•3/22/25, 2:29 AM

That one strange thing is the config file isn't doing anyhting. very strange to me. It didn't save any of the work from before.

Screenshot_2025-03-21_at_10.29.25_PM.png

JJoshua A That one strange thing is the config file isn't doing anyhting. very strange to ...

Furkan Gözükara SECourses•3/22/25, 2:29 AM

click both

Furkan Gözükara SECourses•3/22/25, 2:29 AM

Furkan Gözükara SECourses•3/22/25, 2:29 AM

it has to load values

Furkan Gözükara SECourses•3/22/25, 2:30 AM

FFurkan Gözükara SECourses click both

Joshua AOP•3/22/25, 2:38 AM

It seemed to have saved this time. What do I put if I only want to save the 160th epoch in the "Save every N epochs" .

Screenshot_2025-03-21_at_10.36.02_PM.png

Joshua AOP•3/22/25, 2:39 AM

this is what the config file has loaded minus the save every n epoch number. I added 160.

FFurkan Gözükara SECourses i already did and full dreambooth on flux works as low as 6 GB GPUs

hau•3/22/25, 2:51 AM

amazing --- where are you doing full dreambooth? which script in kohya does that.... not seeing it in the video

hau•3/22/25, 2:55 AM

there doesn't seem to be any script in kohya that lets you do dreambooth with flux

FFurkan Gözükara SECourses ye try it

Joshua AOP•3/22/25, 2:59 AM

I went with the 16gb config file and its working. it looks like there was something that was causing it to fail at 24gb.

I don't know if this matters but I am getting this during the training. however this was also happening during the 16gb training too.

2025-03-22 02:58:21.761881: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.

Hhau there doesn't seem to be any script in kohya that lets you do dreambooth with fl...

hau•3/22/25, 3:01 AM

this script will let you do full flux finetuning on low vram GPUs

https://github.com/kohya-ss/sd-scripts/blob/sd3/flux_train.py

but there is no script that does dreambooth training? so it seems like the video you posted is not using dreambooth -- please correct me if i am wrong

@Furkan Gözükara SECourses

GitHub

sd-scripts/flux_train.py at sd3 · kohya-ss/sd-scripts

Contribute to kohya-ss/sd-scripts development by creating an account on GitHub.

hau•3/22/25, 3:03 AM

unless this line implies using dreambooth: https://github.com/kohya-ss/sd-scripts/blob/sd3/flux_train.py#L88

GitHub

sd-scripts/flux_train.py at sd3 · kohya-ss/sd-scripts

Contribute to kohya-ss/sd-scripts development by creating an account on GitHub.

hau•3/22/25, 3:03 AM

wardensc2•3/22/25, 3:26 AM

Hi @Furkan Gözükara SECourses I just download your config for SDXL training. For training dreambooth for character how many regulation images from your database you think we need to use train to get good dreambooth character model

sai_150•3/22/25, 8:07 AM

Hi @Furkan Gözükara SECourses , I've 33 images of me. can you suggest how many repeats I should consider(config has 40 repeats and 1 epoch, that gives 1320 steps).

sai_150•3/22/25, 8:11 AM

what model should I use for the best results sdxl base or any custom model from Civitai (like realvis). Is this the best method for generating images of a specific person?

JJoshua A It seemed to have saved this time. What do I put if I only want to save the 160t...

Furkan Gözükara SECourses•3/22/25, 12:02 PM

my config doesnt have save every 160 epooch