Software Engineering Courses (SECourses)•10mo ago

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 14.74 GiB of which 14.12 MiB is free. Process 16031 has 14.72 GiB memory in use. Of the allocated memory 14.26 GiB is allocated by PyTorch, and 344.88 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.

sai_150OP•3/13/25, 6:13 PM

I'm getting out of memory error

Ssai_150 torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 h...

Furkan Gözükara SECourses•3/13/25, 6:41 PM

hello

Furkan Gözükara SECourses•3/13/25, 6:41 PM

i just tested and it works

Furkan Gözükara SECourses•3/13/25, 6:41 PM

even multi gpu works

Furkan Gözükara SECourses•3/13/25, 6:41 PM

make sure your all images are 1024x1024

Furkan Gözükara SECourses•3/13/25, 6:41 PM

let me update the notebook

Furkan Gözükara SECourses•3/13/25, 6:42 PM

Furkan Gözükara SECourses•3/13/25, 6:42 PM

training on 2x gpu thus 2x faster

Furkan Gözükara SECourses•3/13/25, 6:42 PM

you can use our configs auto downloaded

Furkan Gözükara SECourses•3/13/25, 6:42 PM

Ssai_150 torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 h...

Furkan Gözükara SECourses•3/13/25, 6:46 PM

use v25 : https://www.patreon.com/posts/88397937

Patreon

A Kaggle Notebook to Train Stable Diffusion 1.5 and XL (SDXL) on a ...

Get more from SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts on Patreon

sai_150OP•3/13/25, 6:54 PM

Thanks for the quick response. In my case, I see only one GPU is used. I'll tweak the accelerate launch parameters and try again.

Ssai_150 Thanks for the quick response. In my case, I see only one GPU is used. I'll twea...

Furkan Gözükara SECourses•3/13/25, 6:55 PM

yes but single gpu should work too

Furkan Gözükara SECourses•3/13/25, 6:55 PM

i tested both

Furkan Gözükara SECourses•3/13/25, 6:56 PM

make sure all images 1024x1024

Furkan Gözükara SECourses•3/13/25, 6:56 PM

and using our config

sai_150OP•3/13/25, 7:06 PM

images are 1024X1024 only, I'll try again with v25 tomorrow

FFurkan Gözükara SECourses yes but single gpu should work too

sai_150OP•3/14/25, 11:25 AM

v25 worked. Thanks for the help. But I couldn't use both the gpus, when I've selected multi_gpu and Number of processes=2 it is throwing an error. I ran it with one gpu.

Ssai_150 v25 worked. Thanks for the help. But I couldn't use both the gpus, when I've sel...

Furkan Gözükara SECourses•3/14/25, 10:44 PM

on windows not working as far as i know

Furkan Gözükara SECourses•3/14/25, 10:44 PM

on linux lora works with 48 gb gpus

FFurkan Gözükara SECourses on windows not working as far as i know

sai_150OP•3/15/25, 2:14 PM

I'm running it in kaggle, providing gpu ids as 0,1 worked. I was able to do the training with both the gpus.

Ssai_150 I'm running it in kaggle, providing gpu ids as 0,1 worked. I was able to do the ...

Furkan Gözükara SECourses•3/15/25, 2:17 PM

for SDXL yes it works

Furkan Gözükara SECourses•3/15/25, 2:17 PM

on kaggle

sai_150OP•3/15/25, 6:08 PM

I saved 3 models during the training steps (1 at every 601 steps). apart from these 3, there are 6 models in the same folder namely Kaggle_SDXL_Base_DreamBooth-000001.safetensors. what are these models? can these be used to extract lora from dreambooth models(Kaggle_SDXL_Base_DreamBooth-step00001803.safetensors)

Ssai_150 I saved 3 models during the training steps (1 at every 601 steps). apart from th...

Furkan Gözükara SECourses•3/15/25, 8:59 PM

you saved both epoch based

Furkan Gözükara SECourses•3/15/25, 8:59 PM

and steps based

sai_150OP•3/16/25, 2:35 AM

got it, now I understand why the memory ran out

sai_150OP•3/16/25, 6:25 AM

I'm testing the model results, I see in some generated photos the background and clothes of training images are coming up. If I include captions while training, will this issue be resolved?

sunflowerm•3/16/25, 7:02 AM

Hi Dr Furkan, Shaun here. I'm planning to fine-tune an SDXL model checkpoint on a large number of images, hoping to improve its base performance. Some examples are like JuggernautXL , RealVis XL etc. Were they trained using dreambooth as well? Is dreambooth the best method to train a general model? As a lot of the examples of dreambooth seem to be only about training on a specific subject.

Ssai_150 I'm testing the model results, I see in some generated photos the background and...

Furkan Gözükara SECourses•3/16/25, 12:15 PM

nope

Furkan Gözükara SECourses•3/16/25, 12:16 PM

in inference instead change your prompt

Furkan Gözükara SECourses•3/16/25, 12:16 PM

define them

Furkan Gözükara SECourses•3/16/25, 12:16 PM

so they wont come

Furkan Gözükara SECourses•3/16/25, 12:16 PM

define something else instead like new background new clothing

Furkan Gözükara SECourses•3/16/25, 12:16 PM

actually if you improve dataset that is best

Ssunflowerm Hi Dr Furkan, Shaun here. I'm planning to fine-tune an SDXL model checkpoint on ...

Furkan Gözükara SECourses•3/16/25, 12:16 PM

yes they do dreambooth without regularization thus it is fine tuning

Furkan Gözükara SECourses•3/16/25, 12:17 PM

if you prepare a large enough dataset with proper captions it should work

Sspirit18 Huge Thanks to <@205854764540362752> 🥹

daquan0435•3/16/25, 2:14 PM

What ıs the promt?

Jackie•3/18/25, 2:32 AM

Hello, I am new to training DreamBooth. May I ask if the Kaggle Notebook on Patreon can be used to train the Illustrious XL model?

FFurkan Gözükara SECourses actually if you improve dataset that is best

sai_150OP•3/18/25, 3:09 AM

I'm using 13 images now with 4 unique backgrounds and clothes, can you tell me how many images are needed? how many steps should I train? config file has train batch size as 2, should I try train batch size =1. Does it help if I remove the background in the train images?