My training Starts! I jus change the server to 1 x RTX 4090 8 vCPU 31 GB RAM!
My training Starts! I jus change the server to 1 x RTX 4090 8 vCPU 31 GB RAM!
S3hrs to go!
SThis is really nice!
R@Dr. Furkan Gözükara have you tested xformers/gradient checkpointing as well as Memory Efficient Attention? I am running without Gradient CH. and xformers but still have memory efficient attention on. Not sure if it decreases quality
RAlso, wit A6000, on runpod, running 14 images dreambooth LoRA, at 4 batch size, I run into OOM, without gradient checkpointing and xformers. Using class images. Any thoughts if there are any memory optimizations better than these two? What do you recommend @Dr. Furkan Gözükara ??
F220 pro photos is the key
Fi dont knoıw if colab supporting captions never tried
Fbut colab is working
Fi did training few days ago
FYouTubeSECourses
Our Discord : https://discord.gg/HbqgGaZVmr. How to do free Stable Diffusion DreamBooth training on Google Colab for free. If I have been of assistance to you and you would like to show your support for my work, please consider becoming a patron on 
https://www.patreon.com/SECourses
Playlist of Stable Diffusion Tutorials, Automatic1111 and Goo...
https://www.patreon.com/SECoursesPlaylist of Stable Diffusion Tutorials, Automatic1111 and Goo...
Fi havent tested but it wasnt working like yesterday
Fit is super strong
Frtx 4090 sper strong on unix
Fi think memory efficient attention doing nothing 
Fdont enable gradient checkpointing
Fenable xformers
Ftry to get max batch size
F= max speed
Rbs 4 i did gradient + xformers. Speed is great already, around 3h compared to 6 hours for batch 2. I will try bs4 with xformers only.
Rquality is pretty awesome from epoch 1 already
Rby the way, what is your understanding of max steps. If batch 1 is good at 8000 steps, when you go to batch 2, you have a total of 4000 steps. Do you have a formula already for how many extra steps to give the higher batch to compensate for loss in quality? Generalisation though seems to improve up to a certain number of batch increases.
Rok, xformers only, no gradient checkpointing goes OOM
Sreally impressed by how cinematic some of these are coming out
Fthere is no formula
Fbut bs 2 means in 1 step it makes 2 steps
F3 means 3 steps
FF
EHey gang, does anyone have experience on downloading safetensor files from runpod, to your local machine? 1 safetensor, which can be 5-7 gb, can take a half hour for me depending on the connection.
KI have this error, I get this error when I train my own face. Does anyone know any solution?
Kand now 
NAre there any parameters that are not up to date?
SGuys. My RTX3060 12gb is very limited. Can’t really train SDXL models on it and SD1.5 dreambooth training takes overnight. Shoul I try some cloud services? How much vram does Kaggle give? Is it enough for SDXL model training? What do you guys use?
Rcool
Rby the way, @Dr. Furkan Gözükara you said text encoder is by default turned off, on kohya_ss sdxl. So, does it matter if i set text encoder LR to 0?
RShouldnt affect the trainging correct?
GHas anyone ran any tests on different optimizers using Kohya ss? to see whatg provides best results?
Dbeen testing for a week now, trained about 40 sdxl models, i'd say either go for adafactor or adamw8bit, I know some people have used Prodigy.
Here Test One with adamw8bit is white shirt sample
and test two adafactor is yellow shirt sample
Here Test One with adamw8bit is white shirt sample
and test two adafactor is yellow shirt sample
GGot it. It's hard to say which one is better off of this. Was thinking about trying Lion Optimizer because I liked that when training in A1111
Fbest way is upload into hugging face
Fthen download from there :d
Fye extension broken
Fyou need a specific version
FFon colab i never optimized parameters
Fit doesnt matter i think
Fi thin adafactor is best