bs 4 i did gradient + xformers. Speed is great already, around 3h compared to 6 hours for batch 2. I
bs 4 i did gradient + xformers. Speed is great already, around 3h compared to 6 hours for batch 2. I will try bs4 with xformers only.
Rquality is pretty awesome from epoch 1 already
Rby the way, what is your understanding of max steps. If batch 1 is good at 8000 steps, when you go to batch 2, you have a total of 4000 steps. Do you have a formula already for how many extra steps to give the higher batch to compensate for loss in quality? Generalisation though seems to improve up to a certain number of batch increases.
Rok, xformers only, no gradient checkpointing goes OOM
Sreally impressed by how cinematic some of these are coming out
Fthere is no formula
Fbut bs 2 means in 1 step it makes 2 steps
F3 means 3 steps
FF
EHey gang, does anyone have experience on downloading safetensor files from runpod, to your local machine? 1 safetensor, which can be 5-7 gb, can take a half hour for me depending on the connection.
KI have this error, I get this error when I train my own face. Does anyone know any solution?
Kand now 
NAre there any parameters that are not up to date?
SGuys. My RTX3060 12gb is very limited. Can’t really train SDXL models on it and SD1.5 dreambooth training takes overnight. Shoul I try some cloud services? How much vram does Kaggle give? Is it enough for SDXL model training? What do you guys use?
Rcool
Rby the way, @Dr. Furkan Gözükara you said text encoder is by default turned off, on kohya_ss sdxl. So, does it matter if i set text encoder LR to 0?
RShouldnt affect the trainging correct?
GHas anyone ran any tests on different optimizers using Kohya ss? to see whatg provides best results?
Dbeen testing for a week now, trained about 40 sdxl models, i'd say either go for adafactor or adamw8bit, I know some people have used Prodigy.
Here Test One with adamw8bit is white shirt sample
and test two adafactor is yellow shirt sample
Here Test One with adamw8bit is white shirt sample
and test two adafactor is yellow shirt sample
GGot it. It's hard to say which one is better off of this. Was thinking about trying Lion Optimizer because I liked that when training in A1111
Fbest way is upload into hugging face
Fthen download from there :d
Fye extension broken
Fyou need a specific version
F
Fon colab i never optimized parameters
Fit doesnt matter i think
Fi thin adafactor is best
Fbut it depends other config too
DWill also test it as well soon, these ones i trained straight on lora though, not on dreambooth
Flion was best with that extension correct
Fbut didnt work well for me with kohya
FDi've noticed more flexibility when uses multiple batch sizes like 3-4-5
Dthough some quality tradeoff
GYou provide in this best settings based on different GB sizes. Did you use runpod to do these tests?
GAlso is this for Loras or Dreambooth? I noticed the # of epochs is 4, that seems small.
Fyes i use them on runpod too
GGot it. Thank you for sharing
F4 epoch is for 40 repeat
Fso 160 epochs
F
PI'm training LoRa SDXL on my 3060 with 12GB VRAM, 768*768 resolution and it's showing 2 hours and 30 minutes of training time. I've used 15 images with 20 repeats. Is this configuration fine?
--num_cpu_threads_per_process=2 "./sdxl_train_network.py"
--network_alpha="1"
--save_model_as=safetensors
--network_module=networks.lora
--text_encoder_lr=0.0004
--unet_lr=0.0004
--network_dim=32
--output_name="local_test_1"
--lr_scheduler_num_cycles="8"
--no_half_vae
--learning_rate="0.0004"
--lr_scheduler="constant"
--train_batch_size="1"
--max_train_steps="4800"
--save_every_n_epochs="1"
--mixed_precision="fp16"
--save_precision="fp16"
--caption_extension=".txt"
--cache_latents
--cache_latents_to_disk
--optimizer_type="Adafactor"
--optimizer_args scale_parameter=False relative_step=False warmup_init=False
--max_data_loader_n_workers="0"
--bucket_reso_steps=64
--gradient_checkpointing
--full_fp16
--xformers
--bucket_no_upscale
--noise_offset=0.0
--lowram
--num_cpu_threads_per_process=2 "./sdxl_train_network.py"
--network_alpha="1"
--save_model_as=safetensors
--network_module=networks.lora
--text_encoder_lr=0.0004
--unet_lr=0.0004
--network_dim=32
--output_name="local_test_1"
--lr_scheduler_num_cycles="8"
--no_half_vae
--learning_rate="0.0004"
--lr_scheduler="constant"
--train_batch_size="1"
--max_train_steps="4800"
--save_every_n_epochs="1"
--mixed_precision="fp16"
--save_precision="fp16"
--caption_extension=".txt"
--cache_latents
--cache_latents_to_disk
--optimizer_type="Adafactor"
--optimizer_args scale_parameter=False relative_step=False warmup_init=False
--max_data_loader_n_workers="0"
--bucket_reso_steps=64
--gradient_checkpointing
--full_fp16
--xformers
--bucket_no_upscale
--noise_offset=0.0
--lowram
MIt's interesting, because I also have an RTX 3060 12 GB card and I can train a person Lora in 20 minutes on SD 1.5 with 512x512
MI would use 7 repeat and classification images with 10 epochs, no captions, 128/64 weights. For Adafactor use classic parameters for LR and Unet LR: 0.0001, TE: 5e-05, token length: 225, model: RealisticVision 5.1, optimizer args: scale_parameter=False relative_step=False warmup_init=False weight_decay=0.01
PHow were the results obtained?
