Dr, can you please explain to me the logic of learning rate? is lower=training better or the other w
Dr, can you please explain to me the logic of learning rate? is lower=training better or the other way around?
Fit is like a tool
Fyou need to have accurate size to
Fcut a pasta
Ftoo big, you cant precisely cut
Ftoo small, you cant cut enough
PSo how do I determind what I need to set it to train specific things?
Fyou cant
Fi do testing
Fi test literally every LR 
Pwhat's the default?
P0.0003?
Fdefault changes according to each config
Fbt usually for lora
F0.0001
Ffine tuning
F0.00001
Pwhat's fine tuning?
Ffully training the model
PI see
Pso 0.0001 trains more I suppose?
Fyep
F10x compared to 0.00001
Fyou can ask llms to get more idea
Flike chatgpt
DThank you. I realized now i missed that step. However, when i google enabling shared vram, i get nothing. I'm unsure how that is done
Fif you didnt disable i think it should be enabled
D
FVideoCardz.com
NVIDIA has a solution to Stable Diffusion maxing out the available memory Drivers have a new feature called “System Memory Fallback”. The NVIDIA Control Panel may still look old, but it still continues to receive feature updates. In a recent driver release, NVIDIA introduced a new option in the Control Panel tailored for consumer GeForce […]
Fok looking good should work
Dyeah it seems i have it
Falso verify this
F
Fprefer system fall back
D
Fok then it should work
PHi Dr, I'm trying to train a flux checkpoint with your 16GB config file but this error repeats every try and I can't figure out what is happening:
Fwhat error?
PTraceback (most recent call last):
File "Q:\kohya_ss\sd-scripts\train_db.py", line 529, in <module>
train(args)
File "Q:\kohya_ss\sd-scripts\train_db.py", line 123, in train
text_encoder, vae, unet, load_stable_diffusion_format = train_util.load_target_model(args, weight_dtype, accelerator)
File "Q:\kohya_ss\sd-scripts\library\train_util.py", line 4825, in load_target_model
text_encoder, vae, unet, load_stable_diffusion_format = _load_target_model(
File "Q:\kohya_ss\sd-scripts\library\train_util.py", line 4780, in _load_target_model
text_encoder, vae, unet = model_util.load_models_from_stable_diffusion_checkpoint(
File "Q:\kohya_ss\sd-scripts\library\model_util.py", line 1005, in load_models_from_stable_diffusion_checkpoint
converted_unet_checkpoint = convert_ldm_unet_checkpoint(v2, state_dict, unet_config)
File "Q:\kohya_ss\sd-scripts\library\model_util.py", line 267, in convert_ldm_unet_checkpoint
new_checkpoint["time_embedding.linear_1.weight"] = unet_state_dict["time_embed.0.weight"]
File "Q:\kohya_ss\sd-scripts\train_db.py", line 529, in <module>
train(args)
File "Q:\kohya_ss\sd-scripts\train_db.py", line 123, in train
text_encoder, vae, unet, load_stable_diffusion_format = train_util.load_target_model(args, weight_dtype, accelerator)
File "Q:\kohya_ss\sd-scripts\library\train_util.py", line 4825, in load_target_model
text_encoder, vae, unet, load_stable_diffusion_format = _load_target_model(
File "Q:\kohya_ss\sd-scripts\library\train_util.py", line 4780, in _load_target_model
text_encoder, vae, unet = model_util.load_models_from_stable_diffusion_checkpoint(
File "Q:\kohya_ss\sd-scripts\library\model_util.py", line 1005, in load_models_from_stable_diffusion_checkpoint
converted_unet_checkpoint = convert_ldm_unet_checkpoint(v2, state_dict, unet_config)
File "Q:\kohya_ss\sd-scripts\library\model_util.py", line 267, in convert_ldm_unet_checkpoint
new_checkpoint["time_embedding.linear_1.weight"] = unet_state_dict["time_embed.0.weight"]
PKeyError: 'time_embed.0.weight'
Traceback (most recent call last):
File "C:\Users\Marcos\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Marcos\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "Q:\kohya_ss\venv\Scripts\accelerate.EXE__main__.py", line 7, in <module>
File "Q:\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 47, in main
args.func(args)
File "Q:\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1017, in launch_command
simple_launcher(args)
File "Q:\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 637, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['Q:\kohya_ss\venv\Scripts\python.exe', 'Q:/kohya_ss/sd-scripts/train_db.py', '--config_file', 'Q://config_dreambooth-20241007-122824.toml']' returned non-zero exit status 1.
Traceback (most recent call last):
File "C:\Users\Marcos\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Marcos\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "Q:\kohya_ss\venv\Scripts\accelerate.EXE__main__.py", line 7, in <module>
File "Q:\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 47, in main
args.func(args)
File "Q:\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1017, in launch_command
simple_launcher(args)
File "Q:\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 637, in simple_launcher
raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['Q:\kohya_ss\venv\Scripts\python.exe', 'Q:/kohya_ss/sd-scripts/train_db.py', '--config_file', 'Q://config_dreambooth-20241007-122824.toml']' returned non-zero exit status 1.
Fincorrect model
Fuse our model downloader
Fuse downloaded models
PHow many repeats per image in this config? with 50 images database
FRepeat always 1 for flux
Pty
SWhat's the practical effect of training a finetune rather than a lora? At the moment I have about 15 loras running in the dev checkpoint. If I run a finetune instead, that can only incorporate 1 lora within it and bloats the size of the file? Or am I just confused about how it works? Can I merge all my loras (they are all character loras) into 1 finetune?
FFine tuning is better quality simply
FLesser overfit
FWith same dataset
FAlso fine tuning is used to make a better general model too