I was wondering, do warmup steps damage a lora?
I was wondering, do warmup steps damage a lora?
Fwarmup steps means lower or higher LR
Fmay damage
YI am making a LoRA of a realistic character. The training data contain face swapped images and most have too much glare around the eyes. If I were to include a caption about eyes having too much glare, (say "eye glare") would the LoRA train to only keep the eye glare if "eye glare" were in the prompt, and otherwise not include the eye glare if it were not mentioned?
Ftraining for flux?
Yyes
Fflux already has automatic inner caption system
Fbut you can add eye glare to caption
Fafter training dont use in prompt
Fmay reduce eye glare effect
Fafter training
YOk that's what I was thinking
YThank you
Fyou are welcome
Xhmm, I've been using AdamW with 0.0002 and warmup steps 3, cosine
NHi have you posted the video yet?
Fediting
Frecorded
F
NI have already prepared the date set, here waiting very meekly for your labour)))))))
Ndisabeling t5 att masking saves 2 seconds on it. Rank 5
Fyep
Falso i am about to put
Ftorch 2.5 installer
Fit will give huge boost
Flet me update
GI documented all my configurations when I was using Kohya_ss. Can these be ported directly into OneTrainer for LoRA model training?
Fvery likely
Fi did for sdxl
Ndoes this mean i missed updating something? thought text encoder was supposed to work now?
024-08-28 09:54:18 INFO text_encoder is not needed for training. deleting to save memory. train_network.py:1013
INFO unet dtype: torch.float8_e4m3fn, device: cuda:0 train_network.py:1030
024-08-28 09:54:18 INFO text_encoder is not needed for training. deleting to save memory. train_network.py:1013
INFO unet dtype: torch.float8_e4m3fn, device: cuda:0 train_network.py:1030
Fi didnt update text encoder configs yet
Fi am still researching
Fwill announce once made update
Fbut you can copy pate unet LR to text encoder
Fand it works 
Fare you using latest speed boost?=
Fwe need to verify it works :d
Ni am, but testing something at 512 so can't compare speed to my 1024 runs
Nrank 5 with updated torch (I guess updated, but I did update)...1024, without t5 att masking
Fi hope it works perfect
Fi will be able to test only during night
NI will not finish this training, so I can't tell you result. But maybe I will try to start training at night. So could be some results tomorrow...or not)
Fi will do same :d
MHello everyone, quick question: what does this script do?
https://huggingface.co/MonsterMMORPG/Generative-AI/resolve/main/kohua_gui_pass_update.sh
As far as I understand, it replaces kohya_ss's
https://huggingface.co/MonsterMMORPG/Generative-AI/resolve/main/kohua_gui_pass_update.sh
As far as I understand, it replaces kohya_ss's
gui.shwhen you use Furkan's kohya installer script (at least on Linux). What's the advantage? Thanks…
Dokay, lets see the local flux kohya training
NI think updates a few libraries to newer versions, then patches original gui to not roll them back? not sure though.
Ddo we still need to use the kohya training folder structure? even with the folder name #_name_class?
Nthat's what kohya is still expecting as of now (space between name and class though, not _ )