turn off t5 text encoder masking attention
turn off t5 text encoder masking attention
Fvery little quality loss
Fspeed up signficiant
Fif still slow reduce training resolut
Flike 768x768
Fthis will hve more impact on quality
Ngot it. Will be trying new parameters tomorrow. Will lyk results
Nthanks
Fyou are welcome
XI was wondering, do warmup steps damage a lora?
Fwarmup steps means lower or higher LR
Fmay damage
YI am making a LoRA of a realistic character. The training data contain face swapped images and most have too much glare around the eyes. If I were to include a caption about eyes having too much glare, (say "eye glare") would the LoRA train to only keep the eye glare if "eye glare" were in the prompt, and otherwise not include the eye glare if it were not mentioned?
Ftraining for flux?
Yyes
Fflux already has automatic inner caption system
Fbut you can add eye glare to caption
Fafter training dont use in prompt
Fmay reduce eye glare effect
Fafter training
YOk that's what I was thinking
YThank you
Fyou are welcome
Xhmm, I've been using AdamW with 0.0002 and warmup steps 3, cosine
NHi have you posted the video yet?
Fediting
Frecorded
F
NI have already prepared the date set, here waiting very meekly for your labour)))))))
Ndisabeling t5 att masking saves 2 seconds on it. Rank 5
Fyep
Falso i am about to put
Ftorch 2.5 installer
Fit will give huge boost
Flet me update
GI documented all my configurations when I was using Kohya_ss. Can these be ported directly into OneTrainer for LoRA model training?
Fvery likely
Fi did for sdxl
Ndoes this mean i missed updating something? thought text encoder was supposed to work now?
024-08-28 09:54:18 INFO text_encoder is not needed for training. deleting to save memory. train_network.py:1013
INFO unet dtype: torch.float8_e4m3fn, device: cuda:0 train_network.py:1030
024-08-28 09:54:18 INFO text_encoder is not needed for training. deleting to save memory. train_network.py:1013
INFO unet dtype: torch.float8_e4m3fn, device: cuda:0 train_network.py:1030
Fi didnt update text encoder configs yet
Fi am still researching
Fwill announce once made update
Fbut you can copy pate unet LR to text encoder
Fand it works 
Fare you using latest speed boost?=
Fwe need to verify it works :d
Ni am, but testing something at 512 so can't compare speed to my 1024 runs
Nrank 5 with updated torch (I guess updated, but I did update)...1024, without t5 att masking
