Fyep 
MYou are getting better results when you train without captions?
Fi will post on patreon and reddit
Mone more thing there is no point in checking Apply T5 Attention Mask when you are training without captions right?
Fi think iy very slightly improves
Falso with T5 training i think unet can go lower
Fit brings some quality
Fbut in some images resemblance is lost
Fso it is a choice
Fi am analyinz
M@Dr. Furkan Gözükara T5 is mainly for text trainings, t5 not worth training at all
Mi did about 60+ trainings with it
Mit'll work amazing if you're training text
Fwhat do you train text
Fwith cpu offloading it uses 23.34 gb
F24 gb may train
Fand 22.44 gb
Fsecondary config
Mkohya mentioned to me in dms.
Mhes added different learning rates for t5 and clip l
Mso he said you'll need to lower t5
to 1e-7 or 1e-6
to 1e-7 or 1e-6
Factually even 05-05 even didnt cook the model at all
Fi mean the T5
Falthough i didnt do text training
Fi compared ohwx man vs full captioned
Mt5 wont do much for people training
Mt5 is a text to text model.
you may get better results adjusting the clip l
kohya suggested to me having clip l 1/10
so for your config unet 5e-05
clip l 5e-06
you may get better results adjusting the clip l
kohya suggested to me having clip l 1/10
so for your config unet 5e-05
clip l 5e-06
Fi tried every clip LR already :d
Mbut then again you're only training with ohwx man so it wont do much
Fi tried full joy captions too
F300 tokens
Mnot worth it, if you're training that many tokens you need to lower all your rates and train for a long time
Fbut may depends on dataset
Fi will focus on fine tuning
Mtried on a100 got quite good results, idk if he fixed the extract lora thing yet, so you need to use higher weight when using it
�How about training clothes with text on them?
MThat could work. it'll work good for people who are holding up signs, text on clothings, product placements
Mbut for people training t5 is not worth it at all
�to train using t5 do I just use doc's config and select train t5 in the GUI?
Midk if he updated his config yet
Mbut you have to enable it in the advanced parameters
�Okay
Fi will update configs hopefully soon
�this right?
Mcorrect and enter this into your additonal parameters, cause i think the cpu offload checkpointing is not working.
--cpu_offlload_checkpointing
--cpu_offlload_checkpointing
�Can I DM you regarding one more thing?
Mif its feet pics no, other than that then yes 
