kohya mentioned to me in dms.
kohya mentioned to me in dms.
Mhes added different learning rates for t5 and clip l
Mso he said you'll need to lower t5
to 1e-7 or 1e-6
to 1e-7 or 1e-6
Factually even 05-05 even didnt cook the model at all
Fi mean the T5
Falthough i didnt do text training
Fi compared ohwx man vs full captioned
Mt5 wont do much for people training
Mt5 is a text to text model.
you may get better results adjusting the clip l
kohya suggested to me having clip l 1/10
so for your config unet 5e-05
clip l 5e-06
you may get better results adjusting the clip l
kohya suggested to me having clip l 1/10
so for your config unet 5e-05
clip l 5e-06
Fi tried every clip LR already :d
Mbut then again you're only training with ohwx man so it wont do much
Fi tried full joy captions too
F300 tokens
Mnot worth it, if you're training that many tokens you need to lower all your rates and train for a long time
Fbut may depends on dataset
Fi will focus on fine tuning 
Mtried on a100 got quite good results, idk if he fixed the extract lora thing yet, so you need to use higher weight when using it
�How about training clothes with text on them?
MThat could work. it'll work good for people who are holding up signs, text on clothings, product placements
Mbut for people training t5 is not worth it at all
�to train using t5 do I just use doc's config and select train t5 in the GUI?
Midk if he updated his config yet
Mbut you have to enable it in the advanced parameters
�Okay
Fi will update configs hopefully soon
�this right?
Mcorrect and enter this into your additonal parameters, cause i think the cpu offload checkpointing is not working.
--cpu_offlload_checkpointing
--cpu_offlload_checkpointing
�Can I DM you regarding one more thing?
Mif its feet pics no, other than that then yes 
PIs it necessary to train using resolution 1024x1024? For example, my image is a rectangle with a ratio of 768x1024, is it optimal for training or do I have to crop it to a square?
DFor the Kohya workflow, best results are achieved with the square ratio, which is the standard recommendation, as far as I know.
PIn the data train, there are many image sources with different resolutions, is that okay? Or should we just get 1 fixed resolution for all train data?
DYou can easily convert various aspect ratios to square size using outpainting. I use 'Expand' function of Adobe Firefly Generative Fill online - fastest and easiest.
Fi suggest you to first train 1024x1024
Fonce you have some base results
Fyou can try different aspect ratio multi res and compare
F1024x1024 most safe approach
Fit appears that onetrainer has dora for flux
Ftime to search it :d
Dhttps://github.com/comfyanonymous/ComfyUI/issues/4695 there have been some issues reported with loading DoRA files trained with OneTrainer in other tools like ComfyUI
GitHub
Feature Idea It looks like the Flux branch for OneTrainer is nearing completion - and the results are looking great! However, it seems as though ComfyUI cannot load the resulting .safetensors file....
Zif we have 12GB of vram, do we simply run the rank5 or 4 inside of kohya? do we still use flux dev considering that is a much bigger model? @Dr. Furkan Gözükara
Fi think he fixed already?
Fyou have to use rank 5 or below
Fuse dev model it auto cast
Fmake sure you have virtual RAM
Zoh ok gotcha, i have 12 GB of vram and 64 GB of system ram, i am watching your video right now and about to start training. I went outside and took some better pictures with the natural lighting outside
ZThank you
DHow much is quality loss in the Rank 4 fast version, compared to Rank 3 slow version?
Z@Dr. Furkan Gözükara I'm still getting OOM errors when trying to train a lora on 12GB, i'm all the way down to rank 8, i have 11.2 GB of Vram available and still getting it
TAnyone else having trouble with Forge? I'm using the same old thing in Flux and I get OutOfMemory error in Massedcompute with the 48gb
