Should I switch that lora config to bf16 instead of fp16? I'm getting OOM with 46GB
Should I switch that lora config to bf16 instead of fp16? I'm getting OOM with 46GB
Fye BF16 better
Fi am getting results will update configs :d
Fwhere do you train on massed compute?
Jok.. I dont really understand this, but adjust maximum image sized from 2048 to 1644 allow me to get it to fit, but now only using 31GB
JRight now I'm on Runpod. I'll use massed about 25% of the time, but I haven't found a way to download the models I trained in any reasonable timeframe
Fupload hugging face :d
Fwhich i am doing
Fi am training at 1024x1024 no bucketing
F128 rank works fine
Fbf16 training not fp8
JI'll give that a try... Is there anyway to install runpodctl on massed instance?
Fyep you can do
Flooks like running model in FP16 precision instead of FP8 makes minimalistic difference
JI got runpodctl setup on massed computer. I'll see how my download speeds are for models. That was really my only complaint
Jon massed computer, when we're on the desktop, open the terminal, git check out, activate venv, then load kohya_gui.py?
JWhen i did that I got the old version, but when I go back to the terminal it says im up to date on the date on the sd3.flux branch
Fyou need to do this
Ffind kohya ss in apps
JI got it
Fopen a terminal
Fgit pull
Fah ok 
Jthank you
Fyou are welcome
JI definitely think it's better to save off steps then epochs though, on that first runpod machine It's taking 1.5 hours to get through 5 epochs... needless to say, I changed that to steps on the massed instance
Fyou can do either way
Fsince i made repet 1 atm i save based on epochs
Iwov it's really amazing
Fuploading all models to do some real testing right now
Iwe are waiting your video tutorial when it's more advance @Dr. Furkan Gözükara
Fwhen i find best config
Fcurrently doing huge experiments to find it :d
Floras getting loaded for testing
F
Nis there a way to make x/y/z lora comparation in comfyui?
Fi dont know but in swarmui it is amazing
Fi am doing right now
ASwarmUI is very nice
Fyep
Fi am glad that kohya doesnt add stupid timestamp to checkpoints
Fi think fp16 training entirely broken
Vlooks amazing
F150 epoch even better let me post in a second
Fok i got some very good info from these results time to start 8 more test while sleeping
time to start 8 more test while sleeping
FSDXL prompts with flux testing no face inpainting amazing quality these are very early tests i am gonna optimize hyper params a lot
no face inpainting amazing quality these are very early tests i am gonna optimize hyper params a lot
F,
Fthese are early results no face inpainting
Nlooking great
Fyep
FPatreon
Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon