i just told this to swarmui
i just told this to swarmui
Cnever used swarm ui, didnt realize it can take natural language configs
Fno i mean the developer :d
Fout of 7 new testing 3 of them are noise
Fjust got a better config out of new tests amazing 100 epoch 1500 steps
Freg images killed likeliness hugely
Ni was noticing that as well - bottom row of Dr. Furkan's grid, 2nd from right - best, 3000 steps, with reg images - bears the worst resemblance
Fyep
Fi am gonna update patreon with best new configs
F
Jthe problem happens not only on SwarmUI even on Forge for me. Other reporting it too, like here: https://github.com/mcmonkeyprojects/SwarmUI/issues/192
GitHub
Expected Behavior Subsequent 16-bit images generated with a LORA in Flux dev should not degrade into fuzz. Actual Behavior When using any LORA, the first image generated after switching to Default ...
Freg images improves environment but likeliness is gone in some cases - 2225 step
Vdrop your prior weight loss to 0.10
Vand more steps
Fnice idea should test
Vreg images not reccomended for flux, but 0.10 and lower prior loss and higher lr ideal
Fjust testing atm. so far got great results and getting even better with finding better params
Vsame as well, been getting amazing results with finetune/dreambooth as well
Vlora is sort of crap because of the different alphas and conv needed
Ffully distant shot 
Fi like fine tuning too. but first started lora and sadly it doesnt work with 12 gb config
Fit uses 18 gb
VMafia boss 
Fi tell this to kohya he says working . i have shown full config toml everything. tested on windows and ubuntu
Fuses 18 gb with half model split training
Vwhen training on the fp8 model probably less vram
Vwhen available
Fit is fp8 plus half model
Fstill uses 18 gb
Vare you training on the full 23gb model?
FFi tested both fp8 model and fp16 model but it doesnt matter the precision i set is fp8 mode
Fthe vram usage is very erratic as shown above
Vhopefully one trainer starts working on it
Vhave your tried the bitsandbytes flux model?
Fwhat you mean by that
Faccelerator has fp8 mode but it doesnt work with kohya :d
Fi dont think they are trainable but didnt test
Vyou should upgrade to torch 2.4.0 and vision 19.0
Vit uses the new cudnn 9.01
Vi think faster
Vtorch==2.4.0+cu124 --index-url https://download.pytorch.org/whl/cu124
torchvision==0.19.0+cu124 --index-url https://download.pytorch.org/whl/cu124
torchvision==0.19.0+cu124 --index-url https://download.pytorch.org/whl/cu124
Vhavent seen training on it yet, so not sure
Flet me test these to see vram usage
Vapprently 50% to 100% speed boost
Vbut i think only for fp16 and bf16
Fnice
Fi am gonna test later
Fi dont want to take risk of breaking quality :d
