<@407561236339752981> you think the same issues will happen with sd3 8b
@crystalwizard you think the same issues will happen with sd3 8b
Fdid you notice any resemblance quality loss?_
Falso which file you added this i will test :d
Cyup. it's just larger. issues don't show up quite as much with a larger model - but they don't vanish just because it's larger
Fif you have cloned repo i can use it
Dnot sure, it's new. just found it interesting to share that it's even possible. I am not proposing that anyone implements it at this point.
Fcan you send me modified file
Fi would like to test 
Fin onetrainer when training sdxl reg images made such a big improvement
Fif we get it on flux would be so amazing
Cyou can not train flux. you get 'something' but you don't get an object that actually does what a lora is supposed to do
Cflux is broken, rigid, untrainable, and there's a reason that black forest never released training code. they KNOW it's not trainable
Cwhat you're geting is about the same as what you'd get if you took a photo and jsut drew over it. you're not updating the photo. you're just drawing over it
Dyou couldn't be more wrong. SD15/XL is more like that. Flux is even able to learn from low res and integrate it into existing model knowledge and generate good high res
Ci'm not wrong. i know what the code is, i know what the neural network is. i know what they did to it to mask the issues it has, and i know it's not trainable
F
Cand you running around creating loras are not creating loras that do anything. you're getting something that gives you image results - but you are NOT getting loras that actually work like loras are supposed to
Fyour idea looks solid from claude 3.5 :d
Fcan you send me modified file please
Dok, but you have to be willing to experiment, I don't know any good settings yet
Fno worries
Fjust send me file that you got those reuslts
Fyou shown
Fno need config
Fonly modified file
Fman my massed compute 1tb not enough
Fsome new discovery perhaps
D@Furkan Gözükara SECourses just in case you are thinking about training TE - because that would make sense with regularization, to separate the concepts further - the code isn't ready for that. it only unhooks the lora from the transformer.
Fwell it doesnt matter. i tested all cases all bleeds huge
Fand it is issue of the dit architecture
Fnot TE or clip training
Fso i will train only unet to test
Dwhat I mean is: if this reg method works well, it might even be working better with TE training. but not yet, so turn it off either way
Fkk
M@Furkan Gözükara SECourses seems prior preservation in kohya is broken
MGitHub
Replace SD3Tokenizer with the original CLIP-L/G/T5 tokenizers.
Extend the max token length to 256 for T5XXL.
Refactor caching for latents.
Refactor caching for Text Encoder outputs
Extract arch...
Extend the max token length to 256 for T5XXL.
Refactor caching for latents.
Refactor caching for Text Encoder outputs
Extract arch...
Mbghira responded
Fi think he didnt see my reply becaue i added. but good
becaue i added. but goodFif this get fixed nice
Dah, and batch size 1!
Fkk
Fbatch size 1 is best
Fanyway for low number of images
Dwith and without
[removed]
[removed]
Chilary doesn't look like hilary
Dthat's the base model, I didn't train Clinton. The point is that Clinton stays as Clinton as she was
Don the left
Cshe didn't though. neither woman, in both images, look like they do on the right