how many regularzation images did you use? did you caption your training images?
how many regularzation images did you use? did you caption your training images?
DI did not use any reg images. it's a code change of the trainer.
Mwhat code did you change?
SVery interesting since prior reg means using reg images
S
Dprior just means original model. I guess the name of "prior preservation" parameter is named that because originally people used images generated by the prior model
SSo what did you change in order to avoid bleeding? Would love to experiment with this aswell
Fcan you eloborate more? last time i tried it wasnt have any improvement :d
Fyes i was gonna ask this
Fbecause it was not working at all last time i tested. which trainer code change fixed?
DOT, but it's experimental.
Dyou might be able to reach a good result with reg images too, because it does something very similar. not exactly though
Cyou do realize that because of what was done to flux - it really is not trainable, right?
Dit's not a secret but this is not a development channel. I've shared on the OT discord
Fso you modified OT code? can you give full details please we can forward to kohya
Malready done
Fcan you link
Fyou forwarded ? i dont see :d
FGitHub
Replace SD3Tokenizer with the original CLIP-L/G/T5 tokenizers.
Extend the max token length to 256 for T5XXL.
Refactor caching for latents.
Refactor caching for Text Encoder outputs
Extract arch...
Extend the max token length to 256 for T5XXL.
Refactor caching for latents.
Refactor caching for Text Encoder outputs
Extract arch...
Mdm on twitter i sent him
Fhe doesnt read there :d
Mhe does
Fcan you reply to this thread
Mhe replies to me
Fhe does but rarely
Mbut sure
Fhe says he doesnt get dm notifications
Cfrom you maybe
Fgive me related code part i will forward
Fnope he replies back my every reply
Che obviously replies to max or max wouldn't have said he does
Fhe also replied my dms but he said he doesnt get notifications so he replies so late
Dhttps://discord.com/channels/1102003518203756564/1188183242747158630/1292072916976009348 the sample and some discussions below that, also a code snippet
Mhe's also japan time zone so keep that in mind
Cmaybe he jsut means he has notifications turned off then
Fok i forwarded code to the thread
Flets see now
FGitHub
Replace SD3Tokenizer with the original CLIP-L/G/T5 tokenizers.
Extend the max token length to 256 for T5XXL.
Refactor caching for latents.
Refactor caching for Text Encoder outputs
Extract arch...
Extend the max token length to 256 for T5XXL.
Refactor caching for latents.
Refactor caching for Text Encoder outputs
Extract arch...
M@crystalwizard you think the same issues will happen with sd3 8b
Fdid you notice any resemblance quality loss?_
Falso which file you added this i will test :d
Cyup. it's just larger. issues don't show up quite as much with a larger model - but they don't vanish just because it's larger
Fif you have cloned repo i can use it
Dnot sure, it's new. just found it interesting to share that it's even possible. I am not proposing that anyone implements it at this point.
Fcan you send me modified file
Fi would like to test 
Fin onetrainer when training sdxl reg images made such a big improvement
Fif we get it on flux would be so amazing
Cyou can not train flux. you get 'something' but you don't get an object that actually does what a lora is supposed to do
Cflux is broken, rigid, untrainable, and there's a reason that black forest never released training code. they KNOW it's not trainable
Cwhat you're geting is about the same as what you'd get if you took a photo and jsut drew over it. you're not updating the photo. you're just drawing over it