Nice I'll have to check it out. I have a 4090
Nice I'll have to check it out. I have a 4090
Fyep
Fgreat
Fuse torch 2.5 to have best speed on windows
Si have like 60 more models to merge but i want to check it out this weekend for sure
S
Sfine tuning and adding flexibility to this 1.5 model
Swhile keeping the most accuracy
Si wish i knew the science behind how merging works but i just mix them up and hope for the best
SThis man Sciences
Fthere is no science 
it averages neural network weights and you expect to work
it averages neural network weights and you expect to workShahah well i just mean in a basic sense, like if i do 50% + 50% of one model and then merge that with another at 50% does it equal out to be 25% +25% +50% or what lol
Sor am i actually losing any data or does other data just cover it up to where i can bring other data forward by adjusting weights of tokens and what not
Dhm, I seem to be using less VRAM than supposed to
odd
odd
Fmaybe torch 2.5 diff
Fare you using torch 2.5? vrams tested on torch 2.4
AThanks to your guides and configs I fine tuned an SDXL model with pics of my dog and of me using OneTrainer (will post some results later). The thing I'm struggling with is extracting a Lora from the model. I was able to do this with kohya_ss but the Lora comes out at almost 2Gb. I was wondering am I using wrong settings in kohya_ss or should I extract the Lora and then run the reduce Lora option in kohya_ss?
Fwhy you need to reduce its size?
Fyou can reduce by
Fset precision to bf16 or fp16 compare each one quality
Fset rank 128 or lower if you need lower size
Fmay reduce quality
Fi use kohya gui extractor
AYeah, using the Kohya gui. Main reason I'd like to reduce the size is I wanted to share it + my hard drive was getting full (I really should buy a bigger drive). I played a bit with the resize option in the gui. Dropped it to a rank of 64 and the final size was about 380MB without noticeable quality loss. I was mainly curious if there was a recommend way to get smaller Loras.
ABut you are right. I should just follow your example of just testing the quality of the Lora at different ranks and precision till I find a good mix.
Dnot sure, I just installed from the files in the latest kohya pack
DI did install the torch 2.5 one too
NAppreciate you sharing your results, may test extracting mine too
SDr Furkan, i hope evertyting is ok, im traying to make my firt lora model in flux, i have a 4070 super ti 16 gigas vram this speed is good? if i install the torcho 2.5 could increase speed? im use the rank 5 setting
Fspeed is good . after this try torch 2.5 and see if improves. i think may improve
Sthank you Dr Furkan Im using 44 images maybe i will try with fewer the next time
PCan I ask if a 1080 8gb card can train lora? If not, is the 3060 capable?
Fit cant since 1080 doesnt have bf16
Fbut 3060 perfectly can
PThank you for letting me know. With 3060 how long does it take to train on average?
Fit would take like 10 hours to get a perfect quality
Fbecause we have to use huge optimization
Fhowever there may come a new technique
Ftraining a single layer
Fwaiting kohya on that
DSeems like vast majority of character Loras work even without including their supposed trigger words. Interesting
Fflux has internal encoding
Fso every image fully captioned
Dnot sure what you two mean. I have used a character lora with flux recently. trigger word (which was the name of the person) was necessary. using the lora without it already did something, but not everything
Fif you dont use trigger word and if it is not overfit you wont get exactly that person
Fbecuase you cant know exact internal encoding / caption 
SDr Furkan i hope evrything is ok, wht do you think could be the best, cost/benefit amount fo dataset image for training?
Fi plan to test with 100 images
Fcurrently works as low as 15 very good
Dhm, does Lora merging work for flux?