i hope we will also solve it eventually
i hope we will also solve it eventually
Fit is poor
Cnot if you keep trying to train an untrainable model, you won't
Fwell people trying distillation i cant say for sure will work
Fbut we will see
DI'm training it right now, many concepts at the same time, lets see if it bleeds
Dnyanko7/flux-dev-de-distill
DI'm trying right now with flux-dev-de-distill If it doesn't wok I will inform you and we will know if you are right
Dthe only problem is the sample images on kohya It needs real cfg and not distilled cfg, may be sampling is not implemented
Cthat could be a problem.
Dyes if so I will have to wait until the training is finished
D@Dr. Furkan Gözükara I think that flux-dev-de-distill is slower training 7s/it is too slow for 3090ti?
Drank 128
DApply T5 Attention Mask off
DTrain T5-XXL off
Fnot very slow but 6 second would be better
Fexpected
Dok thanks, to speed it up for the first test Im training at 512x512 2.77s/it
Dproblem with samples on kohya
Dit puts the real cfg on 1
Dor 0 adk
Dis still traying to use distilled cfg and it doesn't do anything
Chave you tried using onetrainer?
Dnot yet
DI always use kohya
Dno problem, I will train the model without samples and then I can try the lora on forge
OHas anyone tried merging a bunch of fine tunes/Loras together? Does this work to have multiple concepts in one checkpoint or is the quality impacted?
DThe problem with the samples is reported to @kohya_tech
Dquick question, how is everyone captioning for flux training? single trigger word, joy caption long with trigger word, or joy caption without trigger word? or something else
CFor training a person just the instance/class and no captions seem to get the job done very well so probably not worth over thinking. For style most people will parrot that it should be captioned natural language but some of the most popular flux lora creators on civit will use only basic trigger or tags and yield great results and some have tested with no captions and still had good results. Broadly it seemed the best to worst options were VLM/LLM/Joycaption, tagging, trigger only, no caption but with Flux I struggled to find any conclusive info but could find conflicting 'evidence' of various methods so it appears the annoying answer is its probably case by case depending on a review of results for your specific purpose. I would love to know the answer myself too to save that effort of trial and error so if your google skills are better than mine please share if you find something! 
Lcan i train it with flux 1.1? someone has the link for flux 1.1?
Lor i need to do it after extraction of my lora? im little confused
Cyou cannot train flux1.1, the kohya configs on patreon include a script that will download the flux dev model and required files, then you can choose to train a lora or do a fine tune and extract a lora or use the fine tune as is in place of flux
D@Dr. Furkan Gözükara im trying the new flux training scripts. Using the json for 12gb (i have a 3060) and im getting a Cuda oom error. My virtual memory is set to 65gb per the patreon description. Any idea what might be wrong?
D
Dthis is the error
Cdid you search google for that error?
Di searched the cuda out of memory
Csearch the "might be complied" message
Cand make sure you ahve enough paging space
Di have pagefile set at 65gb
Di have that space
Dim not seeing the complied error
Cright under the first out of memory error, look for Cuda kernal errors might be asynchronously reported at some other API call
Cit's a pretty common cuda error.
Dit seems to be --disable-cuda-malloc related
Dfrom what im reading,
Dif im getting it right, but im unsure where to insert that
Dlooks like it goes into the .bat
Lso I need to extract a lora that i like? and then to do a dreambooth training on it?
Dsay you want to make the best quality lora, you would do a fine tune, then there is a function in kohya that looks at the base file(flux) and your fine tune safetensor and it extracts the difference between them. That becomes your final lora