Hello everyone. I am Dr. Furkan Gözükara. PhD Computer Engineer. SECourses is a dedicated YouTube channel for the following topics : Tech, AI, News, Science, Robotics, Singularity, ComfyUI, SwarmUI, ML, Artificial Intelligence, Humanoid Robots, Wan 2.2, FLUX, Krea, Qwen Image, VLMs, Stable Diffusion
quick question, how is everyone captioning for flux training? single trigger word, joy caption long with trigger word, or joy caption without trigger word? or something else
For training a person just the instance/class and no captions seem to get the job done very well so probably not worth over thinking. For style most people will parrot that it should be captioned natural language but some of the most popular flux lora creators on civit will use only basic trigger or tags and yield great results and some have tested with no captions and still had good results. Broadly it seemed the best to worst options were VLM/LLM/Joycaption, tagging, trigger only, no caption but with Flux I struggled to find any conclusive info but could find conflicting 'evidence' of various methods so it appears the annoying answer is its probably case by case depending on a review of results for your specific purpose. I would love to know the answer myself too to save that effort of trial and error so if your google skills are better than mine please share if you find something!
you cannot train flux1.1, the kohya configs on patreon include a script that will download the flux dev model and required files, then you can choose to train a lora or do a fine tune and extract a lora or use the fine tune as is in place of flux
@Dr. Furkan Gözükara im trying the new flux training scripts. Using the json for 12gb (i have a 3060) and im getting a Cuda oom error. My virtual memory is set to 65gb per the patreon description. Any idea what might be wrong?
say you want to make the best quality lora, you would do a fine tune, then there is a function in kohya that looks at the base file(flux) and your fine tune safetensor and it extracts the difference between them. That becomes your final lora