Hello everyone. I am Dr. Furkan Gözükara. PhD Computer Engineer. SECourses is a dedicated YouTube channel for the following topics : Tech, AI, News, Science, Robotics, Singularity, ComfyUI, SwarmUI, ML, Artificial Intelligence, Humanoid Robots, Wan 2.2, FLUX, Krea, Qwen Image, VLMs, Stable Diffusion
After my failure I'm not so sure about "Apply T5 Attention Mask, and Train T5-XXL enabled" you are training the T5, but in inference you load a T5 model that doesn't know your token. forge o SwarmUI load the regular T5 model that was not trained. so I'm not so sure
@Dr. Furkan Gözükara may be Apply T5 Attention Mask can help, but I don't understand how can Train T5-XXL helps, because on inference you load a T5 model that was not trained . forge or SwarmUI load the regular T5 model that was not trained
Is the thing I don't understand, so really IDK now I'm training again with both options disabled, My question can apply to everyone, but in my case is different because I'm training flux-dev-de-distill so in my case is broking something. I'm having artifacts on the images with this on and also is not learning the concepts correctly
@Dr. Furkan Gözükara Update on https://huggingface.co/nyanko7/flux-dev-de-distill I started new training - "T5 Attention Mask and T5-XXL both disabled" same lr 30 epochs now. I tested the lora checkpoint and all problems are fixed, is going great using regular flux-dev-fp8 for inference, I'm training tree people same class, same prompt changing just the name renders the correct subject, is undertrained but goes very well so far
So here is what I suggest, as I watched some parts of all your tutorials, but not all (and sometimes the bookmarks are wrong). Add an image in the config file showing the repeat section and saying it needs to be 1.