Is Hunyuan current state of the art img2vid model?
Is Hunyuan current state of the art img2vid model?
Tin both open-source and SaaS worlds?
GAnyone know the easiest way to finetune your own VLLM for image captioning? I've already got my own dataset, but there doesn't seem to be a straightforward way to actually carry out the finetuning process itself... I know there's already some good captioners out there, but I want to finetune my own
T@Dr. Furkan Gözükara I've noticed that SwarmUI downloads t5xxl_enconly.safetensors file every time you try to use flux even if you set t5xxl model location manually.
Furthermore there is a mention in debug logs that Comfy back-end is typecasting fp8_e4m3fn model into bf16 every time flux type checkpoint loads from scratch, and that typecasting takes significant CPU time.
I looked up in swarmUi code and it seems that it's downloading model from this repo: https://huggingface.co/mcmonkey/google_t5-v1_1-xxl_encoderonly/ That mdel is 4.9 gb that is consistent with VRAM usage of swarmUI when running flux inference.
My concern is that all this time we used the inferior fp8 casted t5xxl model for generations, wich according to many reports decreases output quality much more than casting to fp8 flux itself
Furthermore there is a mention in debug logs that Comfy back-end is typecasting fp8_e4m3fn model into bf16 every time flux type checkpoint loads from scratch, and that typecasting takes significant CPU time.
I looked up in swarmUi code and it seems that it's downloading model from this repo: https://huggingface.co/mcmonkey/google_t5-v1_1-xxl_encoderonly/ That mdel is 4.9 gb that is consistent with VRAM usage of swarmUI when running flux inference.
My concern is that all this time we used the inferior fp8 casted t5xxl model for generations, wich according to many reports decreases output quality much more than casting to fp8 flux itself
TIt also checks model file hash so just replacing file with the same name would't work
TI noticed it while researching why swapping fp16 flux models takes so much time. If my findings are correct it seems half the time is wasted on useless typecasting of t5xxl from fp8 to bf16
TNotice your VRAM usage on 40+ GB vram cards while running flux fp16 inference. It should use at least 33.6 GB vram to fit both flux and t5xxl + clip, but it seems it is using around 30GB vram which is more consistent with downcasted t5xxl theory
Jinteresting! so what should one do?
T- Validate this hyposethis
- Report to SwarmUI repo issues
- Maybe look up in the code and suggest fix as MR to the repo
TMaybe it was just my who idiotically named the file t5xxl_enconly hoping to avoid downloading it and resulting in in being overwritten every time
Tyep, seems like a false alarm, this only affected me as the file was overwritten after I downloaded it by swamui
Fit is text to vid
Fgood question i never saw anyone did
Fyes he does that
Fi reported but he didnt change so far
Fyou can download fp16
Frename
Fit wont download
Vhttps://youtu.be/FvpWy1x5etM?t=4719
here at this part its said something that you cannot update kohya with the updater but should use installers instead? is this still the case? thx!
here at this part its said something that you cannot update kohya with the updater but should use installers instead? is this still the case? thx!
YouTubeSECourses
If you want to train FLUX with maximum possible quality, this is the tutorial looking for. In this comprehensive tutorial, you will learn how to install Kohya GUI and use it to fully Fine-Tune / DreamBooth FLUX model. After that how to use SwarmUI to compare generated checkpoints / models and find the very best one to generate most amazing image...
GReckon it's something you'd be interested in taking a swing at? A nice wrapper would make things so much easier, but that's beyond me
RYouTubeRuairi Robinson
AI is coming for your job. Which side are you on?
Music: Pete Seeger.
Directed by Ruairi Robinson
Made using Veo 2
Music: Pete Seeger.
Directed by Ruairi Robinson
Made using Veo 2
Rmade another thing with VEO 2
V@Furkan Gözükara SECourses in flux finetuning I saw you used 100 epochs for 256 images. so if you have for example 350 images, then what epochs should you choose? or should you reduce images to amount of 256?
Vvery cool stuff. how long does it take you to create such a video?
Fyes still not in main branch
Fprobably max 50
For 75
Vahhh ok I think I understand. more images higher quality and less epochs needed
V@Furkan Gözükara SECourses batch size 1 is better quality? how much percent do you guess?
but its worse in stylized? what is actually stylized?
I looked at the comparison sets but dont see any differences
but its worse in stylized? what is actually stylized?
I looked at the comparison sets but dont see any differences
V@Furkan Gözükara SECourses with batch size 1 and fine tuning can both GPUs be used? seems like it is not used
Rthis was 2 days
V2 days? what is all involved in such a process and steps is most time spent?
what do you think about this tool or gen AI in general? (because I saw in your channel you do movies)
what do you think about this tool or gen AI in general? (because I saw in your channel you do movies)
A@Furkan Gözükara SECourses SOS 
I need suggestions for good lip sync
I need suggestions for good lip syncV@Furkan Gözükara SECourses
in video you say epoch 100 is good for 256 images... but after you mention that epoch 50 is best?
what I dont understand yet... so while training you can already move the generated checkpoints to some different place because these are kind of finished states of the trained model and will only be needed for training in case the training fails. which would mean you could then use the checkpoint as new starting point? is this correct?
in video you say epoch 100 is good for 256 images... but after you mention that epoch 50 is best?
what I dont understand yet... so while training you can already move the generated checkpoints to some different place because these are kind of finished states of the trained model and will only be needed for training in case the training fails. which would mean you could then use the checkpoint as new starting point? is this correct?
1I've read some comments about using a clip interogator to filter out similar images from a dataset. Has anyone used this technique or know of a tool that could do this?
VIs anyone aware of a platform that offers the ability to train a flux lora and then use something like adetailer/udetailer to fix the face on inference, like out of the box or with minimal setup?
Not looking to set up my own thing from scratch on something like runpod, hoping for something more plug and play like Replicate
Not looking to set up my own thing from scratch on something like runpod, hoping for something more plug and play like Replicate
A
MThank you for the Flux workflow, it looks promising. But I have a question I keep getting a "mat1 mat2 mismatch" error for Clip_G looks like I'm using the wrong version, can you share the link to the clip file you're using?
Also, it would be nice if you can share the generation data for one of your images, just for me to check if everything is working correctly.
Also, it would be nice if you can share the generation data for one of your images, just for me to check if everything is working correctly.
MHello
MWhenever i train kohyass lora on a flux model, i get error "returned non-zero exit status 1."
Mother models it works perfectly fine
AI'm getting two additional options for Kohya that don't appear on the exapmle image on the guide on Patreon - "Conv Dimension" and "Minimum Difference". Should I leave them with default values?
Fyes batch size 1 is best
Fi dont know good one atm
Fthe logic is like this
Fwe do up to 200 epochs
Fand we compare and find best one
Fbut if you have too many images
Fit is becoming redundant
Fwe have image similarity app you can try
