Fbut with cfg right?
Dregular flux dev same prompt
Dyes cfg 3.5
Dthe results are promising it behaves like a non distilled model
Dwith cfg at 1
Dhis explanation make sense, idk
DFlux-dev-de-distill
This is an experiment to de-distill guidance from flux.1-dev. We removed the original distilled guidance and make true classifier-free guidance reworks.
Model Details
Following Algorithm 1 in On Distillation of Guided Diffusion Models, we attempted to reverse the distillation process by re-matching guidance scale w. we introduce a student model x(zt) to match the output of the teacher at any time-step t ∈ [0, 1] and any guidance scale w ∈ [1, 4]. We initialize the student model with parameters from the teacher model except for the parameters related to w-embedding.
Since this model uses true CFG instead of distilled CFG, it is not compatible with diffusers pipeline. Please use inference script or manually add guidance in the iteration loop.
Train: 150K Unsplash images, 1024px square, 6k steps with global batch size 32, frozen teacher model, approx 12 hours due to limited compute.
This is an experiment to de-distill guidance from flux.1-dev. We removed the original distilled guidance and make true classifier-free guidance reworks.
Model Details
Following Algorithm 1 in On Distillation of Guided Diffusion Models, we attempted to reverse the distillation process by re-matching guidance scale w. we introduce a student model x(zt) to match the output of the teacher at any time-step t ∈ [0, 1] and any guidance scale w ∈ [1, 4]. We initialize the student model with parameters from the teacher model except for the parameters related to w-embedding.
Since this model uses true CFG instead of distilled CFG, it is not compatible with diffusers pipeline. Please use inference script or manually add guidance in the iteration loop.
Train: 150K Unsplash images, 1024px square, 6k steps with global batch size 32, frozen teacher model, approx 12 hours due to limited compute.
D@Dr. Furkan Gözükara I tried to train it on kohya but it not supported
F150k not bad
F12 hours really low though
Dmulti gpu maybe
F
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=40)
Dvisions of grandeur
Ftrying to find some new prompts
Dthe beard aint blue it doesnt count
DI'm traying it and also OpenFLUX.1, but the quality of Flux-dev-de-distill is much much better
Fdev is way better model
Fnot surprising
Disnt that an art movement
Dde stijl
F
Dlol
Dking of turkey
Dhehe
F
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=16)
Factually prompt is king of russia :d
Dlol
Dwell u did say i made u sound russian
F
DI hope that kohya make Flux-dev-de-distill compatible, let us know if you have any news
Dthat name is a mouthful
Fi think he will add that
Dkling image generator has improved
D
Fman these other faces gets so broken
F
Fkling making a lot of money
Dwell they are one of the leaders in the field
Ddont think the costs are too high, better than runway
Fyep
Fsomeone made a short movie and he said spent like 750$ on kling
F
XI need to check this out!
Fi saw on twitter dont remember now :d
Fit was making consisten images with lots of detailed prompts via midjourney
Fand using kling to animate
Fbut it wasnt that consisten imo
Ftraining flux dev would yield way better
XI’ll find it!
F
