if works it will be big
if works it will be big
Mare you using your 5200 reg images?
Mi dont think dx is using any reg images
Fyep
Fi am testing both cases
Dhttps://huggingface.co/nyanko7/flux-dev-de-distill I'm testing this model on forge and I'm having interesting results, distilled cfg does nothing, is completely disabled
Dregular Flux-dev random prompt
Dde-distilled model same prompt same seed
Don the de-distilled model if you prompt random you get random noise
Don the normal flux-dev you get a coherent output because is allways guided
Fye probably these are just failed experiments
Fnot true destilation
DIf your prompt is not random you get very similar outputs on both models
Dde-distilled model
Fbut with cfg right?
Dregular flux dev same prompt
Dyes cfg 3.5
Dthe results are promising it behaves like a non distilled model
Dwith cfg at 1
Dhis explanation make sense, idk
DFlux-dev-de-distill
This is an experiment to de-distill guidance from flux.1-dev. We removed the original distilled guidance and make true classifier-free guidance reworks.
Model Details
Following Algorithm 1 in On Distillation of Guided Diffusion Models, we attempted to reverse the distillation process by re-matching guidance scale w. we introduce a student model x(zt) to match the output of the teacher at any time-step t ∈ [0, 1] and any guidance scale w ∈ [1, 4]. We initialize the student model with parameters from the teacher model except for the parameters related to w-embedding.
Since this model uses true CFG instead of distilled CFG, it is not compatible with diffusers pipeline. Please use inference script or manually add guidance in the iteration loop.
Train: 150K Unsplash images, 1024px square, 6k steps with global batch size 32, frozen teacher model, approx 12 hours due to limited compute.
This is an experiment to de-distill guidance from flux.1-dev. We removed the original distilled guidance and make true classifier-free guidance reworks.
Model Details
Following Algorithm 1 in On Distillation of Guided Diffusion Models, we attempted to reverse the distillation process by re-matching guidance scale w. we introduce a student model x(zt) to match the output of the teacher at any time-step t ∈ [0, 1] and any guidance scale w ∈ [1, 4]. We initialize the student model with parameters from the teacher model except for the parameters related to w-embedding.
Since this model uses true CFG instead of distilled CFG, it is not compatible with diffusers pipeline. Please use inference script or manually add guidance in the iteration loop.
Train: 150K Unsplash images, 1024px square, 6k steps with global batch size 32, frozen teacher model, approx 12 hours due to limited compute.
D@Dr. Furkan Gözükara I tried to train it on kohya but it not supported
F150k not bad
F12 hours really low though
Dmulti gpu maybe
F
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=40)
Dvisions of grandeur
Ftrying to find some new prompts
Dthe beard aint blue it doesnt count
DI'm traying it and also OpenFLUX.1, but the quality of Flux-dev-de-distill is much much better
Fdev is way better model
Fnot surprising
Disnt that an art movement
Dde stijl
F
Dlol
Dking of turkey
Dhehe
F
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=16)
Factually prompt is king of russia :d
Dlol
Dwell u did say i made u sound russian
F
DI hope that kohya make Flux-dev-de-distill compatible, let us know if you have any news
Dthat name is a mouthful
Fi think he will add that
Dkling image generator has improved
D
Fman these other faces gets so broken
F
Fkling making a lot of money