you need vram, not system ram
you need vram, not system ram
Fok i see his method seems very weak
Fno i doubt that it will ever work 
Fi need both :d
Cnot really. you shouldn't be using that CPU for anything graphics related. and if you have an AMD, you should go through CS1o's AMD guides and make sure it's set up right.
Ceverything should be on your GPU and in vram
Fit is pretty much nonsense
Factually i was expecting training single block would reduce bleeding but it didnt
Fi guess due to transformer arcihecture even single layer changes alls output
Fit depends on how many apps you open at the same time
Cthere's a 12 step program for that
X@Dr. Furkan Gözükara have you tried flux 1.1?
Fit is only api
Fi did a quick test it was better for some mystical stuff like
Fa dinasaur
Fbacking up test data
Ftrainings started
F
DI'm not comfortable with you spending 8 GPUs on my 5 lines of code from thing morning 
Dbut it's up to you
Fye it is fine
Fif works it will be big
Mare you using your 5200 reg images?
Mi dont think dx is using any reg images
Fyep
Fi am testing both cases
Dhttps://huggingface.co/nyanko7/flux-dev-de-distill I'm testing this model on forge and I'm having interesting results, distilled cfg does nothing, is completely disabled
Dregular Flux-dev random prompt
Dde-distilled model same prompt same seed
Don the de-distilled model if you prompt random you get random noise
Don the normal flux-dev you get a coherent output because is allways guided
Fye probably these are just failed experiments
Fnot true destilation
DIf your prompt is not random you get very similar outputs on both models
Dde-distilled model
Fbut with cfg right?
Dregular flux dev same prompt
Dyes cfg 3.5
Dthe results are promising it behaves like a non distilled model
Dwith cfg at 1
Dhis explanation make sense, idk
DFlux-dev-de-distill
This is an experiment to de-distill guidance from flux.1-dev. We removed the original distilled guidance and make true classifier-free guidance reworks.
Model Details
Following Algorithm 1 in On Distillation of Guided Diffusion Models, we attempted to reverse the distillation process by re-matching guidance scale w. we introduce a student model x(zt) to match the output of the teacher at any time-step t ∈ [0, 1] and any guidance scale w ∈ [1, 4]. We initialize the student model with parameters from the teacher model except for the parameters related to w-embedding.
Since this model uses true CFG instead of distilled CFG, it is not compatible with diffusers pipeline. Please use inference script or manually add guidance in the iteration loop.
Train: 150K Unsplash images, 1024px square, 6k steps with global batch size 32, frozen teacher model, approx 12 hours due to limited compute.
This is an experiment to de-distill guidance from flux.1-dev. We removed the original distilled guidance and make true classifier-free guidance reworks.
Model Details
Following Algorithm 1 in On Distillation of Guided Diffusion Models, we attempted to reverse the distillation process by re-matching guidance scale w. we introduce a student model x(zt) to match the output of the teacher at any time-step t ∈ [0, 1] and any guidance scale w ∈ [1, 4]. We initialize the student model with parameters from the teacher model except for the parameters related to w-embedding.
Since this model uses true CFG instead of distilled CFG, it is not compatible with diffusers pipeline. Please use inference script or manually add guidance in the iteration loop.
Train: 150K Unsplash images, 1024px square, 6k steps with global batch size 32, frozen teacher model, approx 12 hours due to limited compute.
D@Dr. Furkan Gözükara I tried to train it on kohya but it not supported
F150k not bad
F12 hours really low though
Dmulti gpu maybe
F
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=40)
Dvisions of grandeur
Ftrying to find some new prompts
Dthe beard aint blue it doesnt count