So the version a Lora is trained on doesn’t matter?
So the version a Lora is trained on doesn’t matter?
Fyou need to use fp16 as base model during training
Fyou can use lora on fp16 or fp8 base model after trained
MHey @Dr. Furkan Gözükara can i follow your instructions to runpod and use aws instead ?
Mcreate a notebook there and just follow the rest of the instructions ?
Fif your aws has python 3.10 cuda and c++ yes
Fand nvidia drivers
Mthanks
Mit just started crashing
Mmaybe its the port
Mill try setting the port
M@Dr. Furkan Gözükara bmaltis is updated now
Mtext encoder training works
Mvram is using much less now
Fnice let me check
VSuccesfully trained a model with clip l text encoder, results are better now and so are prompting
Fvram usage shouldnt be changed
Fyou used same LR?
Vyou can use the fp8 t5
Vuse half learning rate for text encoder
Vor else you will cook your model
Fthis shouldnt make difference
Fi think it auto cast
Fand we train only clip L
Vim getting less vram than before so im not sure whats changed
Flet me check vram too
Fi will still record video waiting clip l training results would delay further 
Vit may be this
VIn FLUX.1 LoRA training, when
--fp8_base is specified, the FLUX.1 model file with fp8 (float8_e4m3fn type) can be loaded directly. Also, in flux_minimal_inference.py, it is possible to load it by specifying fp8 (float8_e4m3fn) in --flux_dtype.Fye it just let you load fp8
Ffp16 was still casted to fp8
Vi used this here --fp8_base_unet
Vand clip is trained at bf16
Vim using sd scripts directly
Fkk
Fon windows or linux?
Vlinux
Vdont bother with windows
Fok. if i can lower LR + clip L
Fit can lesser overfit
V7e-05 for unet
Vjust use higher weight decay for overfitting
Fi already use lower than that
Vyou're using adfactor though
Vadamw8bit i get better results
Fye i use most optimized VRAM to cover more GPUs
Fi got a new finding
Fshift value 1
Vi thought it was common to use shift value 1
Dstarting a training with TE
