for lora it's easy to use a model in low precision, because the values you are changing are still hi
for lora it's easy to use a model in low precision, because the values you are changing are still higher precision. but for full finetune he uses less than 16bit?
![Dazzastrous [4090]](https://cdn.discordapp.com/avatars/871389071686135838/c4a5f4d81d9eeb18d856295d692e2c07.webp?size=40)