Search
Get Started
R
Runpod
•
5mo ago
swousy
How do I set quantization to fp8 in the serverless settings?
If I select
'bitsandbytes
' will that automatically change quant level to fp8
?
Runpod
Join
We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!
20,776
Members
View on Discord
Similar Threads
Was this page helpful?
Yes
No
Similar Threads
Quantization method
R
Runpod / ⚡｜serverless
2y ago
How do I bust the cache on Serverless builds
R
Runpod / ⚡｜serverless
6mo ago
How do i retry worker task in runpod serverless?
R
Runpod / ⚡｜serverless
2y ago
VLLM WORKER ERRROR
R
Runpod / ⚡｜serverless
2y ago
