Search
Star
Feedback
Setup for Free
© 2026 Hedgehog Software, LLC
Twitter
GitHub
Discord
System
Light
Dark
More
Communities
Docs
About
Terms
Privacy
how to run a quantized model on server less? I'd like to run the 4/8 bit version of this model: - Runpod
R
Runpod
•
13mo ago
•
4 replies
codyman4488
how to run a quantized model on server less? I'd like to run the 4/8 bit version of this model:
https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-32B-GGUF
unsloth/DeepSeek-R1-Distill-Qwen-32B-GGUF · Hugging Face
Runpod
Join
We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!
21,202
Members
View on Discord
Resources
ModelContextProtocol
ModelContextProtocol
MCP Server
Recent Announcements
Similar Threads
Was this page helpful?
Yes
No
Similar Threads
How to load model into memory before the first run of a pod?
R
Runpod / ⚡|serverless
2y ago
Run LLM Model on Runpod Serverless
R
Runpod / ⚡|serverless
3y ago
Général advices on the pricing and the use of server less
R
Runpod / ⚡|serverless
3y ago
server less capability check
R
Runpod / ⚡|serverless
3y ago