Search
Star
Feedback
Setup for Free
© 2026 Hedgehog Software, LLC
Twitter
GitHub
Discord
System
Light
Dark
More
Communities
Docs
About
Terms
Privacy
GGUF vllm - Runpod
R
Runpod
•
2y ago
•
15 replies
artbred
GGUF vllm
It seems that the newest version of vllm
's supports gguf models
, have anyone figured out how to make this work in runpod serverless
? Seems like need to set some custom ENV vars
, or maybe anyone knows a way to convert gguf back to safetensors
?
Runpod
Join
We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!
21,202
Members
View on Discord
Resources
ModelContextProtocol
ModelContextProtocol
MCP Server
Recent Announcements
Similar Threads
Was this page helpful?
Yes
No
Similar Threads
GGUF in serverless vLLM
R
Runpod / ⚡|serverless
2y ago
vllm
R
Runpod / ⚡|serverless
2y ago
vllm +openwebui
R
Runpod / ⚡|serverless
15mo ago
Vllm docker
R
Runpod / ⚡|serverless
2y ago