© 2026 Hedgehog Software, LLC

Twitter GitHub Discord

More

Communities Docs About Terms Privacy

Serveless quants - Runpod

Runpod•13mo ago•

9 replies

Serveless quants

Hi, how do you specify a specific gguf quant file from a hf repo when configuring a vllm serveless endpoint? Only seems to let you specify the repo level.

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

21,202Members

Resources

Recent Announcements

Similar Threads

Was this page helpful?

Similar Threads

Streaming responses Serveless Endpoint

RRunpod / ⚡｜serverless

Llama 3.1 + Serveless

RRunpod / ⚡｜serverless

Timeout prior to serveless start

RRunpod / ⚡｜serverless

Serveless prices and technical questions

RRunpod / ⚡｜serverless