© 2026 Hedgehog Software, LLC

Twitter GitHub Discord

More

Communities Docs About Terms Privacy

Selecting a hf quant - Runpod

Runpod•9mo ago•

3 replies

Selecting a hf quant

Hi, using vllm serveless. Is there a way to specify a specific quant to use for a hf gguf model directory url?

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

21,202Members

Resources

Recent Announcements

Similar Threads

Was this page helpful?

Similar Threads

RRunpod / ⚡｜serverless

HF_TOKEN question

RRunpod / ⚡｜serverless

Serveless quants

RRunpod / ⚡｜serverless

how to pass hf_overrides to worker-vllm ?

RRunpod / ⚡｜serverless