Search Setup for Free

Runpod•8mo ago

Selecting a hf quant

Hi, using vllm serveless. Is there a way to specify a specific quant to use for a hf gguf model directory url?

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

20,815Members

View on Discord

Similar Threads

Was this page helpful?

Product

Pricing Docs Communities

Resources

About Blog Changelog Contributors

Legal

Terms Privacy Cookies EULA

Twitter GitHub Discord

© 2026 Hedgehog Software, LLC. All rights reserved.

© 2026 Hedgehog Software, LLC

Twitter GitHub Discord

More

Communities Docs About Terms Privacy

Similar Threads

RRunpod / ⚡｜serverless

HF_TOKEN question

RRunpod / ⚡｜serverless

download from HF is slow like 5mbit

RRunpod / ⛅｜pods

Import SD3.5 from HF to Runpod

RRunpod / ⛅｜pods