© 2026 Hedgehog Software, LLC

Twitter GitHub Discord

More

Communities Docs About Terms Privacy

Trying to load a huge model into serverless - Runpod

Runpod•2y ago•

15 replies

Trying to load a huge model into serverless

https://huggingface.co/cognitivecomputations/dolphin-2.9.2-qwen2-72b

Anyone have any idea how to do this in vLLM?
I've deployed using two 80GB gpus and have had no luck

cognitivecomputations/dolphin-2.9.2-qwen2-72b · Hugging Face

cognitivecomputations/dolphin-2.9.2-qwen2-72b · Hugging Face

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

21,202Members

Resources

Recent Announcements

Similar Threads

Was this page helpful?

Similar Threads

Deploying MIGAN model to Serverless.

RRunpod / ⚡｜serverless

Serverless Load-balancing

RRunpod / ⚡｜serverless

Huge sudden delay times in serverless

RRunpod / ⚡｜serverless

Experiencing huge execution time on Serverless

RRunpod / ⚡｜serverless