I'm interested in running an FP16 DeepSeek R1 and I am wondering if Serverless is the way to go or if a Pod would be better. I need this for 2-3 hours at a time and I would like a 'dedicated' access to this environment.
Which DeepSeek R1 model should I pick (GGUF?) and how should I configure the deployment tool in Serverless to get it to run on an H100?
Thanks in advance for any help.
Continue the conversation
Join the Discord to ask follow-up questions and connect with the community
R
Runpod
We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!