R
RunPod•3mo ago
asherisaac

serverless multi-gpu

Hi, in the serverless endpoint console I'm seeing that you can't have a serverless multi-gpu endpoint except for 2x A40? Is this correct? So essentially the serverless product is only for smaller models?
Solution:
we are slowly allowing more as we get more available capacity
Jump to solution
5 Replies
ashleyk
ashleyk•3mo ago
No description
ashleyk
ashleyk•3mo ago
Its supported by the 48GB tier which includes A40 and A6000, not just A40.
asherisaac
asherisaac•3mo ago
Thanks. Why can I only select 2 GPU's per instance in the 48gb tier? There is no option to do 4x 48gb or whatever for serverless?
ashleyk
ashleyk•3mo ago
To prevent someone from using all available capacity and leaving no capacity for the other customers.
Solution
flash-singh
flash-singh•3mo ago
we are slowly allowing more as we get more available capacity
Want results from more Discord servers?
Add your server
More Posts
Serverless API QuestionHi, I am currently using this guide https://doc.runpod.io/reference/runpod-apis, and attempting to rIssues with changing file permission to 400I have a ssh key that I'm trying to set the permission as 400 by running the following command `chmserverless endpoint, Jobs always 1 in queued, even 3 workers runningafter 600 s ,still 1 jos in queued,and log nothing. How i to see what is running? This morning when Why FileBrowser cant be opened?Why does a "HTTP Error 404" appear when I click on HTTP Service [Port 4040]? Even though the log outAre there very few GPUs that support CUDA 11.8?When I create a GPU Pod on Secure Cloud, if I select the CUDA 11.8 version, there are very few GPUs GPU speed getting slower and slowerYesterday I was using a 3090 and it was writing at 3.5 it/s which was great, but today im using a 30Serverless InferenceHi, I have been using runpod to train my model, and am very interested in using serverless computingServerless can't connect to s3Hey guys! I'm currently trying to get my serverless workers to retrieve a video from an Amazon s3 bu"How can I run multiple templates in one pod?""How can I run multiple templates in one pod?"How do I run Docker in a RunPod environment?I want to run the docker in the gpu pod,but the pod may be a docker container. How can I run the dok