Format of runpod.create_endpoint gpu_ids
It's unclear what the format is, the website is using this format: {
"data": {
"saveEndpoint": {
"gpuIds": "['NVIDIA GeForce RTX 4090'],ADA_24",
"id": "ncipvwdj6cuz8y",
"idleTimeout": 5,
"locations": null,
"name": "7891c33062e411f095a9000000b9f2e1",
"type": "QB",
"networkVolumeId": null,
"scalerType": "QUEUE_DELAY",
"scalerValue": 4,
"templateId": "tsef8csm02",
"userId": "google-oauth2|100781309549972343660",
"workersMax": 3,
"workersMin": 0,
"gpuCount": 1,
"__typename": "Endpoint"
}
}
}
When I just pass NVIDIA GeForce RTX 4090 to gpuIds and check the UI, nothing is selected, only when I select a GPU manually it persists.
7 Replies
Unknown User•2mo ago
Message Not Public
Sign In & Join Server To View
I have a serverless deploy script that creates endpoints programmatically, it creates the endpoint but gpu IDs is not set correctly
Unknown User•2mo ago
Message Not Public
Sign In & Join Server To View
mutation {
saveEndpoint(
input: {
name: "7891c33062e411f095a9000000b9f2e1", templateId: "tsef8csm02", gpuIds: "['NVIDIA GeForce RTX 4090']", networkVolumeId: "", locations: "", idleTimeout: 5, scalerType: "QUEUE_DELAY", scalerValue: 4, workersMin: 0, workersMax: 3, gpuCount: 1
}
) {
id
name
templateId
gpuIds
networkVolumeId
locations
idleTimeout
scalerType
scalerValue
workersMin
workersMax
allowedCudaVersions
gpuCount
}
}
figured out the IDs by reverse engineering: class GPUType(Enum):
"""GPU types available in RunPod using their official GPU ID codes."""
# Ada Lovelace Architecture
ADA_24 = "ADA_24" # L4, RTX 4000 series consumer cards
ADA_32_PRO = "ADA_32_PRO" # Professional Ada cards with 32GB
ADA_48_PRO = "ADA_48_PRO" # L40, L40S, RTX 6000 Ada
ADA_80_PRO = "ADA_80_PRO" # High-end Ada professional cards
# Ampere Architecture AMPERE_16 = "AMPERE_16" # RTX 3060, A2000, A4000 AMPERE_24 = "AMPERE_24" # RTX 3070/3080/3090, A4500, A5000 AMPERE_48 = "AMPERE_48" # A40, RTX A6000 AMPERE_80 = "AMPERE_80" # A100 80GB
# Hopper Architecture HOPPER_141 = "HOPPER_141" # H200 with 141GB memory
# Ampere Architecture AMPERE_16 = "AMPERE_16" # RTX 3060, A2000, A4000 AMPERE_24 = "AMPERE_24" # RTX 3070/3080/3090, A4500, A5000 AMPERE_48 = "AMPERE_48" # A40, RTX A6000 AMPERE_80 = "AMPERE_80" # A100 80GB
# Hopper Architecture HOPPER_141 = "HOPPER_141" # H200 with 141GB memory
Unknown User•2mo ago
Message Not Public
Sign In & Join Server To View
ok, just realized the REST API is documented and I was using the GraphQL API as this is being used by the runpod sdk. is probably worth updating the runpod sdk to match the REST API or is the sdk depcrated for making API calls?
We're working on getting upgrades out for runpod-python to use the REST API đź‘Ť
It's a long term push and technically a cross functional one so it moves slowly but it still moves!