Monitor GPU VRAM - Which GPU to check?

I am trying to monitor the GPU VRAM usage in serverless worker. To do this with pynvml I need to provide the index of the GPU. Is there a way I can obtain the index of the GPU my worker is using? I did not see this info in the ENV variables. I do see RUNPOD_GPU_COUNT but not sure if that helps.

Seems that RunPod is monitoring cpu, gpu stats as they present that information in their web interface. Does the RunPod python module expose those stats, without having to code our own?

Below is a code snippet that reports VRAM usage in a %.

import pynvml
import time

# Initialize NVML
pynvml.nvmlInit()

handle = pynvml.nvmlDeviceGetHandleByIndex(0)  # Assuming you have only one GPU

while True:
    # Get the memory information for the GPU
    memory_info = pynvml.nvmlDeviceGetMemoryInfo(handle)

    used_vram = memory_info.used // (1024 ** 2)  # Convert bytes to MB
    total_vram = memory_info.total // (1024 ** 2)  # Convert bytes to MB
    vram_usage_percentage = round((used_vram / total_vram) * 100)

    print(f'vram usage: {vram_usage_percentage}%')

    time.sleep(5)

import pynvml
import time

# Initialize NVML
pynvml.nvmlInit()

handle = pynvml.nvmlDeviceGetHandleByIndex(0)  # Assuming you have only one GPU

while True:
    # Get the memory information for the GPU
    memory_info = pynvml.nvmlDeviceGetMemoryInfo(handle)

    used_vram = memory_info.used // (1024 ** 2)  # Convert bytes to MB
    total_vram = memory_info.total // (1024 ** 2)  # Convert bytes to MB
    vram_usage_percentage = round((used_vram / total_vram) * 100)

    print(f'vram usage: {vram_usage_percentage}%')

    time.sleep(5)

import pynvml
import time

# Initialize NVML
pynvml.nvmlInit()

handle = pynvml.nvmlDeviceGetHandleByIndex(0)  # Assuming you have only one GPU

while True:
    # Get the memory information for the GPU
    memory_info = pynvml.nvmlDeviceGetMemoryInfo(handle)

    used_vram = memory_info.used // (1024 ** 2)  # Convert bytes to MB
    total_vram = memory_info.total // (1024 ** 2)  # Convert bytes to MB
    vram_usage_percentage = round((used_vram / total_vram) * 100)

    print(f'vram usage: {vram_usage_percentage}%')

    time.sleep(5)

import pynvml
import time

# Initialize NVML
pynvml.nvmlInit()

handle = pynvml.nvmlDeviceGetHandleByIndex(0)  # Assuming you have only one GPU

while True:
    # Get the memory information for the GPU
    memory_info = pynvml.nvmlDeviceGetMemoryInfo(handle)

    used_vram = memory_info.used // (1024 ** 2)  # Convert bytes to MB
    total_vram = memory_info.total // (1024 ** 2)  # Convert bytes to MB
    vram_usage_percentage = round((used_vram / total_vram) * 100)

    print(f'vram usage: {vram_usage_percentage}%')

    time.sleep(5)

Thanks!

Monitor GPU VRAM - Which GPU to check?

Similar Threads

Similar Threads

Similar Threads