Using Same GPU for multiple requests?
Hello @here,
I am using ComfyUI + my own custom scripts to generate images. I have set it up in RunPod Serverless (A100) GPUs in the following way:
The request contains an image URL.
The image is downloaded and processed, and the output image is put to S3.
The task takes around 30 seconds
However, only 10% of the GPU memory is used by 1 request AT MAX. I want multiple requests to use the same GPU so that it is faster.
Is there a way to do this? Is there some existing template to handle this scenario?
I am using ComfyUI + my own custom scripts to generate images. I have set it up in RunPod Serverless (A100) GPUs in the following way:
The request contains an image URL.
The image is downloaded and processed, and the output image is put to S3.
The task takes around 30 seconds
However, only 10% of the GPU memory is used by 1 request AT MAX. I want multiple requests to use the same GPU so that it is faster.
Is there a way to do this? Is there some existing template to handle this scenario?
