Encyrption
Encyrption
RRunPod
Created by Encyrption on 12/6/2024 in #⚡|serverless
With new pre-built serverless images how do we learn the API schema?
I see we can now select from some pre-built images for serverless. How can we learn the API schema for the input for these pre-built images? Thanks! 🙂
2 replies
RRunPod
Created by Encyrption on 9/16/2024 in #⚡|serverless
All workers saying Retrying in 1 second.
No description
1 replies
RRunPod
Created by Encyrption on 9/11/2024 in #⚡|serverless
Can we set public-read with rp_upload?
Using the boto module directly I can set the following when doing an s3 upload:
ExtraArgs={'ACL': 'public-read'}
ExtraArgs={'ACL': 'public-read'}
Is there a way I can apply that when using rp_upload? Below is an example of how I am using it:
image = rp_upload.upload_image(job_id, local_image_path)
image = rp_upload.upload_image(job_id, local_image_path)
This allows the uploaded bucket file to be accessible to the public for downloading.
2 replies
RRunPod
Created by Encyrption on 8/27/2024 in #⚡|serverless
v1 API definitions?
Is there any documentation for RunPod v1 endpoints? specifically looking for documentation for: https://hapi.runpod.net/v1/pod/{POD_ID}/logs This seems to be what RunPod uses to stream their logs from serverless workers to their website. I would like to implement similar functionality into my web app rather than streaming those logs over a web socket, with custom code as I do today. Is there an equivalent documented API call I can use to do this? If not, where can I find information on how to use the above API call in my code?
7 replies
RRunPod
Created by Encyrption on 8/26/2024 in #⚡|serverless
Monitor GPU VRAM - Which GPU to check?
I am trying to monitor the GPU VRAM usage in serverless worker. To do this with pynvml I need to provide the index of the GPU. Is there a way I can obtain the index of the GPU my worker is using? I did not see this info in the ENV variables. I do see RUNPOD_GPU_COUNT but not sure if that helps. Seems that RunPod is monitoring cpu, gpu stats as they present that information in their web interface. Does the RunPod python module expose those stats, without having to code our own? Below is a code snippet that reports VRAM usage in a %.
import pynvml
import time

# Initialize NVML
pynvml.nvmlInit()

handle = pynvml.nvmlDeviceGetHandleByIndex(0) # Assuming you have only one GPU

while True:
# Get the memory information for the GPU
memory_info = pynvml.nvmlDeviceGetMemoryInfo(handle)

used_vram = memory_info.used // (1024 ** 2) # Convert bytes to MB
total_vram = memory_info.total // (1024 ** 2) # Convert bytes to MB
vram_usage_percentage = round((used_vram / total_vram) * 100)

print(f'vram usage: {vram_usage_percentage}%')

time.sleep(5)
import pynvml
import time

# Initialize NVML
pynvml.nvmlInit()

handle = pynvml.nvmlDeviceGetHandleByIndex(0) # Assuming you have only one GPU

while True:
# Get the memory information for the GPU
memory_info = pynvml.nvmlDeviceGetMemoryInfo(handle)

used_vram = memory_info.used // (1024 ** 2) # Convert bytes to MB
total_vram = memory_info.total // (1024 ** 2) # Convert bytes to MB
vram_usage_percentage = round((used_vram / total_vram) * 100)

print(f'vram usage: {vram_usage_percentage}%')

time.sleep(5)
Thanks! 🙂
30 replies
RRunPod
Created by Encyrption on 8/12/2024 in #⚡|serverless
Any limits on execution timeout?
If I uncheck Enable Execution Timeout on an endpoint can I run a serverless worker indefinitely? If not what is the actual limit? Thanks!
2 replies
RRunPod
Created by Encyrption on 8/6/2024 in #⚡|serverless
Does webhook work when testing locally?
I am trying to test serverless worker locally and everything runs fine except it doesn't call the webhook I provided in the test_input.json file. Here is an example of the JSON I am sending is this correct for calling a webhook? { "input": { "sample": "testvalue" }, "webhook": "https://myhost.com/log" } Thanks! 🙂
14 replies
RRunPod
Created by Encyrption on 8/5/2024 in #⚡|serverless
Are the 64 / 128 Core CPU workers gone for good?
I noticed when selecting CPU workers for serverless endpoints that we are no longer given the option of the 64 or 128 vCPUs anymore. I know the 64/128 vCPUs were having issues running jobs. I am wondering if they are going to come back or are they gone for good? Thanks! 🙂
5 replies
RRunPod
Created by Encyrption on 7/26/2024 in #⚡|serverless
Is privileged mode possible?
I have an application that requires a kernel module be loaded. For an image to add a kernel modules requires privileged mode from the host. Is there anyway to get privileged mode enabled on my images so that I can add a kernel module to it?
2 replies
RRunPod
Created by Encyrption on 6/12/2024 in #⚡|serverless
Can serverless endpoints make outbound TCP connections?
I know endpoints can make http/https requests but is there any limit on outbound connections? Is there a FW or are all ports open? What about bandwidth limitations, etc.? Thanks!
15 replies
RRunPod
Created by Encyrption on 6/10/2024 in #⚡|serverless
Anyone have example template for OpenVoice V2 serverless?
I would like to deploy https://github.com/myshell-ai/OpenVoice on serverless. It has a huggingface module can it be implement in the Serverless vLLM quick deploy? If so are there any instructions for doing so? If not what are my options for getting it installed?
11 replies
RRunPod
Created by Encyrption on 6/8/2024 in #⚡|serverless
CPU Instances on 64 / 128 vCPUs FAIL
I can deploy my app on all instances except for 64 & 128 vCPU. Both of these run on AMD EPYC 9754 128-Core Processor. When it tries to run it gets stuck in QUEUE with the error (pasted below). When this happens it then just loops between "start container" and "failed to create shim task: the file python was not found: unknown". Any ideas what is causing this and how to resolve? There is similar issue reported in pods section here but I am using serverless and getting same problem. ERROR from instance: error creating container: container: create: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.43/containers/03f5da1a67e9f72498f779b9923cb7927a703cc84d173fa038041e72a7caac9b/start": context deadline exceeded
19 replies