RunPod Header Timing is Off
These responses from the stream endpoint are coming in every second, yet the header date is saying, at points, that each response is 10 seconds apart. When this happens, nothing is shown in the stream despite the fact that I yield something every second. Is there a way to force the timing or another way I can get around this?
Huge sudden delay times in serverless
I'm using a webui forge serverless template for my endpoint, with a network volume attached and sometimes the results are very inconsistent. For example in the last two results you can see I use the same worker but one has a delay time of 3s and another 80.39s, the second request was submitted 4-5 seconds later after the first request, so there was no long time gap either.
I know the forge/automatic1111 templates usually take time to load but all this time up until this monday/tuesday or so it only took about 10-20 second delay time, but now I'm having 80-90 second delays. Didn't make any change in my code either. Anyone know the reason for this?...
Testing Async Handler Locally
Hi,
I am trying to test the async handler locally. I am following the documentation very closely (attached is an example of something I tried). However, I cannot seem to get the /status/{job_id} endpoint to return anything while the job is in progress (I expected it to return IN_PROGRESS and perhaps the values that have thus far been yielded) -- that is, I recieve no response. I am testing locally by running the handler with the --rp_serve_api flag.
Is there something I am doing wrong? Why can't I see the status of a job in progress?
...
