RunPod Serverless Endpoint Issue - Jobs Complete But No Output Returned
Problem: My serverless endpoint jobs are completing successfully but returning empty results.
Endpoint ID: su6ufhaephnw03 (Stable Diffusion XL)
Symptoms:
Jobs submit successfully (200 OK)
Status shows COMPLETED
Stream endpoint returns {"status": "COMPLETED", "stream": []}
No output/images returned despite successful completion
Example Job:
Job ID: 3906a491-6431-48f2-8c6b-afe60e5f5be0-u1
Status: COMPLETED
Stream: Empty array
What I've Tried:
β
Simplified payload to minimal parameters
β
Fixed tensor precision issues (use_fp16: false)
β
Verified API authentication and endpoints
β
Multiple test runs with same result
Timeline: Issue started around July 31st, affecting all jobs since then.
Question: Is there a known issue with result retrieval for this endpoint? The jobs appear to process (they take normal execution time ~1s) but no output is returned through either the status or stream endpoints.
Additional Info: Using Python requests library, following standard serverless API patterns. Happy to provide more technical details if needed.
Any guidance would be appreciated! π
14 Replies
Unknown Userβ’4w ago
Message Not Public
Sign In & Join Server To View
Hey Jason! π
Thanks for the suggestion to check worker logs. Here's what I found:
GOOD NEWS: My precision fix worked! β
Recent jobs show no more tensor errors
Pipeline loads successfully (100% complete)
Jobs go from Started β Finished cleanly
MYSTERY: No handler return logging π€
2025-08-01 11:06:35.765 | info | 9x9qgwlpdx4b4j | Started.
2025-08-01 11:06:35.765 | info | 9x9qgwlpdx4b4j | Finished.
Missing from logs:
No return value logging
No handler output statements
No errors between Started/Finished
Question: Should the handler be logging what it's trying to return? The worker completes but I'm getting empty streams: {"status": "COMPLETED", "stream": []}
My payload (minimal):
json
{
"input": {
"prompt": "A realistic photograph of a modern office building",
"use_fp16": false,
"precision": "fp32"
}
}
Is there a way to add debug logging to see what the handler is actually returning before it hits the result callback system?
Unknown Userβ’4w ago
Message Not Public
Sign In & Join Server To View
K thanks !! Really appreciated
Unknown Userβ’4w ago
Message Not Public
Sign In & Join Server To View
correct. Worked previously then no more
New Endpoint: β
Creating fresh endpoint with 24GB GPU config - should be ready shortly.
Handler Code Request: I don't have direct access to the handler file in the RunPod container, but I suspect it's the standard Stable Diffusion XL template from RunPod's marketplace.
Key Questions:
Should the handler be logging return values? Currently seeing Started β Finished with no output logs
Is there a standard way to add debug logging to see what's being returned before the callback?
Could this be a template issue? The handler might be returning data in a format the callback system doesn't recognize
Current Behavior:
β
Jobs complete successfully
β
No errors in recent runs
β Stream returns empty: {"status": "COMPLETED", "stream": []}
β No handler return logging visible
Next Steps:
Testing new endpoint once created
Happy to share any specific handler code if you can point me to where it's accessible
Would love guidance on adding debug logging to troubleshoot return values
Should I be looking somewhere specific for the handler file, or is this typically built into the RunPod template?
Unknown Userβ’4w ago
Message Not Public
Sign In & Join Server To View
Will do, thanks again
Unknown Userβ’4w ago
Message Not Public
Sign In & Join Server To View
Handler File Location: The handler file is inside the RunPod container, not in my local codebase. I'm using the standard Stable Diffusion XL template from RunPod's marketplace.
What I have access to:
β
My client-side batch processor code
β
API requests/responses
β
Worker logs from RunPod
β The actual handler file inside the container
Questions:
Where can I find the handler file? Is it accessible through RunPod's interface or GitHub?
Is this the standard SDXL template handler? If so, do you have a link to the source?
Can I customize/override the handler? To add debug logging for return values?
Current Evidence:
Jobs complete successfully (Started β Finished)
No handler return logging visible in worker logs
Stream returns empty: {"status": "COMPLETED", "stream": []}
Using minimal payload: {"prompt": "...", "use_fp16": false, "precision": "fp32"}
My batch processor code (client-side) is working perfectly - the issue seems to be in the container's handler not returning data properly.
Would love guidance on accessing/debugging the actual handler! π
Created new endpoint
Old endpoint pattern:
Started. β Finished. β Failed to return job results.
New endpoint pattern:
Started. β Failed to return job results. (NO "Finished" log!)
This suggests:
Old endpoint: Job completes but can't return results
New endpoint: Job fails during execution (no "Finished" log)
Both endpoints have the same result callback failure (400 Bad Request), but the new endpoint isn't even finishing the job.
Question: Should there always be a Finished log when jobs complete? The missing Finished log on the new endpoint suggests the handler is crashing or failing silently during image generation.
Job ID for debugging: 0c67a99b-cfe1-4b5d-bd6e-a03da9bae9b7-u1 (worker: 3fhwe3blgxcl2h)
This pattern difference might be the key to understanding what's happening
Unknown Userβ’4w ago
Message Not Public
Sign In & Join Server To View
@bizetools
Escalated To Zendesk
The thread has been escalated to Zendesk!
Ticket ID: #21278
Unknown Userβ’4w ago
Message Not Public
Sign In & Join Server To View
You are a great help. Thanks heaps!