Help Needed: Chatterbox TTS Server on Runpod Serverless - Jobs Stuck, Handler Not Reached
Hi everyone, we're struggling to get the Chatterbox TTS Server ( https://github.com/resemble-ai/chatterbox ) running correctly on Runpod Serverless. Any insights would be massively appreciated!
Current Situation:
- We're deploying a Dockerized version of the Chatterbox TTS server.
- The goal is to use it as a Runpod Serverless endpoint.
- Main Problem: After the image is pulled and the worker starts, jobs sent to the Runpod API endpoint (e.g., /runsync) remain "IN_QUEUE" indefinitely and eventually time out. The actual TTS processing never seems to happen.
Runpod Worker Logs Show:
- The NVIDIA CUDA preamble.
- Our FastAPI application (server.py) successfully starts up via Uvicorn on port 8004.
- The Chatterbox TTS model loads successfully within the FastAPI app.
We've adapted our setup based on Runpod Serverless documentation, which requires a specific handler pattern using their runpod Python SDK.
Example Test Payload Sent to Runpod API (/runsync):
{
"input": {
"text": "Hello Robert, this is a test using the Runpod handler.",
"voice_mode": "clone",
"reference_audio_filename": "Robert.wav"
}
}
We've been through many iterations, including simplifying CMD, adding detailed logging (which isn't appearing from handler.py), and ensuring the latest image is pulled. The FastAPI app seems fine on its own, but the "glue" (Runpod handler) isn't working.
Any advice on debugging this problem is much appreciated!
GitHub
GitHub - resemble-ai/chatterbox: SoTA open-source TTS
SoTA open-source TTS. Contribute to resemble-ai/chatterbox development by creating an account on GitHub.
5 Replies
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View
Here's my handler.py file.
I followed the guide where someone was told to write a handler and it still doesn't work I'm completely lost on this.
It simply doesn't send the request. I don't get it. No matter what I do I can't debug anything because I can't see the logs that are meaningful and will point me in a direction to fix the issue.
I don't see anything in the logs that indicate runpod.serverless.start() was initiated since I can't see any logs indicating so.
Here are the logs when I try to make a request to the /run endpoint with this as the object, it's loading the TTS server but it's not handling the request from what I can see in the logs.
Unknown User•3mo ago
Message Not Public
Sign In & Join Server To View