"IN QUEUE" and nothing happeneds
Hello everyone,
I'm currently running a TGI container (ghcr.io/huggingface/text-generation-inference:2.2.0) within a serverless environment, alongside my model from Hugging Face.
Issue: Although the status indicates "connected," there seems to be no further activity. The logs display various INFO and WARNING messages but do not show any errors. This has left me puzzled as to the root cause of the problem.
I have attached the relevant logs and screenshots for reference.
Any insights or suggestions to troubleshoot this issue would be greatly appreciated.
5 Replies
Unknown User•16mo ago
Message Not Public
Sign In & Join Server To View
@nerdylive sorry, but even i do curl -X POST https://www.runpod.io/console/serverless/user/endpoint/jhfkefjljllckt -H 'Content-Type: application/json' -H 'Authorization: Bearer xxxxx' -d '{"input": {"prompt": "cancer is:"}}' , it is just in queue and nothing happends.
what exactly should i find in the documentation? can you explain please
Can you provide source code of your handler? You can identify the name of your handler by looking for a line like this in your Python code:
In this example handler in the name of handler so in your code look for code similar to this:
#Handler function that will be used to process jobs.
If you can provide this code it may shed light on your issue. Also, this is a good place to add print commands, to print out debug info. This debug info will appear in your logs.
@Encyrption dear friend, you have just explained everything I need. Indeed I have forgotten handler, silly me. now everything works perfectly! Thank you!
Great news! 🙂