vLLM worker v2.11.3 on runpod broken
Hey guys hope you are well, we're an AI company testing your cloud provisions for the first time and have run into some odd behaviour. Pods are great, however with serverless we seem to be running into strange behaviour.
We've deployed 4x v2.11.3 vLLM workers (via your templates) for Qwen3/GPT-OSS-120b with a variety of configs. All models output pure jibberish:
Here's one example:
Input:
{
"input": {
"prompt": "Hello How are you?",
"temperature": 1
}
}
Response:
" \n\n! This is a day? \nThis is a test? \n=\n(Defect ID: ) ""<Email>"" has been removed first."
Given that these exact models, with the exact configs work perfectly on pods - could you please help?
Thanks
We've deployed 4x v2.11.3 vLLM workers (via your templates) for Qwen3/GPT-OSS-120b with a variety of configs. All models output pure jibberish:
Here's one example:
Input:
{
"input": {
"prompt": "Hello How are you?",
"temperature": 1
}
}
Response:
" \n\n! This is a day? \nThis is a test? \n=\n(Defect ID: ) ""<Email>"" has been removed first."
\n\nIt looks like you want to extract the content of an email, split by the separators Given that these exact models, with the exact configs work perfectly on pods - could you please help?
Thanks
