Serverless vLLM changing engine arguments

Hi, I got vLLM Serverless worker up and running, but want to change one engine argument (which is not overridable through environment variables), specifically

--limit-mm-per-prompt

--limit-mm-per-prompt

, how could I do that with your custom image

runpod/worker-v1-vllm:v2.3.0stable-cuda12.1.0

runpod/worker-v1-vllm:v2.3.0stable-cuda12.1.0

that endpoints use? Thanks

Communities Docs About Terms Privacy

Serverless vLLM changing engine arguments - Runpod

Serverless vLLM changing engine arguments

Serverless vLLM changing engine arguments

Similar Threads