Not possible to set temperature / top_p using Serverless vLLM via quick deploy?
By default, vLLM loads sampling parameters (e.g.
Because RunPod's worker-vllm (https://github.com/runpod-workers/worker-vllm) doesn't expose an environment variable to pipe through a
And the solutions is a customer Docker image / Worker?
temperaturetop_pgeneration_config.json--generation-configBecause RunPod's worker-vllm (https://github.com/runpod-workers/worker-vllm) doesn't expose an environment variable to pipe through a
--generation-configtemperaturetop_pgeneration_config.jsonAnd the solutions is a customer Docker image / Worker?
