max_num_seqs
I'm deploying a Serverless endpoint using deepseek-ai/DeepSeek-Coder-V2-Base, and encountering repeated engine failures due to the environment variable max_num_seqs being interpreted as a string ('1') instead of an integer.
This triggers:
TypeError: '<' not supported between instances of 'int' and 'str'Which breaks the Pydantic validation during vLLM scheduler configuration. Despite setting max_num_seqs=1 in the environment config (without quotes), the engine logs show it’s still passed as '1', which causes the crash. Would it be possible to: 1. Coerce max_num_seqs and similar numeric env vars to proper types before injection? 2. Or allow safe overrides via editable templates? Thanks in advance, would love to keep using DeepSeek V2 Base for plugin scaffolding!
2 Replies
Unknown User•4mo ago
Message Not Public
Sign In & Join Server To View