vLLM override open ai served model name
Overriding the served model name on the vllm serverless pod doesn't seem to take effect. Configuring a new endpoint through the explore page on runpod's interface creates a worker with the env variable
The logs show :
Setting the env variable
OPENAI_SERVED_MODEL_NAME_OVERRIDEThe logs show :
engine.py: AsyncEngineArgs(model='hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4', served_model_name=None...Error with model object='error' message='The model 'model_name' does not exist.' type='NotFoundError' param=None code=404Setting the env variable
SERVED_MODEL_NAMEengine.py: Engine args: AsyncEngineArgs(model='hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4', served_model_name='model_name'...
