Atherion
RRunPod
•Created by LisT_99 on 2/6/2025 in #⚡｜serverless
vLLM serverless output cutoff
I was running into the same issue. There is a key for sampling_params, found the solution here https://docs.runpod.io/serverless/workers/vllm/get-started#sample-api-requests
5 replies