Is VLLM Automatic Prefix Caching enabled by default?
Hello!
I setup a Serverless quick deployment for text generation and I was wondering if VLLM Automatic Prefix Caching is enabled by default? Also see:
https://docs.vllm.ai/en/latest/automatic_prefix_caching/apc.html
Kind regards,
-- Freddy
2 Replies
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
Thanks @nerdylive, will look in to it!