Deploy BART on serverless
Hi!
Does anyone know how to deploy bart-large-mnli on serverless? Been trying with the hugging face template (ghcr.io/huggingface/text-generation-inference) but I always get "Error: ShardCannotStart"
I already tried setting NUM_SHARD = 1 in env but still failing.
Repo for reference: https://huggingface.co/facebook/bart-large-mnli
Let me know if you need further details!
Thank you
Does anyone know how to deploy bart-large-mnli on serverless? Been trying with the hugging face template (ghcr.io/huggingface/text-generation-inference) but I always get "Error: ShardCannotStart"
I already tried setting NUM_SHARD = 1 in env but still failing.
Repo for reference: https://huggingface.co/facebook/bart-large-mnli
Let me know if you need further details!
Thank you
