Plans to support 400B models like llama 3?
Is runpod thinking about how they will support vvllms like 400B Llama model that is expected to release later this year?
8 Replies
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
I doubt 2 x 80GB are sufficient to load a 400B model.
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
I see for 48GB tier, you can have up to 10 GPU per worker which is cool.
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
I am referring to serverless.
Unknown User•2y ago
Message Not Public
Sign In & Join Server To View
we're pretty far from 400B release afaik, limits will likely be different then