When serverless uses a worker, is that worker shared between other serverless endpoints?
Or is it dedicated to the worker until terminated?
TAnd would running an on-demand pod with the same GPU be expected to yield lower execution times for the same workload? Or would they be the same?
3One worker is one unique and stateless instance of the container. You can save or share files between them with network volume.
The same GPU + datacenter location on serverless and pod should theoretically have similar performance. The pod takes longer to start - you also have to pay for the container image download.
The same GPU + datacenter location on serverless and pod should theoretically have similar performance. The pod takes longer to start - you also have to pay for the container image download.
Tgreat thank you, I assume the main reason for the pods is to avoid cold starts/pay less if you have continuous use then
3Basically, yes. In-depth, there are more differences. With pods, you can SSH into them or use a virtual desktop - it can serve as a remote computer if you want. With serverless, you're expected to interact with the endpoint via API requests and define the logic inside a handler function.
