Best workflow to test if serverless docker containers work?
Currently, this is my workflow -> make a code change locally in Dockerfile -> cloud build happens (30mins) -> docker pushed to dockerhub -> I restart runpod serverless workers manually -> I send a request using the test page -> runpod downloads the docker (10 mins) -> model actually runs (10mins)
Thus, it takes 1hr for me to test any change I make on serverless docker containers. What is a good way to build dockers or test changes more quickly?
4 Replies
Before I was an employee I would just test all my changes by syncing my files to a Pod and hoping it ran when I did whatever the Docker image did :thinkMan:
We're working on some way to smooth out this process at the moment, I definitely know the struggle. If you have any ideas feel free to share!
I will share part of my development method.
To save on image downloads in runpod, divide your Docker image into finer layers.
If a serverless worker has already been deployed once, it will utilize the existing cache. (A new worker will need to download everything from scratch.)
By splitting the Docker image layers into smaller, distributed parts, you can significantly reduce the difference in image downloads, resulting in much faster deployment.
For updates with only minor changes, the deployment time can be completed within a few minutes.
(By avoiding the use of the "latest" tag and specifying versions explicitly, you can prevent unintended updates.)
Also, it is recommended to use two workers.
If you use more than two, deployments may run simultaneously or standby workers may initialize, so by keeping two, you can quickly test the worker that finishes deployment first.
When creating an early-stage container image, it is better not to use a serverless worker.
Instead, simply SSH into a normal Pod and run it directly.
Set the CMD of the image you are using to something like sleep 99999, deploy it to the Pod, then SSH in and make the necessary manual modifications.
Once it works, you can incorporate the successful changes back into the image.
Well, the fastest way to test is to use the GPU on your development machine, but that’s not always possible if you are developing on macOS.
I develop on macOS, but I have an Ubuntu machine with an RTX 4090 at home, so I use Docker Context to run it there.
This is the fastest and most production-like way to develop.
To help, here's a CMD that installs ssh, copies your SSH key into it and allows you to connect to any image as a Pod:
I use this personally :)
