🐳 Workspace DIND — “directory not empty” error when stopping workspace automatically
Hey everyone,
I’m running a self-hosted Coder (latest version) setup, with a Docker-in-Docker (DIND) workspace template using sysbox-runc.
I’ve configured automatic start/stop for my workspaces.
When the workspace stops, if there are still containers running inside Docker, I get this error and the stop fails:
It looks like the workspace container can’t fully unmount overlay2 layers because dockerd is killed before it has time to clean up.
Here’s my setup:
- Terraform workspace template (using the Docker provider)
- Workspace container launched with sysbox-runc
- Docker-in-Docker started in startup_script via sudo dockerd &
Question:
What’s the recommended way (in a Docker-based, self-hosted setup) to gracefully stop dockerd and its inner containers when Coder stops or autostops the workspace?
Ideally I’d like a built-in way (like a “preStop” hook or a shutdown script) to ensure DIND cleans up properly and avoid this overlay2 error.
Any best practices or official guidance for handling Docker-in-Docker shutdown inside Coder workspaces would be much appreciated
I’m running a self-hosted Coder (latest version) setup, with a Docker-in-Docker (DIND) workspace template using sysbox-runc.
I’ve configured automatic start/stop for my workspaces.
When the workspace stops, if there are still containers running inside Docker, I get this error and the stop fails:
Error deleting container: ... directory not empty, driver "overlay2": unlinkat /var/lib/docker/overlay2/...
It looks like the workspace container can’t fully unmount overlay2 layers because dockerd is killed before it has time to clean up.
Here’s my setup:
- Terraform workspace template (using the Docker provider)
- Workspace container launched with sysbox-runc
- Docker-in-Docker started in startup_script via sudo dockerd &
Question:
What’s the recommended way (in a Docker-based, self-hosted setup) to gracefully stop dockerd and its inner containers when Coder stops or autostops the workspace?Ideally I’d like a built-in way (like a “preStop” hook or a shutdown script) to ensure DIND cleans up properly and avoid this overlay2 error.
Any best practices or official guidance for handling Docker-in-Docker shutdown inside Coder workspaces would be much appreciated
