Cannot run Docker in Docker anymore
My Coder instance started behaving very strangely this (or last) week. I'm running it through docker and have installed sysbox do be able to run docker containers within my workspaces. It's been working perfectly fine up until now. Now every time I try to start a docker container I get this:
I've tried trouble shooting by
- using different versions of Docker Engine in the workspaces
- using different versions of Coder docker image
- recreating workspaces
- used different templates and users
- comparing output of
So far my only conclusion is that old workspaces created before this week still work fine but any new ones I created will have the above error. Anyone else have or has had this problem? What else can I do to trouble shoot?
Cheers /Jon
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "proc" to rootfs at "/proc": mount src=proc, dst=/proc, dstFd=/proc/thread-self/fd/9, flags=0xe: no such file or directory: unknown.
I've tried trouble shooting by
- using different versions of Docker Engine in the workspaces
- using different versions of Coder docker image
- recreating workspaces
- used different templates and users
- comparing output of
docker inspectSo far my only conclusion is that old workspaces created before this week still work fine but any new ones I created will have the above error. Anyone else have or has had this problem? What else can I do to trouble shoot?
Cheers /Jon
Solution
Hah, I found the issue. If someone ever has the same problem: https://github.com/containerd/containerd/issues/11083
Works if downgrading containerd to version 1.7.23
RUN apt-get --reinstall install -y --allow-downgrades containerd.io=1.7.23-1
