`distrobox create name example nvidia

distrobox create --name example-nvidia-toolkit --additional-flags "--runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=all" --image nvidia/cuda
27 Replies
akdev
akdev9mo ago
didn't seem to work, this is the full command line:
podman create
--hostname "ubuntu-latest.toronto.hq.akdev.xyz"
--name "ubuntu-latest"
--privileged
--security-opt label=disable
--user root:root
--ipc host
--network host
--pid host
--label "manager=distrobox"
--env "SHELL=/usr/bin/zsh"
--env "HOME=/var/home/akdev"
--volume /:/run/host:rslave
--volume /dev:/dev:rslave
--volume /sys:/sys:rslave
--volume /tmp:/tmp:rslave
--volume "/usr/bin/distrobox-init":/usr/bin/entrypoint:ro
--volume "/usr/bin/distrobox-export":/usr/bin/distrobox-export:ro
--volume "/usr/bin/distrobox-host-exec":/usr/bin/distrobox-host-exec:ro
--volume "/var/home/akdev":"/var/home/akdev":rslave
--volume /sys/fs/selinux
--volume /var/log/journal
--volume /run/user/1000:/run/user/1000:rslave
--volume /etc/hosts:/etc/hosts:ro
--volume /etc/resolv.conf:/etc/resolv.conf:ro
--ulimit host
--annotation run.oci.keep_original_groups=1
--mount type=devpts,destination=/dev/pts
--userns keep-id --device=nvidia.com/gpu=all
--entrypoint /usr/bin/entrypoint
ubuntu:latest
--verbose
--name "akdev"
--user 1000
--group 1000
--home "/var/home/akdev"
--init "0"
--nvidia "0"
--pre-init-hooks ""
--additional-packages ""
-- ''
podman create
--hostname "ubuntu-latest.toronto.hq.akdev.xyz"
--name "ubuntu-latest"
--privileged
--security-opt label=disable
--user root:root
--ipc host
--network host
--pid host
--label "manager=distrobox"
--env "SHELL=/usr/bin/zsh"
--env "HOME=/var/home/akdev"
--volume /:/run/host:rslave
--volume /dev:/dev:rslave
--volume /sys:/sys:rslave
--volume /tmp:/tmp:rslave
--volume "/usr/bin/distrobox-init":/usr/bin/entrypoint:ro
--volume "/usr/bin/distrobox-export":/usr/bin/distrobox-export:ro
--volume "/usr/bin/distrobox-host-exec":/usr/bin/distrobox-host-exec:ro
--volume "/var/home/akdev":"/var/home/akdev":rslave
--volume /sys/fs/selinux
--volume /var/log/journal
--volume /run/user/1000:/run/user/1000:rslave
--volume /etc/hosts:/etc/hosts:ro
--volume /etc/resolv.conf:/etc/resolv.conf:ro
--ulimit host
--annotation run.oci.keep_original_groups=1
--mount type=devpts,destination=/dev/pts
--userns keep-id --device=nvidia.com/gpu=all
--entrypoint /usr/bin/entrypoint
ubuntu:latest
--verbose
--name "akdev"
--user 1000
--group 1000
--home "/var/home/akdev"
--init "0"
--nvidia "0"
--pre-init-hooks ""
--additional-packages ""
-- ''
fails with:
Error: OCI runtime error: unable to start container "097cd5ee14db0b0a102b7f9cd6b9e1eba6d9d409a46466ee48ce7f3910059dd4": crun: error executing hook `/usr/bin/nvidia-container-runtime-hook` (exit code: 1)
Error: OCI runtime error: unable to start container "097cd5ee14db0b0a102b7f9cd6b9e1eba6d9d409a46466ee48ce7f3910059dd4": crun: error executing hook `/usr/bin/nvidia-container-runtime-hook` (exit code: 1)
me thinks this is because of the --user root:root in podman by default your user maps to root so therefore no permission issues distrobox uses --userns keep-id and --user root:root which means your user maps to itself and root maps to root works if you use distrobox with --root
bsherman
bsherman9mo ago
change the /etc/nvidia-container-runtime/config.toml to remove no-cgroups = true ? ah
akdev
akdev9mo ago
No description
akdev
akdev9mo ago
I think at least for bazzite this should work ok, just add the stuff in the assemble file
1/4 Life
1/4 Life9mo ago
eugh, does it have to be root? that's unfortunate and distrobox is going to start enforcing root containers having their own passwords which means I can't just assemble them unless I also pass a flag to disable password checks at all, which is very dangerous
akdev
akdev9mo ago
not really, it works on rootless podman it just that distrobox combination of flags makes it using that flag is the lowest-effort workaround mm this probably means it will incompatible with toolbox images, at least semantically as toolbox images require :NOPASSWD on sudo anyway I guess the effect is that any password won't do anything when using these images as sudo will just not ask anyway
1/4 Life
1/4 Life9mo ago
No description
1/4 Life
1/4 Life9mo ago
at least this might solve the controller issue
bsherman
bsherman9mo ago
the issue where it has to be connected before the container starts?
1/4 Life
1/4 Life9mo ago
yea
bsherman
bsherman9mo ago
nice!
akdev
akdev9mo ago
That affects flatpak too I think, we need a udev event proxy 🧐
1/4 Life
1/4 Life8mo ago
so what exactly is needed to get this working then? I'm down to make some test images
akdev
akdev8mo ago
For the nvidia thing?
1/4 Life
1/4 Life8mo ago
Yeah, saw something about generating a file and deleting an existing one Just making sure I have all the pieces since I can't test this myself Unless that's upstream already
akdev
akdev8mo ago
You just need to use the latest version of the nvidia container toolkit And generate the CDI file
1/4 Life
1/4 Life8mo ago
Ah cool, that's easy 🙂
akdev
akdev8mo ago
CDI configuration is hardware specific
1/4 Life
1/4 Life8mo ago
That CDI is optional, no? I saw something about being able to skip it and generate on the fly via nvidia.com/all as an arg
akdev
akdev8mo ago
No I think that just selects all gpus I didn’t try on the new container toolkit version though because ublue is outdated in that part
1/4 Life
1/4 Life8mo ago
Nvidia's docs say that CDI can change from driver updates So unless we're generating that often that may be a problem Let me find what I was reading just in case we can avoid this
bsherman
bsherman8mo ago
i should get that updated today sometime, so tomorrow's builds should have newer toolit
1/4 Life
1/4 Life8mo ago
No description
akdev
akdev8mo ago
I guess that works Unclear what the difference is
bsherman
bsherman8mo ago
GitHub
feat: update to new repo for nvidia-container-toolkit · ublue-os/ak...
This enables us to get a newer nvidia container toolkit version 1.14 which now provides root and rootless podman modes without configuration files, plus CDI. Added some links to docs.