45 replies

`distrobox create name example nvidia

distrobox create --name example-nvidia-toolkit --additional-flags "--runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=all" --image nvidia/cuda

distrobox create --name example-nvidia-toolkit --additional-flags "--runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=all" --image nvidia/cuda

distrobox create --name example-nvidia-toolkit --additional-flags "--runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=all" --image nvidia/cuda

distrobox create --name example-nvidia-toolkit --additional-flags "--runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all -e NVIDIA_DRIVER_CAPABILITIES=all" --image nvidia/cuda

akdev•10/5/23, 2:50 AM

didn't seem to work, this is the full command line:

podman create
--hostname "ubuntu-latest.toronto.hq.akdev.xyz"
--name "ubuntu-latest"
--privileged
--security-opt label=disable
--user root:root
--ipc host
--network host
--pid host
--label "manager=distrobox"
--env "SHELL=/usr/bin/zsh"
--env "HOME=/var/home/akdev"
--volume /:/run/host:rslave
--volume /dev:/dev:rslave
--volume /sys:/sys:rslave
--volume /tmp:/tmp:rslave
--volume "/usr/bin/distrobox-init":/usr/bin/entrypoint:ro
--volume "/usr/bin/distrobox-export":/usr/bin/distrobox-export:ro
--volume "/usr/bin/distrobox-host-exec":/usr/bin/distrobox-host-exec:ro
--volume "/var/home/akdev":"/var/home/akdev":rslave
--volume /sys/fs/selinux
--volume /var/log/journal
--volume /run/user/1000:/run/user/1000:rslave
--volume /etc/hosts:/etc/hosts:ro
--volume /etc/resolv.conf:/etc/resolv.conf:ro
--ulimit host
--annotation run.oci.keep_original_groups=1
--mount type=devpts,destination=/dev/pts
--userns keep-id  --device=nvidia.com/gpu=all
--entrypoint /usr/bin/entrypoint
ubuntu:latest
--verbose
--name "akdev"
--user 1000
--group 1000
--home "/var/home/akdev"
--init "0"
--nvidia "0"
--pre-init-hooks ""
--additional-packages ""
-- ''

podman create
--hostname "ubuntu-latest.toronto.hq.akdev.xyz"
--name "ubuntu-latest"
--privileged
--security-opt label=disable
--user root:root
--ipc host
--network host
--pid host
--label "manager=distrobox"
--env "SHELL=/usr/bin/zsh"
--env "HOME=/var/home/akdev"
--volume /:/run/host:rslave
--volume /dev:/dev:rslave
--volume /sys:/sys:rslave
--volume /tmp:/tmp:rslave
--volume "/usr/bin/distrobox-init":/usr/bin/entrypoint:ro
--volume "/usr/bin/distrobox-export":/usr/bin/distrobox-export:ro
--volume "/usr/bin/distrobox-host-exec":/usr/bin/distrobox-host-exec:ro
--volume "/var/home/akdev":"/var/home/akdev":rslave
--volume /sys/fs/selinux
--volume /var/log/journal
--volume /run/user/1000:/run/user/1000:rslave
--volume /etc/hosts:/etc/hosts:ro
--volume /etc/resolv.conf:/etc/resolv.conf:ro
--ulimit host
--annotation run.oci.keep_original_groups=1
--mount type=devpts,destination=/dev/pts
--userns keep-id  --device=nvidia.com/gpu=all
--entrypoint /usr/bin/entrypoint
ubuntu:latest
--verbose
--name "akdev"
--user 1000
--group 1000
--home "/var/home/akdev"
--init "0"
--nvidia "0"
--pre-init-hooks ""
--additional-packages ""
-- ''

akdev•10/5/23, 2:50 AM

fails with:

Error: OCI runtime error: unable to start container "097cd5ee14db0b0a102b7f9cd6b9e1eba6d9d409a46466ee48ce7f3910059dd4": crun: error executing hook `/usr/bin/nvidia-container-runtime-hook` (exit code: 1)

Error: OCI runtime error: unable to start container "097cd5ee14db0b0a102b7f9cd6b9e1eba6d9d409a46466ee48ce7f3910059dd4": crun: error executing hook `/usr/bin/nvidia-container-runtime-hook` (exit code: 1)

me thinks this is because of the

--user root:root

--user root:root

akdev•10/5/23, 2:50 AM

in podman by default your user maps to root so therefore no permission issues

akdev•10/5/23, 2:51 AM

distrobox uses

--userns keep-id

--userns keep-id

and

--user root:root

--user root:root

which means your user maps to itself and root maps to root

akdev•10/5/23, 2:53 AM

works if you use

distrobox

distrobox

with

--root

--root

bsherman•10/5/23, 2:53 AM

change the

/etc/nvidia-container-runtime/config.toml

/etc/nvidia-container-runtime/config.toml

to remove

no-cgroups = true

no-cgroups = true

bsherman•10/5/23, 2:53 AM

akdev•10/5/23, 2:55 AM

akdev•10/5/23, 2:57 AM

I think at least for bazzite this should work ok, just add the stuff in the assemble file

Aakdev Click to see attachment

Kyle GospoOP•10/5/23, 3:01 AM

eugh, does it have to be root?

Kyle GospoOP•10/5/23, 3:01 AM

that's unfortunate

Kyle GospoOP•10/5/23, 3:03 AM

and distrobox is going to start enforcing root containers having their own passwords

Kyle GospoOP•10/5/23, 3:03 AM

which means I can't just assemble them

Kyle GospoOP•10/5/23, 3:03 AM

unless I also pass a flag to disable password checks at all, which is very dangerous

KKyle Gospo eugh, does it have to be root?

akdev•10/5/23, 3:07 AM

not really, it works on rootless podman

akdev•10/5/23, 3:07 AM

it just that distrobox combination of flags makes it using that flag is the lowest-effort workaround

KKyle Gospo and distrobox is going to start enforcing root containers having their own passw...

akdev•10/5/23, 3:09 AM

mm this probably means it will incompatible with toolbox images, at least semantically as toolbox images require

:NOPASSWD

:NOPASSWD

on sudo anyway

I guess the effect is that any password won't do anything when using these images as

sudo

sudo

will just not ask anyway

Kyle GospoOP•10/5/23, 3:33 AM

at least this might solve the controller issue

KKyle Gospo at least this might solve the controller issue

bsherman•10/5/23, 3:34 AM

the issue where it has to be connected before the container starts?

Kyle GospoOP•10/5/23, 3:34 AM

yea

bsherman•10/5/23, 3:34 AM

nice!

akdev•10/5/23, 3:43 AM

That affects flatpak too I think, we need a udev event proxy

Kyle GospoOP•10/5/23, 4:25 AM

so what exactly is needed to get this working then? I'm down to make some test images

akdev•10/5/23, 11:30 AM

For the nvidia thing?

Kyle GospoOP•10/5/23, 4:33 PM

Yeah, saw something about generating a file and deleting an existing one

Kyle GospoOP•10/5/23, 4:33 PM

Just making sure I have all the pieces since I can't test this myself

Kyle GospoOP•10/5/23, 5:06 PM

Unless that's upstream already

akdev•10/5/23, 5:06 PM

You just need to use the latest version of the nvidia container toolkit

akdev•10/5/23, 5:06 PM

And generate the CDI file

Kyle GospoOP•10/5/23, 5:06 PM

Ah cool, that's easy

akdev•10/5/23, 5:07 PM

CDI configuration is hardware specific

Kyle GospoOP•10/5/23, 5:07 PM

That CDI is optional, no? I saw something about being able to skip it and generate on the fly

Kyle GospoOP•10/5/23, 5:07 PM

via nvidia.com/all as an arg

akdev•10/5/23, 5:08 PM

No I think that just selects all gpus

akdev•10/5/23, 5:08 PM

I didn’t try on the new container toolkit version though because ublue is outdated in that part

Kyle GospoOP•10/5/23, 5:10 PM

Nvidia's docs say that CDI can change from driver updates

Kyle GospoOP•10/5/23, 5:10 PM

So unless we're generating that often that may be a problem

Kyle GospoOP•10/5/23, 5:12 PM

Let me find what I was reading just in case we can avoid this

Aakdev I didn’t try on the new container toolkit version though because ublue is outdat...

bsherman•10/5/23, 5:18 PM

i should get that updated today sometime, so tomorrow's builds should have newer toolit

Kyle GospoOP•10/5/23, 5:21 PM

@akdev https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/cdi-support.html#using-cdi-with-non-cdi-enabled-runtimes

Kyle GospoOP•10/5/23, 5:21 PM

akdev•10/5/23, 5:45 PM

I guess that works

akdev•10/5/23, 5:45 PM

Unclear what the difference is

Kyle GospoOP•10/5/23, 6:08 PM

https://github.com/ublue-os/bazzite/pull/408

GitHub

feat: Switch to rootful bazzite-arch container so we can use the Nv...

Aakdev I didn’t try on the new container toolkit version though because ublue is outdat...

bsherman•10/5/23, 8:42 PM

nothing like bypassing merge protection

https://github.com/ublue-os/akmods/commit/45719c8d4f2320772f8a4584c70a3b715fbfd88b

GitHub

feat: update to new repo for nvidia-container-toolkit · ublue-os/ak...

This enables us to get a newer nvidia container toolkit version 1.14
which now provides root and rootless podman modes without configuration
files, plus CDI.

Added some links to docs.

podman create --hostname "ubuntu-latest.toronto.hq.akdev.xyz" --name "ubuntu-latest" --privileged --security-opt label=disable --user root:root --ipc host --network host --pid host --label "manager=distrobox" --env "SHELL=/usr/bin/zsh" --env "HOME=/var/home/akdev" --volume /:/run/host:rslave --volume /dev:/dev:rslave --volume /sys:/sys:rslave --volume /tmp:/tmp:rslave --volume "/usr/bin/distrobox-init":/usr/bin/entrypoint:ro --volume "/usr/bin/distrobox-export":/usr/bin/distrobox-export:ro --volume "/usr/bin/distrobox-host-exec":/usr/bin/distrobox-host-exec:ro --volume "/var/home/akdev":"/var/home/akdev":rslave --volume /sys/fs/selinux --volume /var/log/journal --volume /run/user/1000:/run/user/1000:rslave --volume /etc/hosts:/etc/hosts:ro --volume /etc/resolv.conf:/etc/resolv.conf:ro --ulimit host --annotation run.oci.keep_original_groups=1 --mount type=devpts,destination=/dev/pts --userns keep-id --device=nvidia.com/gpu=all --entrypoint /usr/bin/entrypoint ubuntu:latest --verbose --name "akdev" --user 1000 --group 1000 --home "/var/home/akdev" --init "0" --nvidia "0" --pre-init-hooks "" --additional-packages "" -- ''

Error: OCI runtime error: unable to start container "097cd5ee14db0b0a102b7f9cd6b9e1eba6d9d409a46466ee48ce7f3910059dd4": crun: error executing hook `/usr/bin/nvidia-container-runtime-hook` (exit code: 1)

`distrobox create name example nvidia

Similar Threads

Similar Threads

Similar Threads