so i m wondering what it looks like to
so, i'm wondering what it looks like to modernize
ublue-nvidia ...?
Bi'll thread it 
Ba few things have been brought up
B1) we install an selinux policy ... and it provides an selinux domain to use in the container label to use nvidia stuff without full on disabling selinux (at least for the container runtime)
B(example:
podman run --security-opt label=type:nvidia_container_t --rm nvcr.io/nvidia/cuda nvidia-smi)Bbut... @akdev has comments which suggest possibly removing it...
it's a repo which is old (2 years since last update) https://github.com/NVIDIA/dgx-selinux/tree/master/src/nvidia-container-selinux
even then, it was meant to be a reference for RHEL-based systems....not verbatim as used in ublue now...
it's a repo which is old (2 years since last update) https://github.com/NVIDIA/dgx-selinux/tree/master/src/nvidia-container-selinux
even then, it was meant to be a reference for RHEL-based systems....not verbatim as used in ublue now...
Balso... the
nvidia-container-toolkit RPM includs oci hooks... but those are deprecatedBshould we be removing the old hook.json?
Band should instead package a unit to generate
/etc/cdi/nvidia.yaml ?
AI think so
Bi don't really know the consequences of the change... i think with the CDI way, we can use
--device=nvidia.com/gpu=XYXAyeah and we can't use the
-e NVIDIA_GPU_DEVICES wayAother than that not sure
Bi kinda wonder if all this is too opinionated
Awdym
Bmaybe we remove the selinux policy because we can't really verify it's correct in the first place (seems unlikely)
Bbut how much auto-configuring a system for podman to run nvidia is too much?
Bhmm... would be nice to have the newer container-toolkit ... we are still on 1.13.5 and 1.14 is where CDI becomes "legit"
AI think in the future the podman team will remove the default hooks-dir, so we'd have to set that instead if that happens
Athough tbh that seems far in the future so we don't have to move to CDI now
Aand even then setting the hooks-dir in distrobox assemble or whatever would be easy at that point too
Byeah... currently our nvidia-container-toolkit is coming from nvidia's rhel9.0 repo so...
Alooks like they have a new one:
curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
Bhuh, and that actually points to the centos8 repos...
Byeah
Ait's not distro specific anymore seems like
Bi'm going to test that on the ucore side now since I'm setup there right now
Bbooyah! new container-toolkit
Bi guess it pays to be hard headed if it leads to progress
BBand... the oci-hook.json is gone 
Bi think we have our answer on this part at least
Bthe old SElinux policy still works... so... i guess i'm hesitant to remove it, even if the default way is for people to just
label=disable it doesn't hurtBAfair enough, the main concern is that it may break in the future so if that happens I guess we can remove it
Bagreed
Banother interesting side-effect of the upgrade to container-toolkit 1.14
Bthey don't provide a default config.toml anymore
AWhat’s that file?
B/etc/nvidia-container-runtime/config.toml
was where we were setting
was where we were setting
no_usecgroups = true to enable rootless modeBi think they fixed it so both rootful and rootless work now
AOh that’s nice
BAWould explain why that disappeared
Bvery nice!
ATry distrobox I guess maybe it even works
A(But probably not)
Bhah, well, i don't have distrobox on ucore
Bi can layer i suppose
Bcorrection! i do have it
Bwhat am i trying?
