447 replies

ublue/sysext Prototyping

sysext

follows

bketelsenOP•1/17/24, 8:03 PM

it turns out I work with most of the people who created and maintain systemd. And almost all of the people behind overlays and systemd-sysext. How convenient!

bketelsenOP•1/17/24, 8:03 PM

I had a call today with some of them to talk about strategy for bluefin-dx as a sysext. Here are my take-aways:

HikariKnight•1/17/24, 8:04 PM

that is very convenient

bketelsenOP•1/17/24, 8:05 PM

* If you're building or shipping dynamically linked things (complicated apps like lxd/incus/virt-manager etc) then your sysext needs to be built on and for the same os that you target.

bketelsenOP•1/17/24, 8:06 PM

* UNLESS you build into your own prefix (/usr/bluefin anyone?) and install all binaries and libraries there

bketelsenOP•1/17/24, 8:06 PM

* systemd-sysext merges images to /usr

bketelsenOP•1/17/24, 8:06 PM

* systemd-confext merges images to /etc

bketelsenOP•1/17/24, 8:07 PM

* Yes, we can ship things to /etc in an overlay

bketelsenOP•1/17/24, 8:07 PM

* Technically you can build your extension once and package it twice, once as a sysext and once as a confext

bketelsenOP•1/17/24, 8:10 PM

To put all of this into some sort of practice I refactored the ublue-os/sysext code a little bit. I changed the incus build to build all deps and incus itself with a prefix of /usr/bluefin/usr/bluefin. When merged the extension is completely harmless unless you have systemd units calling binaries in /usr/bluefin/bin directly. Libraries installed there can't conflict with the base system either.

bketelsenOP•1/17/24, 8:10 PM

I then refactored a bit more to move the sysext building into three phases: prep, compiling/wrangling, image building

bketelsenOP•1/17/24, 8:11 PM

finally I changed some just recipes to build metameta extensions that combine more than one of the previous extensions together. This serves as a proof of concept for one bluefin-dxbluefin-dx sysext with all the dev stuff included.

bketelsenOP•1/17/24, 8:11 PM

It worked perfectly.

bketelsenOP•1/17/24, 8:13 PM

finally, one of my coworkers wrote an OCI proxy for sysext so you can store your extensions in an oci repo like GHCR

bketelsenOP•1/17/24, 8:14 PM

so if we go down this route we'd create a matrix of builds for each fedora version we support [38,39,40] then we can build each extension and push them to ghcr. Updating them can be part of the ublue-update service.

bketelsenOP•1/17/24, 8:15 PM

Technically you can live merge/unmerge extensions but I think we shouldn't.

bketelsenOP•1/17/24, 8:15 PM

that's about it.

bketelsenOP•1/17/24, 8:16 PM

I'm very very excited about what this means for all of ublue, because the extensions we build will work outside of bluefin on any of our variants. bazzite-dxbazzite-dx comes for free.

bketelsenOP•1/17/24, 8:17 PM

@bsherman @Kyle Gospo @j0rge F40 ships systemd v255 which supports starting systemd units in extensions automatically

M2•1/17/24, 8:17 PM

so our hacky workaround scripts would then be fine?

bketelsenOP•1/17/24, 8:17 PM

so I think we should target ublue-dx alpha for the F40 release

MM2 so our hacky workaround scripts would then be fine?

bketelsenOP•1/17/24, 8:18 PM

which ones, specifically?

bketelsenOP•1/17/24, 8:19 PM

(I have a lot of hacky workaround scripts :))

M2•1/17/24, 8:19 PM

Groups, putting things like a default vscode config, our SELinux stuff

M2•1/17/24, 8:19 PM

capabilities not being right

M2•1/17/24, 8:19 PM

I'm guessing this would be in the sysext so would be fine

bketelsenOP•1/17/24, 8:19 PM

ah yes, all of those things can be put in systemd-confext images

bketelsenOP•1/17/24, 8:20 PM

groups get written to /etc/sysusers.d/ for example

bketelsenOP•1/17/24, 8:20 PM

when the extension merges, systemd creates the groups

bketelsenOP•1/17/24, 8:21 PM

last thing - the ublue-os/sysext repo is doing things manually - the hard way

bketelsenOP•1/17/24, 8:21 PM

but mkiso v20 and higher support building sysexts directly

bketelsenOP•1/17/24, 8:22 PM

create a base image, then a sysext image. mkiso ships the delta between the base & sysext

bketelsenOP•1/17/24, 8:22 PM

so that's really slick and powerful

Noel•1/17/24, 8:25 PM

I am very interested to see where this goes. If I'm understanding correctly, this could be useful for packaging binaries and config files that can be very easily switched on and off while the system is running?

bketelsenOP•1/17/24, 8:25 PM

correct @Noel

bketelsenOP•1/17/24, 8:25 PM

kind of like Neo "I know kung-fu"

bketelsenOP•1/17/24, 8:25 PM

"I have vscode and docker"

Noel•1/17/24, 8:27 PM

So is there more that has to be done to get the binaries in the form needed or can they be sourced from RPM or Deb packages?

Noel•1/17/24, 8:27 PM

Or are they compiled from source?

bketelsenOP•1/17/24, 8:27 PM

that's a choice we would make early.

bketelsenOP•1/17/24, 8:28 PM

if we're based on fedora 39 and we use f39 rpms to install stuff we'd be fine

bketelsenOP•1/17/24, 8:29 PM

the biggest issue is libraries and compatibility. that's the story behind my 1st and 2nd points in this thread

bketelsenOP•1/17/24, 8:29 PM

it's safest to put everything in it's own prefix. That's how I did incus, neovim, and vscode. So nothing they bring with them could hurt the base (host) OS.

bketelsenOP•1/17/24, 8:30 PM

but... if we're only installing things that are available by RPM there should be no problem just using them in their default locations. That's what redhat/fedora do for us, make sure all that crap works together.

j0rge•1/17/24, 8:31 PM

yeah if they're built on the OS then we should be fine? the /usr/bluefin thing, dunno, I'd need to read further

j0rge•1/17/24, 8:32 PM

like we don't need to build in a prefix right? we would just build it on a fedora builder?

bketelsenOP•1/17/24, 8:32 PM

the OS owns /usr, the host owns /usr/local, and bluefin would own /usr/bluefin. it's the same concept, and not included in any PATHs by default so that makes it safe

bketelsenOP•1/17/24, 8:32 PM

just like crap you install in ~/.local can't mess with /usr

Jj0rge like we don't need to build in a prefix right? we would just build it on a fedor...

bketelsenOP•1/17/24, 8:33 PM

correct yes, but some apps bring in deps not packaged in fedora so you risk polluting system libs

j0rge•1/17/24, 8:33 PM

ah ok

ublue/sysext Prototyping

Similar Threads

Similar Threads

Similar Threads