Oh I think I've figured it out... rpm-ostree places `/etc/` files from layered RPMs in `/usr/etc/` a

Oh I think I've figured it out... rpm-ostree places

/etc/

/etc/

files from layered RPMs in

/usr/etc/

/usr/etc/

and then systemd attempts to merge them into

/etc/

/etc/

automatically, if possible. This is possible if the file is new or if the edit is trivial. But I am guessing kernel args are non-trivial and won't auto-merge, hence the need for manually editing the grub config.

ArcitecOP•5/14/23, 2:25 AM

So tl;dr: Your boot args are in your etc file permanently and will stay there. is most likely just a tool for quickly editing the etc file.

ragectl•5/14/23, 2:26 AM

This older Fedora article indicates they are using

/etc/kernel/cmdline

/etc/kernel/cmdline

https://fedoramagazine.org/manual-action-required-to-update-fedora-silverblue-kinoite-and-iot-version-36/

Fedora MagazineTimothée Ravier

Manual action required to update Fedora Silverblue, Kinoite and IoT...

Manual action required to update Fedora36 Silverblue, Kinoite and IoT

Rragectl This older Fedora article indicates they are using `/etc/kernel/cmdline` https...

ArcitecOP•5/14/23, 2:27 AM

Yeah I remember that bug. Either way it's all

/etc/

/etc/

related which explains why some things can't be automatically merged without intervention. But your system as it is now has already had those args applied and should be permanently ready now.

AArcitec Oh I think I've figured it out... rpm-ostree places `/etc/` files from layered R...

pythoner6•5/14/23, 2:29 AM

My understanding was that the etc merge works like this: if you have modified the file in any way, the file is left alone and you don't get the update. If you haven't modified it it'll be updated to what's in the new /usr/etc

pythoner6•5/14/23, 2:31 AM

(where "modified in any way" means "doesn't match what's in the old /usr/etc)

akdev•5/14/23, 2:31 AM

It'll apply a diff if it can

akdev•5/14/23, 2:31 AM

Or at least it's supposed to, I haven't checked

pythoner6•5/14/23, 2:32 AM

Huh, that's not how I had seen it described.

Ppythoner6 My understanding was that the etc merge works like this: if you have modified th...

ArcitecOP•5/14/23, 2:32 AM

It applies a 3-way merge which solves simple conflicts automatically. But if there's a mismatch it leaves your file alone and you have to run a command to merge manually.

Ppythoner6 Huh, that's not how I had seen it described.

akdev•5/14/23, 2:33 AM

It's honestly harder to explain the actual behaviour and not worth going into details most of the time imo but here it is https://blog.verbum.org/2014/01/24/why-ostree-requires-usretc/

Colin WaltersColin Walters

Why OSTree requires “/usr/etc”

While I often lump “dpkg/rpm” together since they’re very similar architecturally, their implementation of configuration file management is a small but nontrivial difference. This…

ArcitecOP•5/14/23, 2:33 AM

One cool thing is that since

/usr/etc/

/usr/etc/

are the "ground truth templates", you can always run an ostree config diff to see what files you have changes in or files that you;'ve kept that don't exist upstream.

akdev•5/14/23, 2:33 AM

I think of it mostly as you explained because I don't count on that three way merge to work reliably

Aakdev It's honestly harder to explain the actual behaviour and not worth going into de...

pythoner6•5/14/23, 2:33 AM

Yes that's what I was saying

It doesn’t attempt to understand the contents of files – if you have modified a config file in any way, that wins

akdev•5/14/23, 2:34 AM

Mm you're right

Ppythoner6 Yes that's what I was saying > It doesn’t attempt to understand the contents of ...

ArcitecOP•5/14/23, 2:34 AM

I read elsewhere that it's running a patch/diff, similar to this: Generate a patch between your

/usr/etc/

/usr/etc/

and the new deployment's

/usr/etc/

/usr/etc/

and then apply those patches to your

/etc/

/etc/

and just ignore the ones that failed to apply.

akdev•5/14/23, 2:35 AM

Not sure why he calls it a 3-way merge though if it is not merging anything

ArcitecOP•5/14/23, 2:35 AM

Yeah that blog is a mess of mixed up terms.

ArcitecOP•5/14/23, 2:36 AM

Official docs are here and they mention the thing I just said: https://ostreedev.github.io/ostree/deployment/

pythoner6•5/14/23, 2:37 AM

https://github.com/ostreedev/ostree/blob/16cb47489e582da9c139fee20acdac7079867843/docs/atomic-upgrades.md?plain=1#L76

GitHub

ostree/atomic-upgrades.md at 16cb47489e582da9c139fee20acdac70798678...

Operating system and container binary deployment and upgrades - ostree/atomic-upgrades.md at 16cb47489e582da9c139fee20acdac7079867843 · ostreedev/ostree

Ppythoner6 https://github.com/ostreedev/ostree/blob/16cb47489e582da9c139fee20acdac707986784...

ArcitecOP•5/14/23, 2:37 AM

But read this from the OSTree website:

When OSTree creates a deployment, it performs a 3-way merge using the old default configuration, the active system’s /etc, and the new default configuration. In the final filesystem tree for a deployment then, /etc is a regular writable directory.

pythoner6•5/14/23, 2:37 AM

How it works is:
Files in the currently booted deployment's
/etc
/etc
which were modified from the default
/usr/etc
/usr/etc
(of the same deployment) are retained.
Files in the currently booted deployment's
/etc
/etc
which were not modified from the default
/usr/etc
/usr/etc
(of the same deployment) are upgraded to the new defaults from the new deployment's
/usr/etc
/usr/etc
.

ArcitecOP•5/14/23, 2:38 AM

Why does their own website contradict the part you just quoted?

pythoner6•5/14/23, 2:39 AM

I don't think it's a contradiction do much as ambiguous use of the term "merge"

ArcitecOP•5/14/23, 2:39 AM

Yeah it could just be poorly explained. Grr.

pythoner6•5/14/23, 2:39 AM

Individual files aren't being merged, but the entire /etc tree is being merged in a very coarse grained way

ArcitecOP•5/14/23, 2:40 AM

Oh yeah that makes sense. What a poor way to phrase that on their site. I've always thought it did a 3-way merge per-file, not directory-wise.

ArcitecOP•5/14/23, 2:40 AM

I've never even thought of directory comparisons as "3-way merging".

pythoner6•5/14/23, 2:41 AM

Yeah it's definitely not the best way to explain it without a bit more detail

ArcitecOP•5/14/23, 2:41 AM

Yeah, thanks for finding that, it's very useful to know. That changes how I'll maintain my system. I will run the "config diff" more often to resolve actual changes.

pythoner6•5/14/23, 2:42 AM

It's probably just a case of the people writing the docs being familiar enough with how it works that the explanation seems obvious enough - even though it's a bit unclear without the extra detail to someone not as familiar.

ArcitecOP•5/14/23, 2:43 AM

Yeah it's a common curse when trying to document a complex system that you're very familiar with.

pythoner6•5/14/23, 2:43 AM

Yeah my plan is to try to avoid touching things in /etc at all where possible

AArcitec Yeah, thanks for finding that, it's very useful to know. That changes how I'll m...

akdev•5/14/23, 2:43 AM

it'd be cool to have a script that does this and notifies

ArcitecOP•5/14/23, 2:43 AM

Yep, one thing I always do is edit the wireplumber config to add a script that disables audio device sleeping, to avoid that silent "popping" whenever audio begins playing again. I am doing it via some big edits.

I've been meaning to look for a way to install that in a more granular way that doesn't need maintenance constantly whenever wireplumber changes config.

Aakdev it'd be cool to have a script that does this and notifies

ArcitecOP•5/14/23, 2:44 AM

Yeah good idea, some kinda GUI that runs as autostart on login and pops up if there's changes.

dreams

pythoner6•5/14/23, 2:44 AM

Yeah I think having some kind of automated check that can keep tabs on the diff to prevent drift would be interesting

akdev•5/14/23, 2:44 AM

you don't really need a gui, just run the command if there are changes send a dbus notification

pythoner6•5/14/23, 2:44 AM

A bit complicated because there's a bunch of things that are probably going to be changed just by installing though, like /etc/passwd

Aakdev you don't really need a gui, just run the command if there are changes send a db...

ArcitecOP•5/14/23, 2:44 AM

Great idea

Ppythoner6 A bit complicated because there's a bunch of things that are probably going to b...

akdev•5/14/23, 2:45 AM

I wish anaconda put this stuff in /usr/etc before commiting the deployment to disk...

pythoner6•5/14/23, 2:46 AM

Eh but then should you be ending up with a new ostree deployment every time you change your password?

ArcitecOP•5/14/23, 2:47 AM

I don't think it compares those files, like

shadow

shadow

and

passwd

passwd

. That would be bad.

pythoner6•5/14/23, 2:48 AM

$ sudo ostree admin config-diff
M    adjtime
M    group
M    passwd
M    gshadow
M    shadow
M    machine-id
M    subgid
M    subuid
...

$ sudo ostree admin config-diff
M    adjtime
M    group
M    passwd
M    gshadow
M    shadow
M    machine-id
M    subgid
M    subuid
...

ArcitecOP•5/14/23, 2:48 AM

W!!@ADD

ArcitecOP•5/14/23, 2:48 AM

That's dumb.

akdev•5/14/23, 2:48 AM

yeah they do appear as they are not in /usr/etc

Ppythoner6 Eh but then should you be ending up with a new ostree deployment every time you ...

akdev•5/14/23, 2:48 AM

I'd be okay with that but there are some implications ...

akdev•5/14/23, 2:49 AM

but I would like it if

rm -rf /etc/*

rm -rf /etc/*

does not destroy the system

akdev•5/14/23, 2:49 AM

conceptually that should reset the system to default state

Aakdev I'd be okay with that but there are some implications ...

pythoner6•5/14/23, 2:50 AM

Yeah, in a future where things are more image based "native" I could see something along those lines being the case