well to properly optimize for wire download what would we need to do is for each package have its ow

well to properly optimize for wire download what would we need to do is for each package have its own run command which then goes into deleting cache or more properly just delete /var everytime becuase its actually problem to have it anyway

BBigpod well to properly optimize for wire download what would we need to do is for each...

pythoner6•5/18/23, 11:54 PM

The idea being that that way a single package having a difference means you only redownload the layer with just that package, rather than a layer containing 50 other unchanged packages too, I assume

Ppythoner6 The idea being that that way a single package having a difference means you only...

BigpodOP•5/18/23, 11:54 PM

yes

Arcitec•5/18/23, 11:55 PM

Yep, but there's one problem that brian and I talked about that we're not sure about:

- The RPM database is stored in every layer where a

rpm-ostree install

rpm-ostree install

command executes. So if anything in an earlier RPM layer updates the RPM cache, or file timestamps, we'd end up with new layers anyway.

- I suspect that Silverblue OCI upstream builds their layers in a smarter way with some other tools, which properly caches files separately from the RPM database?

Arcitec•5/18/23, 11:55 PM

They're also working on new tools that will automatically diff files and only generate layers for modified files (I'm getting tons of emails, they're working hard at it). Hopefully F39 or F40...

BigpodOP•5/18/23, 11:56 PM

yea they are working on some better optimizations for oci builds

pythoner6•5/18/23, 11:57 PM

https://youtu.be/QaKl5z6dFlM?t=209 (which got linked in #

general ) mentions "responsible chunked" images from ostree compose imageostree compose image, though I'm not sure exactly how that works

j0rge•5/18/23, 11:58 PM

a month ago, how did we miss this!

Arcitec•5/18/23, 11:58 PM

Awesome, that video looks like it's about all of this. Perhaps it explains how they're solving the RPM db stuff.

Arcitec•5/18/23, 11:59 PM

I think it requires a tool that's aware of the RPM db and generates layers like:

- 1 layer: files from X packages; NO RPM db
- 1 layer: files from Y packages; NO RPM db
- final layer: the merged RPM db

And only updating the separate layers if files from packages are changed.

Arcitec•5/18/23, 11:59 PM

Gonna watch that whole video but not atm, still sick.

j0rge•5/19/23, 12:00 AM

yeah drink water!

Arcitec•5/19/23, 12:00 AM

Yeah got the ice water bottle here. It helps.

j0rge•5/19/23, 12:00 AM

I know we'll be in a preso at some point because one of them PMed me asking if we'd be cool with it

pythoner6•5/19/23, 12:00 AM

Was just watching it myself, I don't think it really gets too much into the details there, the bit about compose image was just mentioned in passing. But an interesting watch regardless

j0rge•5/19/23, 12:01 AM

I have found that every video is important because there's just too much for a single team to consume so we need every single bit of info we can get

Jj0rge I know we'll be in a preso at some point because one of them PMed me asking if w...

Arcitec•5/19/23, 12:01 AM

That's so cool. I look forward to the explosion in popularity as we approach critical mass.

Jj0rge I have found that every video is important because there's just too much for a s...

Arcitec•5/19/23, 12:02 AM

Yeah. Ideally, someone from upstream gets interested enough to join us and help out with our infrastructure. So to achieve that, we need more growth. To get more growth, we need ads. To get alluring ads, we need to offer people something useful. To offer something useful we just need to keep doing what we're doing and getting better every day!

j0rge•5/19/23, 12:02 AM

I'd like to do things like fix the broken ISOs and stuff before we deep dive into this stuff though

Arcitec•5/19/23, 12:02 AM

business speech mode

Jj0rge I'd like to do things like fix the broken ISOs and stuff before we deep dive int...

Arcitec•5/19/23, 12:02 AM

Yeah I heard the ISOs don't retry during download failures for example. :S

Arcitec•5/19/23, 12:04 AM

It's just a kickstart script so I dunno how we could solve that. Checking exit code and retrying a few times?

j0rge•5/19/23, 12:04 AM

I mean this one: https://github.com/ublue-os/main/issues/132

GitHub

Plasma Discover invalid reference for OCI when installed using the ...

What happened? When using the iso to install ublue a rpm-ostree-backend error occurs mentioning invalid reference for OCI container ostree format. Relevant log output $ plasma-discover rpm-ostree-b...

j0rge•5/19/23, 12:04 AM

the installer issue is Anaconda, not us

Arcitec•5/19/23, 12:05 AM

That's strange, docker://docker://?

j0rge•5/19/23, 12:05 AM

hopefully all the info is in the ticket

Ppythoner6 I'd bet that could be what's happening based on <https://github.com/ostreedev/os...

j0rge•5/19/23, 12:06 AM

can you toss the results of your findings in the issue if you don't mind?

pythoner6•5/19/23, 12:06 AM

Sure yeah

Jj0rge hopefully all the info is in the ticket

Arcitec•5/19/23, 12:07 AM

I see, so the installer is rewriting the URL to docker://docker://. But that doesn't seem to be the actual issue anyway, they quoted docs which say it's equivalent.

Jj0rge @bsherman ok I'm back now and can work on announcement stuff

bsherman•5/19/23, 12:08 AM

do you need something from me on this? the main thing I know is missing was when I discovered this:

https://github.com/ublue-os/isogenerator/issues/64

I don't know if we want that sorted before announce or no

j0rge•5/19/23, 12:08 AM

Where do we put the keys in this repo?

j0rge•5/19/23, 12:08 AM

like is it just inline replacing the files?

j0rge•5/19/23, 12:09 AM

https://github.com/ublue-os/isogenerator/tree/main/installer/securebootkeys

GitHub

isogenerator/installer/securebootkeys at main · ublue-os/isogenerator

An action to generate custom ISOs of OCI images. Contribute to ublue-os/isogenerator development by creating an account on GitHub.

Jj0rge Where do we put the keys in this repo?

bsherman•5/19/23, 12:09 AM

no, not a replace... we need both, and to import both, I can handle it, just wondering if we want it wrapped up before announce

pythoner6•5/19/23, 12:09 AM

To my understanding: the actual issue is ostree supports both ostree-unverified-image:registry:ostree-unverified-image:registry: and ostree-unverified-image:docker://ostree-unverified-image:docker://, but discover only currently expects ostree-unverified-registry:ostree-unverified-registry:, not anything else, so ostree-unverified-image:registry:ostree-unverified-image:registry: or ostree-unverified-image:docker://ostree-unverified-image:docker:// which should both be equivalent cause it to not work properly.

Jj0rge I know we'll be in a preso at some point because one of them PMed me asking if w...

BigpodOP•5/19/23, 12:10 AM

back reading what does preso mean?

j0rge•5/19/23, 12:11 AM

presentation

Ppythoner6 To my understanding: the actual issue is ostree supports both `ostree-unverified...

j0rge•5/19/23, 12:13 AM

yeah willing to bet this is just the discover rpm-ostree work being unfinished

j0rge•5/19/23, 12:13 AM

because a) that's the only thing that makes sense

j0rge•5/19/23, 12:13 AM

and b) we know that it's in progress, heh.

j0rge•5/19/23, 12:15 AM

https://invent.kde.org/plasma/discover/-/issues

GitLab

Issues · Plasma / Discover · GitLab

KDE and Plasma resources management GUI

j0rge•5/19/23, 12:15 AM

but hasn't been reported yet

Ppythoner6 To my understanding: the actual issue is ostree supports both `ostree-unverified...

Arcitec•5/19/23, 12:18 AM

This makes total sense. Nice find.

pythoner6•5/19/23, 12:18 AM

Once I finish writing things up on 132, I can go ahead and file the issue on the discover side if you want

j0rge•5/19/23, 12:22 AM

that'd be amazing. It's our most popular image by far so we should help where we can.

Ppythoner6 https://youtu.be/QaKl5z6dFlM?t=209 (which got linked in #⭐general ) mentions "re...

bsherman•5/19/23, 12:24 AM

watching this! awesome! Colin has created starter OCI for CentOS to use with bootc AND bootc has opinionated LUKS support ... it can enable tpm2-luks out of the box! no need for my custom scripts anymore i guess

j0rge•5/19/23, 12:27 AM

ok so am I uploading a new key to this isogenerator dir?

j0rge•5/19/23, 12:28 AM

what should I call it, I'm assuming I put it alongside the existing key?

Jj0rge ok so am I uploading a new key to this isogenerator dir?

bsherman•5/19/23, 12:28 AM

i'm making a PR now

bsherman•5/19/23, 12:28 AM

we also need to modify script

well to properly optimize for wire download what would we need to do is for each package have its ow

Similar Threads

Similar Threads

Similar Threads