ublue HWE (hardware enablement) image builds

Original message was deleted

bsherman•3/20/24, 6:33 PM

For a while now, I've been looking at how we have a LOT of hardware enablement images with distinct repos, workflow, etc... and it's a serious pain to maintain. I've got a goal to clean it up.

This is something I've chatted about with various members of the team, but wanted to get down in a thread.

bsherman•3/20/24, 6:45 PM

So just a quick sketch of the plan after I've looked at various methods we use for matrix and workflow management in bluefin and ucore...

I will create a repo hwehwe which will be the new, consolidated home for all our images which are built to explicitly enable hardware. This will effectively merge together the following repos:

asus
asus-nvidia
framework
nvidia
surface
surface-nvidia

This will greatly simplify the maintenance burden by having a single repo to modify when we need to fix something related to workflow or container image builds for hardware enablement.

The 6 repos themselves will be archived after the final migration.

bsherman•3/20/24, 6:46 PM

@j0rge @Kyle Gospo @EyeCantCU @Noel @Robert (p5)

I'm sure I should ping others, but this is a start.

bsherman•3/20/24, 6:49 PM

The distinction between

main

main

repo's product and hwehwe is the hardware enablement concept...

main

main

images are generic, no kmods added (once we drop F38) ... while the hwehwe images may include kmods (eg, nvidia) or custom kernels, plus other details specific to the hardware.

EyeCantCU•3/20/24, 6:55 PM

I think having everything under a single HWE repo makes a lot of sense. The one thing I'd be concerned about is the number of concurrent builds we'd have to support

Something else also comes to mind... Kernel fsync supports like literally everything out of the box. Could be worth maintaining just fsync images?

EyeCantCU•3/20/24, 6:56 PM

Hmm... I guess what makes that slightly more challenging is Surface still needs libwacom-surface and Asus asusctl...

Kyle Gospo•3/20/24, 6:57 PM

we could probably break it up by hardware layer or by image

Kyle Gospo•3/20/24, 6:57 PM

so it builds in stages

EyeCantCU•3/20/24, 6:57 PM

That would work!

EEyeCantCU I think having everything under a single HWE repo makes a lot of sense. The one ...

bsherman•3/20/24, 7:07 PM

we can manage the concurrent builder issue

KKyle Gospo so it builds in stages

bsherman•3/20/24, 7:08 PM

yes, exactly

EEyeCantCU I think having everything under a single HWE repo makes a lot of sense. The one ...

bsherman•3/20/24, 7:09 PM

regarding "just using fsync" that's scope creep

my mission is maintainability... once we can build stuff and make a change by only touching 4 repos, not 9 or 10... then we can see about that

Bbsherman we can manage the concurrent builder issue

j0rge•3/20/24, 7:12 PM

Yeah and I've been talking to github, assign that problem to me, ya'll handle the tech.

Jj0rge Yeah and I've been talking to github, assign that problem to me, ya'll handle th...

bsherman•3/20/24, 7:13 PM

yeah, i'll let you know when/if we hit a problem on it

EEyeCantCU I think having everything under a single HWE repo makes a lot of sense. The one ...

fiftydinar•3/20/24, 7:18 PM

The issue with fsync is that some unexpected issues can arise, like what @bsherman experienced with audio crackling.

But it is indeed more convenient to have everything in a kernel & in a single image, than to maintain multiple ones.

Kyle Gospo•3/20/24, 7:18 PM

fsync is a hard no from me, it's nice but it's not upstream

Kyle Gospo•3/20/24, 7:18 PM

good for bazzite

bsherman•3/20/24, 7:21 PM

yeah, really, just want to keep this thread about repo and build workflow consolidation... don't want to grow the scope any more

Robert•3/20/24, 7:23 PM

So will it be the same structure as Bluefin - a packages.json file containing the different packages to install for each image, and a separate directory for config files to copy over to each?

|--asus/
|--|--usr/
|--surface/
|--|--usr/
|--Containerfile
|--packages.json

|--asus/
|--|--usr/
|--surface/
|--|--usr/
|--Containerfile
|--packages.json

RRobert So will it be the same structure as Bluefin - a packages.json file containing th...

bsherman•3/20/24, 7:28 PM

i've still got to look at some details per repo.

right now, only surfacesurface has a packages.json, the rest don't

so, I'll just do my best to merge it all together in a sane way

bsherman•3/20/24, 7:29 PM

i do plan to keep nvidianvidia 's history as the authoritative git history ... as it's had the longest life

Bbsherman regarding "just using fsync" that's scope creep 🙂 my mission is maintainabilit...

EyeCantCU•3/20/24, 7:44 PM

Asahi is about to give me a huge dose of scope creep

But yeah, no. You and Kyle are correct

Noel•3/20/24, 10:55 PM

Do we have an issue open for this? We should probably have one so we can track it.

NNoel Do we have an issue open for this? We should probably have one so we can track i...

bsherman•3/20/24, 11:02 PM

goodcall

NNoel Do we have an issue open for this? We should probably have one so we can track i...

bsherman•3/23/24, 6:41 PM

created the issue https://github.com/ublue-os/nvidia/issues/206

sorry for confusion

Bbsherman created the issue <https://github.com/ublue-os/nvidia/issues/206> sorry for co...

Noel•3/23/24, 6:42 PM

no worries, probably should've confirmed

At least we have one.

Robert•3/24/24, 12:18 AM

Would it be easier to manage if we keep Nvidia separate? I can see us wanting surface-nvidia, framework-nvidia etc, so keeping Nvidia as another "layer" should make it easier to create the pipelines and dependencies for these.

We're not going to have a surface-framework, so there's no dependencies between these and can therefore be built concurrently, but we will have dependencies on the Nvidia images for all others

j0rge•3/24/24, 12:22 AM

yeah I suspect that's what will happen, but bsherman is sick so he wanted to grab some shuteye

j0rge•3/24/24, 12:23 AM

framework can go away entirely, it's basically all upstream now other than the kargs

j0rge•3/25/24, 2:43 PM

@Kyle Gospo so if we do a kmod for framework that removes the need for framework images. Could we do the same for asus?

bsherman•3/25/24, 3:03 PM

What kmod are we talking about for framework?

j0rge•3/25/24, 3:04 PM

he was talking for flipping the boot paramater

j0rge•3/25/24, 3:04 PM

so the user wouldn't need to add the karg themselves, dunno, this part of the stack is a magic box for me

Bbsherman What kmod are we talking about for framework?

Kyle Gospo•3/25/24, 3:44 PM

There's a driver we have to unload to get parts of framework working

Kyle Gospo•3/25/24, 3:44 PM

The idea is to just patch that so it detects the framework DMI and never loads in the first place

Kyle Gospo•3/25/24, 3:45 PM

That way that hardware doesn't need a device layer

bsherman•3/25/24, 3:45 PM

so... you could build that as a kmod instead of using asus-kernelasus-kernel?

bsherman•3/25/24, 3:45 PM

and the custom asus-firmware?

Kyle Gospo•3/25/24, 3:46 PM

Separate from asus, this is framework specific. We never shipped a kernel for them, just a whole bunch of Kernel arguments

Kyle Gospo•3/25/24, 3:46 PM

Which can simply be implemented better

bsherman•3/25/24, 3:46 PM

sorry, i switched gears in my brain, oops

bsherman•3/27/24, 1:20 AM

@j0rge i just want to get clear on if there's stuff we DON'T need me to merge ... i was planning to pull in asus, asus-nvidia, framework, surface, surface-nvidia

but there's some chatter making me think we don't need framework?

j0rge•3/27/24, 1:21 AM

I marked framework as deprecated last month

j0rge•3/27/24, 1:21 AM

Kyle is going to enable it in bazzifin directly via that akmod

bsherman•3/27/24, 1:21 AM

well, this is a good time to be done with it maybe then

j0rge•3/27/24, 1:21 AM

let me check the framework thread

bsherman•3/27/24, 1:21 AM

i can just leave it alone until we are ready for archival... but don't need to bother about merging it here

j0rge•3/27/24, 1:22 AM

actually you know what, let's leave framework alone for now. I need to make an announcement in the thread that at F40 people will need to rebase and I don't wanna leave them hanging without an OS

Jj0rge actually you know what, let's leave framework alone for now. I need to make an a...

bsherman•3/27/24, 1:23 AM

right, i just won't touch it for the moment... i'll merge other stuff and we can see where that leaves us

j0rge•3/27/24, 1:23 AM

nod

ublue HWE (hardware enablement) image builds

Similar Threads

Similar Threads

Similar Threads