jstone 7474 I chatted with Jorge the
@.jstone I chatted with Jorge the other night about working on improving build scheduling, triggering downstream builds, etc
well, i haven't got too far, but i did learn something possibly helpful for that and/or other github action workflow...
I know your build workflows have several cases of using step output in subsequent steps, but I was trying to use job output to gate and/or provide context to the next job. Really the idea is to have a single job that can decide if things should proceed before spawning the matrix jobs.
Here's what I came up with: https://github.com/bsherman/ucore-kmods/blob/main/.github/workflows/build.yml#L19-L51
You may know that trick, but it took me a while to sort it out so I wanted to share.
14 Replies
the key items are defining job outputs, which the
>> $GITHUB_OUTPUT populates
and using needs: jobId in the gated jobThe tricky part about akmods isn't necessarily whether to rebuild whenever an akmod is updated. It's really whether to know whether to build on a kernel update due to how frequently the kernel package changes on a given day. What we could do is have a ublue-os/akmods image that has builds triggered whenever ublue-os/main gets updated, which will generate akmod RPMs as part of a buildcache as well as having an installer script for downstream images like ublue-os/nvidia to use. This script can check if the downstream image has a matching kernel for the akmod RPM, and if it does then install it as a "cache hit", otherwise it'll 'cache miss" by building an akmod from source
Yep. I 100% agree the main complexity is in the decision of build or not build based on kernel changes.
I didn't really mean this comment to be specific about akmods though, simply that I had a hard time finding an example of a "first" job which would gate a more complex matrix job and then, if successful, provide input to said matrix job.
Is there not a release-monitoring.org entry for akmods-nvidia?
https://release-monitoring.org/project/5454/
There's the base project and it does have a mapping for RPM Fusion
nvidia · Anitya
Monitoring project releases
I don't mean nvidia, I mean akmods-nvidia specifically
See how different the NVRs are between nvidia and akmods-nvidia https://forksystems.mm.fcix.net/rpmfusion/nonfree/fedora/releases/37/Everything/x86_64/os/repoview/akmod-nvidia-470xx.html https://opencolo.mm.fcix.net/rpmfusion/nonfree/fedora/updates/37/x86_64/repoview/xorg-x11-drv-nvidia-470xx.html
yep, i understand, i was trying to dig through the data provided by release-monitoring.org ... hopeful that since they mapped the 'nvidia' project to RPM Fusion, maybe they'd have the related data, but i don't see anything useful
Considering build.sh script can pass this along to the next build step with a vars file that can be added with
source, we can generalize this a bit with an earlier job that builds nvidia for each Fedora release and driver stream and has them cached in a builder image for later steps https://github.com/ublue-os/nvidia/blob/main/build.shGitHub
nvidia/build.sh at main · ublue-os/nvidia
Fedora variants with Nvidia drivers added. Contribute to ublue-os/nvidia development by creating an account on GitHub.
TBH I wanted to do this earlier, but rpm-ostree was quite finicky in having me cache RPMs so we'll likely still have to download a bunch of RPMs with each variant. But it's still going to be faster than the 5-7 minutes spent building akmods
Yes, definitely faster to not rebuild them.
I see the vars file you are already creating, but it's not getting used anywhere yet, correct?
It gets passed in for install.sh
ah, ok i see ... i was overlooking the relevant COPY
The containerfile is structured as build (build akmods) -> install (install RPMs) -> post-install (do anything that can't be done with rpm-ostree directly)
my turn to get back to my "real job" 🙂