Other Registries

If we hosted our images anywhere other than GHCR, we could push and sign in PRs

bsherman•23d ago

talk to me about this i feel dense, and i've tried to understand more deeply this issue, but i still feel somewhat ignorant

p5•23d ago

So...
Currently our actions use a single GITHUB_TOKEN secret (abstracted from us) which provides read/write permissions to all registries in that GitHub repo. Any PR has access to this GITHUB_TOKEN, so could in theory overwrite the "production" tags (user-facing tags - i.e. ":latest"). If we had two Quay (or other provider) registries, we can store credentials for each in GitHub Actions. One "production" registry with creds only accessible from the main branch in GitHub, and another "sandbox" registry with creds accessible in PRs. So a build in a PR will use the sandbox credentials, and the exact same workflow running from the main branch would have access to the prod registry credentials. There's no way for an image built and pushed from a PR would end up in the production registry. We could even sign with a different key The only requirement for this to work is separate credentials for each registry

bsherman•23d ago

thank you for a good explanation. it seems this 100% is tied to the fact that the default GITHUB_TOKEN is used for everything in github actions simple to understand when you explain it like this 😄

j0rge•23d ago

I'd be down to use/push to quay.io - ideally later because there could also be people who have bad github connectivity.

Noel•23d ago

I am going to implement this in my own custom image. As a proof of concept. I already push and sign PR images anyway because my use case is so simple and I have no contributors on it.

p5•23d ago

All we'd need to switch over is a systemd unit that runs the rebase command I think. Add it in the final image pushed to GHCR and remove it in the first image pushed to Quay

Noel•23d ago

as far as removing our key?

p5•23d ago

I think our key can stay. I was meaning to automate rpm-ostree rebase to the new quay registry

Noel•23d ago

Also @j0rge I think we should be on more than just ghcr and quay. I think we should be on the docker registry too.

j0rge•23d ago

yeah but that one is more complicated

Noel•23d ago

the docker registry? how so?

j0rge•23d ago

ratelimiting etc.

Noel•23d ago

oh, would we have to pay or something?

j0rge•23d ago

like we have to apply to be exempt, it's paperwork agree we should tho

Noel•23d ago

helps avoid a single point of failure. though if Github goes down, we are screwed anyway lol

p5•23d ago

I want a proxy infront of them all Might experiment tonight

Noel•23d ago

actually. that is a really good idea. we could do a hosted proxy through cloudflare or something.

p5•23d ago

I'd love to say it was my own, but it's been mentioned before

j0rge•23d ago

we investigated this earlier in the project but nacked it

Noel•23d ago

if we had a proxy, folks could pull down from whatever registry is closest to them

j0rge•23d ago

(it's why ublue.it exists)

Noel•23d ago

yeah, I don't think it is a big enough problem, ghcr has served us well enough. I do think having a separate sandbox registry would solve our problem though for PRs.

M2•23d ago

Would also be nice to show people you could point to your own caching registry if you want to only pull from the registries to one local server

bsherman•17d ago

the gitea discussion in #🦈bluefin got me wondering about this thread again... so... we need distinct credentials... What about this idea? Create a PAT which only allows publishing images to GHCR on a distinct Github org (eg, https://github.com/ublue-os-test/) and provide said PAT to our workflows, then our workflows can publish all PRs to that location. hmm... it works, I guess, but doesn't protect from a bad actor changing the workflow in PR to publish to our main repo

p5•17d ago

Would kinda need to be the other way around - move the "production" registry to another org. We can restrict access to secrets based on the branches, but not restrict usage of GITHUB_TOKEN

bsherman•17d ago

exactly i wonder if we can remove the "publish to registry" provilege from the GITHUB_TOKEN default permissions... then we can use a PAT by environment for both PRODUCTION (ublue-os) and STAGING (ublue-os-test)

p5•17d ago

GITHUB_TOKEN permissions are not able to be limited like that. You can set the default permissions for each repo / the entire org to read-only, but can't prevent someone adding permissions:\n packages: write in the workflow

bsherman•17d ago

i think we can hmm

bsherman•17d ago

https://docs.github.com/en/packages/learn-github-packages/configuring-a-packages-access-control-and-visibility#disabling-automatic-inheritance-of-access-permissions-in-an-organization

GitHub Docs

Configuring a package's access control and visibility - GitHub Docs

p5•17d ago

YES! That would work!

bsherman•17d ago

https://tenor.com/bLPo0.gif

Tenor

bsherman•17d ago

I DO think we probably need to implement an org management tool to enforce settings/permissions before trying to implement something like this... seems easy to miss something manually

p5•17d ago

No API for this setting when I checked 2 months ago 😦

bsherman•17d ago

@j0rge forgive me for asking probably the 10th time... what's the latest product you think we should try for org management? i want to create an issue in project plan

j0rge•17d ago

Minder I thought I disconnected from it so you could manage it?

bsherman•17d ago

you did... but I lose track of things quickly without a tracker 😄 https://github.com/stacklok/minder

j0rge•17d ago

You want their cloud service

bsherman•17d ago

draft item in project

j0rge•17d ago

I'll be home soon and whack in some details

bsherman•17d ago

Goal for doing environments to enable pushing images on PRs https://github.com/orgs/ublue-os/projects/1/views/1?pane=issue&itemId=61241610 @Noel @Kyle Gospo @EyeCantCU