Other Registries

If we hosted our images anywhere other than GHCR, we could push and sign in PRs
40 Replies
bsherman
bsherman2mo ago
talk to me about this i feel dense, and i've tried to understand more deeply this issue, but i still feel somewhat ignorant
p5
p52mo ago
So...
Currently our actions use a single GITHUB_TOKEN secret (abstracted from us) which provides read/write permissions to all registries in that GitHub repo. Any PR has access to this GITHUB_TOKEN, so could in theory overwrite the "production" tags (user-facing tags - i.e. ":latest"). If we had two Quay (or other provider) registries, we can store credentials for each in GitHub Actions. One "production" registry with creds only accessible from the main branch in GitHub, and another "sandbox" registry with creds accessible in PRs. So a build in a PR will use the sandbox credentials, and the exact same workflow running from the main branch would have access to the prod registry credentials. There's no way for an image built and pushed from a PR would end up in the production registry. We could even sign with a different key The only requirement for this to work is separate credentials for each registry
bsherman
bsherman2mo ago
thank you for a good explanation. it seems this 100% is tied to the fact that the default GITHUB_TOKEN is used for everything in github actions simple to understand when you explain it like this 😄
j0rge
j0rge2mo ago
I'd be down to use/push to quay.io - ideally later because there could also be people who have bad github connectivity.
Noel
Noel2mo ago
I am going to implement this in my own custom image. As a proof of concept. I already push and sign PR images anyway because my use case is so simple and I have no contributors on it.
p5
p52mo ago
All we'd need to switch over is a systemd unit that runs the rebase command I think. Add it in the final image pushed to GHCR and remove it in the first image pushed to Quay
Noel
Noel2mo ago
as far as removing our key?
p5
p52mo ago
I think our key can stay. I was meaning to automate rpm-ostree rebase to the new quay registry
Noel
Noel2mo ago
Also @j0rge I think we should be on more than just ghcr and quay. I think we should be on the docker registry too.
j0rge
j0rge2mo ago
yeah but that one is more complicated
Noel
Noel2mo ago
the docker registry? how so?
j0rge
j0rge2mo ago
ratelimiting etc.
Noel
Noel2mo ago
oh, would we have to pay or something?
j0rge
j0rge2mo ago
like we have to apply to be exempt, it's paperwork agree we should tho
Noel
Noel2mo ago
helps avoid a single point of failure. though if Github goes down, we are screwed anyway lol
p5
p52mo ago
I want a proxy infront of them all Might experiment tonight
Noel
Noel2mo ago
actually. that is a really good idea. we could do a hosted proxy through cloudflare or something.
p5
p52mo ago
I'd love to say it was my own, but it's been mentioned before
j0rge
j0rge2mo ago
we investigated this earlier in the project but nacked it
Noel
Noel2mo ago
if we had a proxy, folks could pull down from whatever registry is closest to them
j0rge
j0rge2mo ago
(it's why ublue.it exists)
Noel
Noel2mo ago
yeah, I don't think it is a big enough problem, ghcr has served us well enough. I do think having a separate sandbox registry would solve our problem though for PRs.
M2
M22mo ago
Would also be nice to show people you could point to your own caching registry if you want to only pull from the registries to one local server
bsherman
bsherman4w ago
the gitea discussion in #🦈bluefin got me wondering about this thread again... so... we need distinct credentials... What about this idea? Create a PAT which only allows publishing images to GHCR on a distinct Github org (eg, https://github.com/ublue-os-test/) and provide said PAT to our workflows, then our workflows can publish all PRs to that location. hmm... it works, I guess, but doesn't protect from a bad actor changing the workflow in PR to publish to our main repo
p5
p54w ago
Would kinda need to be the other way around - move the "production" registry to another org. We can restrict access to secrets based on the branches, but not restrict usage of GITHUB_TOKEN
bsherman
bsherman4w ago
exactly i wonder if we can remove the "publish to registry" provilege from the GITHUB_TOKEN default permissions... then we can use a PAT by environment for both PRODUCTION (ublue-os) and STAGING (ublue-os-test)
p5
p54w ago
GITHUB_TOKEN permissions are not able to be limited like that. You can set the default permissions for each repo / the entire org to read-only, but can't prevent someone adding permissions:\n packages: write in the workflow
bsherman
bsherman4w ago
i think we can hmm
p5
p54w ago
YES! That would work!
No description
bsherman
bsherman4w ago
I DO think we probably need to implement an org management tool to enforce settings/permissions before trying to implement something like this... seems easy to miss something manually
p5
p54w ago
No API for this setting when I checked 2 months ago 😦
bsherman
bsherman4w ago
@j0rge forgive me for asking probably the 10th time... what's the latest product you think we should try for org management? i want to create an issue in project plan
j0rge
j0rge4w ago
Minder I thought I disconnected from it so you could manage it?
bsherman
bsherman4w ago
you did... but I lose track of things quickly without a tracker 😄 https://github.com/stacklok/minder
j0rge
j0rge4w ago
You want their cloud service
bsherman
bsherman4w ago
draft item in project
j0rge
j0rge4w ago
I'll be home soon and whack in some details
bsherman
bsherman4w ago
Goal for doing environments to enable pushing images on PRs https://github.com/orgs/ublue-os/projects/1/views/1?pane=issue&itemId=61241610 @Noel @Kyle Gospo @EyeCantCU