BlueBuild•7mo ago

Build Cache for GitLab.com

I am currently using BlueBuild with a GitHub Action that utilizes the GitHub Actions (GHA) build cache. This feature significantly improves my build times, and I’m very pleased with the results. However, I am considering migrating my project to GitLab.com and have noticed that there doesn't seem to be a similar caching feature available on that platform. I believe it might be possible to implement a caching solution akin to the GHA cache using "Inline cache" or "Registry cache" for GitLab.com. Could anyone provide insights on whether such features are available or if there are plans to implement them in the future? Thank you for your assistance!

Solution:

Patch is out, image is building now https://github.com/blue-build/cli/actions/runs/15378226385

GitHub

chore: Release · blue-build/cli@054c167

BlueBuild's command line program that builds custom Fedora Atomic images based on your recipe.yml - chore: Release · blue-build/cli@054c167

Jump to solution

20 Replies

Luke Skywunker•7mo ago

That would be something that I would have to code up. I believe we've already got an issue for it.

Robert•5mo ago

With our project https://gitlab.com/eu-os/workspace-images/eu-os-base-demo we would also benefit from this caching. Buildah got recently the options --layers --cache-from … --cache-to. Man here: https://manpages.ubuntu.com/manpages/plucky/man1/buildah-build.1.html With Gitlab CI, this could look like:

podman build -t container-name:$VERSION --layers --cache-to $CI_REGISTRY_IMAGE/cache-image --cache-from $CI_REGISTRY_IMAGE/cache-image -f Containerfile .

The cache-image could possibly made configurable. @Luke Skywunker I did not find the issue in the cli repo. If you tell me where it is, I can also add this info to the issue (and track the issue myself). 🙂

GitLab

EU OS / EU OS Workspace Images / EU OS Base Image Demo · GitLab

GitLab.com

Luke Skywunker•5mo ago

It looks like we don't have one. I can try to remember to make one later, or you can write it up

Luci • 🍄•5mo ago

Could you share this cache stuff? I'd love to use it myself.

Robert•5mo ago

I have created this issue here: https://github.com/blue-build/cli/issues/410

GitHub

feat: support podman/buildah image cache to reach feature parity on...

Hi, following the discussion on Discord, I propose to add to blue-build options to use image caching also on other platforms than Github. One way to do this would be to expose existing caching feat...

Luke Skywunker•5mo ago

Cool, thanks for that!

ironOP•5mo ago

I've been using a caching strategy similar to this example from the GitLab documentation in other projects, and it works great:

docker buildx build --push -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA \
  --cache-to type=registry,ref=$CI_REGISTRY_IMAGE/cache-image,mode=max \
  --cache-from type=registry,ref=$CI_REGISTRY_IMAGE/cache-image .

docker buildx build --push -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA \
  --cache-to type=registry,ref=$CI_REGISTRY_IMAGE/cache-image,mode=max \
  --cache-from type=registry,ref=$CI_REGISTRY_IMAGE/cache-image .

I think we should follow a similar approach to BB_BUILDKIT_CACHE_GHA to support this caching mechanism by adding another if statement in the build_tag_push function in docker_drivers.rs. What do you all think?

Make Docker-in-Docker builds faster with Docker layer caching | Git...

GitLab product documentation.

Luke Skywunker•5mo ago

Yeah that's the plan. Just haven't gotten to it

ironOP•4mo ago

Great news! It works perfectly now! For anyone facing similar issues, @gmpinder has resolved issue https://github.com/blue-build/cli/issues/410. Starting from version 0.9.12, you can enable registry caching by setting the BB_CACHE_LAYERS environment variable. Thank you, @gmpinder, for your excellent work!

GitHub

Issues · blue-build/cli

BlueBuild's command line program that builds custom Fedora Atomic images based on your recipe.yml - Issues · blue-build/cli

Luke Skywunker•4mo ago

The GitHub action should also include the change. The use_cache option will turn this on You can also use the --cache-layers arg

Robert•4mo ago

Dear @Luke Skywunker , what value does BB_CACHE_LAYERStake? I tried with


    BB_CACHE_LAYERS: true

in my variables dictionary in the gitlab-ci yaml file, but I get now the error: "invalid reference format"

Luke Skywunker•4mo ago

It needs to be a string So "true" Gitlab variables in the yaml only except numbers and strings

Robert•4mo ago

Then it must be another reason. I still get the same error: https://gitlab.com/eu-os/workspace-images/eu-os-base-demo/-/jobs/10217540472

GitLab

build_image: [recipe.yml] (#10217540472) · Jobs · EU OS / EU OS W...

GitLab.com

Luke Skywunker•4mo ago

Oh am I not replacing / with _ for tags? Rookie mistake I can get a fix out for that

Robert•4mo ago

I do not follow. For which string such replacement would be due? Can I set this manually while waiting for a fix? Our hackathon starts in 13h 🙈

Luke Skywunker•4mo ago

The branch name It will get used for generating tags And / is not allowed in a tag

Robert•4mo ago

Luke Skywunker•4mo ago

Yeah I've just gotta add some string replacement logic That's gonna be 4 patches this weekend lol Whatever, gotta make it better

Solution

Luke Skywunker•4mo ago

Patch is out, image is building now https://github.com/blue-build/cli/actions/runs/15378226385

GitHub

chore: Release · blue-build/cli@054c167

BlueBuild's command line program that builds custom Fedora Atomic images based on your recipe.yml - chore: Release · blue-build/cli@054c167

Luke Skywunker•4mo ago

@Robert you should be good now btw Good luck on your hackathon!

Gaming

Programming

Build Cache for GitLab.com

Did you find this page helpful?