Cloudflare Developers•3y ago

how can i get started with build caching?

Bbigansh how can i get started with build caching?

Cyb3r-Jak3•9/27/23, 11:44 AM

Can fill out the form on build-caching-beta

CCyb3r-Jak3 Can fill out the form on https://discord.com/channels/595317990191398933/1144554...

biganshOP•9/27/23, 12:26 PM

done

Jacob•9/27/23, 6:06 PM

Heya! Y'all should have access to it now

You can enable it in project settings -> Builds & deployments -> Build caching

JJacob Heya! Y'all should have access to it now 🙂 You can enable it in project setting...

biganshOP•9/27/23, 8:29 PM

awesome. thank you

Jacob•9/28/23, 1:24 PM

It's live! https://blog.cloudflare.com/race-ahead-with-build-caching/

The Cloudflare Blog

Race ahead with Cloudflare Pages build caching

Unleash the fast & furious in your builds with Cloudflare Pages' build caching. Build cache is a feature designed to reduce your build times by caching and reusing previously computed project components. Now in Beta for select frameworks and package managers.

Vero•9/28/23, 1:45 PM

^ @945732495889014814

VVero ^ <@&945732495889014814>

Luke•9/28/23, 1:47 PM

Join us on our Discord Server [link]

The blog post needs updating with the link

LLuke > Join us on our Discord Server [link] The blog post needs updating with the lin...

Vero•9/28/23, 1:47 PM

thanks, raising

GGlare Is Nuxt already included or is that in the "to come"?

Cyb3r-Jak3•9/28/23, 1:47 PM

Only lists nextjs, gatsby and Astro to start.

Jacob•9/28/23, 3:12 PM

We also cache dependencies for Yarn 1, npm, pnpm, and bun!

JJacob It's live! https://blog.cloudflare.com/race-ahead-with-build-caching/

Avi•9/29/23, 4:50 AM

hi, i'm not seeing where it's possible to enable this...

Avi•9/29/23, 4:51 AM

worth noting that, because we need to integrate our pages deployments into our CI process, we deploy to pages manually via

wrangler

wrangler

Avi•9/29/23, 4:52 AM

i didn't see anything in the announcement about this being unavailable for that setup

AAvi worth noting that, because we need to integrate our pages deployments into our C...

Hello, I’m Allie!•9/29/23, 6:57 AM

Build caching only applies when using Pages Builds

HHello, I’m Allie!Build caching only applies when using Pages Builds

Avi•9/29/23, 6:58 AM

can you clarify exactly what that means? this is a pages project

AAvi can you clarify exactly what that means? this is a pages project

Hello, I’m Allie!•9/29/23, 6:58 AM

Yes, but you are using a Direct Upload, not the Pages Build-from-Git system

Hello, I’m Allie!•9/29/23, 6:59 AM

It only applies when you don’t build it yourself

Avi•9/29/23, 6:59 AM

ahh, Pages Builds = build from Git?

Hello, I’m Allie!•9/29/23, 6:59 AM

Or your CI does

Avi•9/29/23, 6:59 AM

i see

AAvi ahh, Pages Builds = build from Git?

Hello, I’m Allie!•9/29/23, 6:59 AM

Yes

Avi•9/29/23, 6:59 AM

that’s a bummer

AAvi that’s a bummer

Hello, I’m Allie!•9/29/23, 6:59 AM

It wouldn’t provide any benefit anyway, even if it was added tho

Avi•9/29/23, 6:59 AM

although i guess i could set up equivalent caching in my ci

AAvi although i guess i could set up equivalent caching in my ci

Hello, I’m Allie!•9/29/23, 6:59 AM

Probably the way to go

Avi•9/29/23, 6:59 AM

word. thanks!

Andrew Ferk•9/29/23, 6:55 PM

We just upgraded from yarn 1 to yarn 3

Does this mean we don't get the caching benefits of the package installer, given docs only say it supports yarn 1?

AAndrew Ferk We just upgraded from yarn 1 to yarn 3 😢 Does this mean we don't get the cachi...

JohnDotAwesome•10/1/23, 6:40 PM

We found that the guidance for caching yarn 3 was to commit the cache files to your repo https://yarnpkg.com/features/caching

yarn 3 caching has always been supported in a way

Cache strategies | Yarn

How to ensure your installs perform the absolute minimal amount of work.

AAndrew Ferk We just upgraded from yarn 1 to yarn 3 😢 Does this mean we don't get the cachi...

JohnDotAwesome•10/1/23, 6:41 PM

That being said, if you find something global and related to yarn3 that should be cached, please let us know

JJohnDotAwesome We found that the guidance for caching yarn 3 was to commit the cache files to y...

Andrew Ferk•10/2/23, 4:51 PM

This is something we were discussing last week. We currently are using the default yarn config, which is to do global caching. With the default yarn 3 setup, we are seeing the fetch and link step doing a lot of work. I don't think all of our stuff can be cached with yarn's capabilities, because some of our packages require machine/architecture dependencies to build. I assume then caching would need to be solved by the build machine.

You can see the fetch and link step below, when doing back-to-back builds of the same commit. These should essentially be noop with build caching.

11:28:27.392    ➤ YN0000: ┌ Fetch step
11:29:42.693    ➤ YN0013: │ 12 packages were already cached, 2585 had to be fetched
11:29:42.721    ➤ YN0000: └ Completed in 1m 16s
11:29:42.898    ➤ YN0000: ┌ Link step
11:31:08.063    ➤ YN0007: │ sharp@npm:0.30.7 must be built because it never has been before or the last one failed
... more ...
11:31:23.386    ➤ YN0000: └ Completed in 1m 41s

11:28:27.392    ➤ YN0000: ┌ Fetch step
11:29:42.693    ➤ YN0013: │ 12 packages were already cached, 2585 had to be fetched
11:29:42.721    ➤ YN0000: └ Completed in 1m 16s
11:29:42.898    ➤ YN0000: ┌ Link step
11:31:08.063    ➤ YN0007: │ sharp@npm:0.30.7 must be built because it never has been before or the last one failed
... more ...
11:31:23.386    ➤ YN0000: └ Completed in 1m 41s

Andrew Ferk•10/2/23, 5:14 PM

Specifically, I think

.yarn/cache

.yarn/cache

needs to be cached for the "Fetch step" and

.yarn/install-state.gz

.yarn/install-state.gz

needs to be cached for the "Link step".

reitznerdomenik•10/3/23, 8:14 AM

Hi, is there any plan to extend the build caching to nuxt3 as well?
Or is there a way to hack around it and use those predefined folders for caching?

Georges•10/3/23, 9:02 AM

I was wondering if you would consider adding the

.jampack/cache

.jampack/cache

used by jampack. A user has been asking for it https://github.com/divriots/jampack/issues/106

Georges•10/3/23, 9:04 AM

jampack

jampack

is too small for you to consider it, could you add a generic folder like

".build_cache"

".build_cache"

or something so projects don't need to highjack existing folders to work around it?

Aadaptive Was the beta suspended? I can't see it on the dashboard.

James•10/3/23, 4:46 PM

If you're using v2 build image (system), it should just be an option in the UI: https://i.james.pub/file/2023/10/6dbcb864-d5f0-4f77-b03a-ad7b1ffe1a3f.png

James•10/3/23, 4:47 PM

Actually I think the UI is always present, but it only works in v2 build image afaik

Cretezy•10/3/23, 8:48 PM

Is there a way to customize the cached paths?
We have Next.js under a sub-directory, so

.next/cache

.next/cache

doesn't work. We don't want to update the root directory option (since we have

functions

functions

and

web

web

/Next.js as top-level directories in our repo)

James•10/4/23, 12:16 AM

Not at this time, as far as I now

Cretezy•10/4/23, 5:01 AM

Okay, thank you. That seems like an important feature

groenroos•10/4/23, 12:53 PM

Turning on build cache makes the build fail with

Error: Output directory "build" not found.

Error: Output directory "build" not found.

is there something I can do to avoid this?

Erisa•10/4/23, 1:00 PM

And if you turn cache off and try the same build again, it works?

Isaac McFadyen•10/4/23, 6:07 PM

Cross-posting from next-on-pages: interesting suggestion around possibly having some sort of config file to configure caching behavior or what folders are cached? https://canary.discord.com/channels/595317990191398933/1135899572141568050/1159189844069715998

JohnDotAwesome•10/4/23, 7:14 PM

Configurable cache directories is definitely something on our mind. We actually had them at the beginning but took them out because we wanted to be extra sure/deliberate about how we were exposing that config to users. Expect configurable cache in the future

EErisa And if you turn cache off and try the same build again, it works?

groenroos•10/9/23, 12:24 PM

Correct - also the fail only happened on the second build after turning it on, and then consistently after that until turning it off

Erisa•10/9/23, 12:24 PM

?pages-deployment-id

Flare•10/9/23, 12:24 PM

The Pages deployment ID is a unique build identifier.
It's the UUID in the browser bar (for example, a URL would be

dash.cloudflare.com/ACCOUNT_ID/pages/view/PROJECT/DEPLOYMENT_ID

dash.cloudflare.com/ACCOUNT_ID/pages/view/PROJECT/DEPLOYMENT_ID

where the deployment ID looks something like a398d794-7322-4c97-96d9-40b5140a8d9b).
This ID can help troubleshoot some issues with Pages builds so if you have a failing build make sure you grab that ID for the Pages team to use.

11:28:27.392 ➤ YN0000: ┌ Fetch step 11:29:42.693 ➤ YN0013: │ 12 packages were already cached, 2585 had to be fetched 11:29:42.721 ➤ YN0000: └ Completed in 1m 16s 11:29:42.898 ➤ YN0000: ┌ Link step 11:31:08.063 ➤ YN0007: │ sharp@npm:0.30.7 must be built because it never has been before or the last one failed ... more ... 11:31:23.386 ➤ YN0000: └ Completed in 1m 41s

how can i get started with build caching?

Similar Threads

Similar Threads

Similar Threads