Cloudflare Developers•3y ago

Build caching only applies when using Pages Builds

HHello, I’m Allie!Build caching only applies when using Pages Builds

Avi•9/29/23, 6:58 AM

can you clarify exactly what that means? this is a pages project

AAvi can you clarify exactly what that means? this is a pages project

Hello, I’m Allie!OP•9/29/23, 6:58 AM

Yes, but you are using a Direct Upload, not the Pages Build-from-Git system

Hello, I’m Allie!OP•9/29/23, 6:59 AM

It only applies when you don’t build it yourself

Avi•9/29/23, 6:59 AM

ahh, Pages Builds = build from Git?

Hello, I’m Allie!OP•9/29/23, 6:59 AM

Or your CI does

Avi•9/29/23, 6:59 AM

i see

AAvi ahh, Pages Builds = build from Git?

Hello, I’m Allie!OP•9/29/23, 6:59 AM

Yes

Avi•9/29/23, 6:59 AM

that’s a bummer

AAvi that’s a bummer

Hello, I’m Allie!OP•9/29/23, 6:59 AM

It wouldn’t provide any benefit anyway, even if it was added tho

Avi•9/29/23, 6:59 AM

although i guess i could set up equivalent caching in my ci

AAvi although i guess i could set up equivalent caching in my ci

Hello, I’m Allie!OP•9/29/23, 6:59 AM

Probably the way to go

Avi•9/29/23, 6:59 AM

word. thanks!

Andrew Ferk•9/29/23, 6:55 PM

We just upgraded from yarn 1 to yarn 3

Does this mean we don't get the caching benefits of the package installer, given docs only say it supports yarn 1?

AAndrew Ferk We just upgraded from yarn 1 to yarn 3 😢 Does this mean we don't get the cachi...

JohnDotAwesome•10/1/23, 6:40 PM

We found that the guidance for caching yarn 3 was to commit the cache files to your repo https://yarnpkg.com/features/caching

yarn 3 caching has always been supported in a way

Cache strategies | Yarn

How to ensure your installs perform the absolute minimal amount of work.

AAndrew Ferk We just upgraded from yarn 1 to yarn 3 😢 Does this mean we don't get the cachi...

JohnDotAwesome•10/1/23, 6:41 PM

That being said, if you find something global and related to yarn3 that should be cached, please let us know

JJohnDotAwesome We found that the guidance for caching yarn 3 was to commit the cache files to y...

Andrew Ferk•10/2/23, 4:51 PM

This is something we were discussing last week. We currently are using the default yarn config, which is to do global caching. With the default yarn 3 setup, we are seeing the fetch and link step doing a lot of work. I don't think all of our stuff can be cached with yarn's capabilities, because some of our packages require machine/architecture dependencies to build. I assume then caching would need to be solved by the build machine.

You can see the fetch and link step below, when doing back-to-back builds of the same commit. These should essentially be noop with build caching.

11:28:27.392    ➤ YN0000: ┌ Fetch step
11:29:42.693    ➤ YN0013: │ 12 packages were already cached, 2585 had to be fetched
11:29:42.721    ➤ YN0000: └ Completed in 1m 16s
11:29:42.898    ➤ YN0000: ┌ Link step
11:31:08.063    ➤ YN0007: │ sharp@npm:0.30.7 must be built because it never has been before or the last one failed
... more ...
11:31:23.386    ➤ YN0000: └ Completed in 1m 41s

11:28:27.392    ➤ YN0000: ┌ Fetch step
11:29:42.693    ➤ YN0013: │ 12 packages were already cached, 2585 had to be fetched
11:29:42.721    ➤ YN0000: └ Completed in 1m 16s
11:29:42.898    ➤ YN0000: ┌ Link step
11:31:08.063    ➤ YN0007: │ sharp@npm:0.30.7 must be built because it never has been before or the last one failed
... more ...
11:31:23.386    ➤ YN0000: └ Completed in 1m 41s

Andrew Ferk•10/2/23, 5:14 PM

Specifically, I think

.yarn/cache

.yarn/cache

needs to be cached for the "Fetch step" and

.yarn/install-state.gz

.yarn/install-state.gz

needs to be cached for the "Link step".

reitznerdomenik•10/3/23, 8:14 AM

Hi, is there any plan to extend the build caching to nuxt3 as well?
Or is there a way to hack around it and use those predefined folders for caching?

Georges•10/3/23, 9:02 AM

I was wondering if you would consider adding the

.jampack/cache

.jampack/cache

used by jampack. A user has been asking for it https://github.com/divriots/jampack/issues/106

Georges•10/3/23, 9:04 AM

jampack

jampack

is too small for you to consider it, could you add a generic folder like

".build_cache"

".build_cache"

or something so projects don't need to highjack existing folders to work around it?

Aadaptive Was the beta suspended? I can't see it on the dashboard.

James•10/3/23, 4:46 PM

If you're using v2 build image (system), it should just be an option in the UI: https://i.james.pub/file/2023/10/6dbcb864-d5f0-4f77-b03a-ad7b1ffe1a3f.png

James•10/3/23, 4:47 PM

Actually I think the UI is always present, but it only works in v2 build image afaik

Cretezy•10/3/23, 8:48 PM

Is there a way to customize the cached paths?
We have Next.js under a sub-directory, so

.next/cache

.next/cache

doesn't work. We don't want to update the root directory option (since we have

functions

functions

and

web

web

/Next.js as top-level directories in our repo)

James•10/4/23, 12:16 AM

Not at this time, as far as I now

Cretezy•10/4/23, 5:01 AM

Okay, thank you. That seems like an important feature

groenroos•10/4/23, 12:53 PM

Turning on build cache makes the build fail with

Error: Output directory "build" not found.

Error: Output directory "build" not found.

is there something I can do to avoid this?

Erisa•10/4/23, 1:00 PM

And if you turn cache off and try the same build again, it works?

Isaac McFadyen•10/4/23, 6:07 PM

Cross-posting from next-on-pages: interesting suggestion around possibly having some sort of config file to configure caching behavior or what folders are cached? https://canary.discord.com/channels/595317990191398933/1135899572141568050/1159189844069715998

JohnDotAwesome•10/4/23, 7:14 PM

Configurable cache directories is definitely something on our mind. We actually had them at the beginning but took them out because we wanted to be extra sure/deliberate about how we were exposing that config to users. Expect configurable cache in the future

EErisa And if you turn cache off and try the same build again, it works?

groenroos•10/9/23, 12:24 PM

Correct - also the fail only happened on the second build after turning it on, and then consistently after that until turning it off

Erisa•10/9/23, 12:24 PM

?pages-deployment-id

Flare•10/9/23, 12:24 PM

The Pages deployment ID is a unique build identifier.
It's the UUID in the browser bar (for example, a URL would be

dash.cloudflare.com/ACCOUNT_ID/pages/view/PROJECT/DEPLOYMENT_ID

dash.cloudflare.com/ACCOUNT_ID/pages/view/PROJECT/DEPLOYMENT_ID

where the deployment ID looks something like a398d794-7322-4c97-96d9-40b5140a8d9b).
This ID can help troubleshoot some issues with Pages builds so if you have a failing build make sure you grab that ID for the Pages team to use.

Erisa•10/9/23, 12:25 PM

Do you have deployment IDs for those?

groenroos•10/9/23, 12:30 PM

I'm being affected by the Pages incident today, and so the admin panel is being a bit temperamental, but I managed to eke this out;

aed17acf-cdcc-4916-94fe-64933705c961

aed17acf-cdcc-4916-94fe-64933705c961

Ggroenroos I'm being affected by the Pages incident today, and so the admin panel is being ...

Jacob•10/12/23, 9:44 PM

Can you try add ing

.PHONY: <target goes here>

.PHONY: <target goes here>

to your

make

make

target? It looks like

make

make

isn't actually running anything for some reason

Kamran•10/27/23, 1:50 PM

I'm rewriting our site in Docusaurus on CF Pages, so having

.docusaurus

.docusaurus

cached would be amazing too

Piotr•10/29/23, 5:00 PM

I've enabled the cache and it seems to be working for my 11ty site. Still have to disable one addPassthroughCopyaddPassthroughCopy for a small thumbnail folder. Unsure if it will work as I think it will so would be nice to get some clarification.

Right now I have original article images in git with the project and 11ty generates optimized AVIF/WebP out of them that then are displayed. For now, I stored them in git as well and just copied the folder to _site/_site/ output-dir during the build process so the build never generated thumbnails.

What I want to do is to stop storing thumbnails in git and just generate them directly to the _site/ folder - so using the existing cache during deployment should generate only missing thumbnails, not all of them. Will it work like that? (is the _site_site folder fully cached?)

Also if the cache would be lost the deployment could get screwed as generating all thumbnails from scratch does take quite a while. In such a case, it would be nice to recreate it from a deployed version or something similar.

JohnDotAwesome•11/3/23, 6:55 PM

@nevika FR's for ^ Remix, Docusaurus, and 11ty here ^

Dahoom•11/20/23, 1:51 PM

Why does it say that build caching is not supported for my project at the end of my building process? I'm building it with sveltekit with the adapter @sveltejs/adapter-cloudflare@sveltejs/adapter-cloudflare if that helps.

DDahoom Why does it say that build caching is not supported for my project at the end of...

JohnDotAwesome•11/29/23, 3:47 PM

We have not yet targeted Sveltekit for caching support. @nevika Another FR

Aatom64 hey guys i have an astro 4.0 project and switched to the new Build System V2 and...

Isaac McFadyen•12/12/23, 11:35 PM

This sounds like an Astro issue based on the error... you might want to bring it up with them.

Vero•1/4/24, 11:54 AM

------------------------------

This channel is archived, please use pages-help or pages-discussions

------------------------------

Build caching only applies when using Pages Builds

Similar Threads

Similar Threads

Similar Threads