Cloudflare Developers•3y ago

Waiting for them to open it up to just being able to configure directories to cache

Vero•8/25/23, 2:44 PM

The form should be open now

HHello, I’m Allie!Waiting for them to open it up to just being able to configure directories to ca...

JohnDotAwesome•8/25/23, 2:44 PM

Boy have I got an environment variable for you!

James•8/25/23, 2:47 PM

Very excited about this

Vero•8/25/23, 2:49 PM

I see, you were the first

Iitsmatteomanf So… none of my projects meet the criteria 🙈😂

aninibread•8/25/23, 2:51 PM

Oh quick clarification! Build cache will support Next.js and Gastby.js, and the pacakge managers npm, pnpm, and yarn v1. So if you employ either the supported frameworks and/or the supported package managers then this feature will work for your project.

If the project has the supported frameworks then build caching for build outputs will work. If the project has the supported package managers then build caching for dependencies will work. If both, then both will work!

Aaninibread Oh quick clarification! Build cache will support Next.js and Gastby.js, and the ...

itsmatteomanf•8/25/23, 2:52 PM

Ohhhhh, I see then I can enable that

will fill the form later

itsmatteomanf•8/25/23, 2:52 PM

I have a site that takes like 90s of which 85s are overhead ahah

JJohnDotAwesome Boy have I got an environment variable for you!

itsmatteomanf•8/25/23, 2:53 PM

Oh, do explain more

Iitsmatteomanf I have a site that takes like 90s of which 85s are overhead ahah

itsmatteomanf•8/25/23, 2:53 PM

Astro’s SSR builds with like 5 pages are so fast ahah

Jacob•8/25/23, 3:48 PM

hey folks, super excited about y'all helping us test this feature! Will be able to start enabling projects shortly

Jacob•8/25/23, 4:02 PM

Enabled everyone who signed up so far

Vero•8/25/23, 4:14 PM

Thanks Jacob!

James•8/25/23, 5:07 PM

I think John already knows what I'd ask for

JJames I think John already knows what I'd ask for 😅

JohnDotAwesome•8/25/23, 5:11 PM

Bun?!

JJohnDotAwesome Bun?!

James•8/25/23, 5:11 PM

JohnDotAwesome•8/25/23, 5:11 PM

I think technically you could get it working... You're already setting SKIP_DEPENDENCY_INSTALL, right?

James•8/25/23, 5:12 PM

Negative, the only projects using it are the ones on your branch

JohnDotAwesome•8/25/23, 5:15 PM

I had someone else asking for bun as a package manager yesterday. That FR is up to 10 likes. I think if you ask @Vero enough times, she could make it happen :p

Vero•8/25/23, 5:15 PM

I've asked for that a lot too hahah

James•8/25/23, 5:15 PM

Vero•8/25/23, 5:16 PM

so yeah, keep asking and I'll keep insisting

James•8/25/23, 5:16 PM

@Vero bun pls

Vero•8/25/23, 5:16 PM

ahhahah, I'm actually sending my newsletter today and that's the request # one

VVero ahhahah, I'm actually sending my newsletter today and that's the request # one

DaniFoldi•8/25/23, 6:45 PM

Bun pls

itsmatteomanf•8/25/23, 7:24 PM

Filled the form

itsmatteomanf•8/25/23, 7:25 PM

Let's try one project, then go from there lul

itsmatteomanf•8/25/23, 7:28 PM

Ok, let's be daring and do two lul

James•8/25/23, 7:54 PM

With

npm

npm

, which directories are cached? ~/.npm~/.npm?

JJames With `npm`, which directories are cached? `~/.npm`?

JohnDotAwesome•8/25/23, 7:58 PM

Just

node_modules

node_modules

right now. What goes into ~/.npm~/.npm?

JJohnDotAwesome Just `node_modules` right now. What goes into `~/.npm`?

James•8/25/23, 7:58 PM

Oh.. that's a bad idea

James•8/25/23, 7:58 PM

Let me get you some docs, sec

JohnDotAwesome•8/25/23, 7:59 PM

Unless you're running

npm install

npm install

as a follow up to Pages running it for you, caching ~/.npm~/.npm wouldn't have an effect

James•8/25/23, 8:00 PM

The issue with caching

node_modules

node_modules

is that it can break across node versions, OS versions, and won't work with npm cinpm ci since npm cinpm ci always nukes

node_modules

node_modules

first. You want to cache the npm cache result, which is ~/.npm~/.npm unless you're overriding the npm cache location.

James•8/25/23, 8:00 PM

Github's own cache action hit this with early version, and you can see their docs about this here for example: https://github.com/actions/cache/blob/main/examples.md#node---npm

JJames The issue with caching `node_modules` is that it can break across node versions,...

JohnDotAwesome•8/25/23, 8:00 PM

We skip running npm cinpm ci altogether and (ought to) incorporate node version in the cache key

James•8/25/23, 8:01 PM

I thought the build image v2 ran npm cinpm ci and not

npm install

npm install

to give reproducible builds?

JJames I thought the build image v2 ran `npm ci` and not `npm install` to give reproduc...

JohnDotAwesome•8/25/23, 8:02 PM

With caching, under certain conditions, we know that step can be skipped altogether

James•8/25/23, 8:03 PM

I strongly suspect you're going to hit issues with caching the

node_modules

node_modules

folder. It's a path well trodden by other CI platforms and folks for years: https://stackoverflow.com/questions/42521884/should-i-have-travis-cache-node-modules-or-home-npm

James•8/25/23, 8:03 PM

In my project as an example, I have my site in a separate folder, so I cd site && npm ci && ...cd site && npm ci && ... as my build command

James•8/25/23, 8:04 PM

If ~/.npm~/.npm was cached, I would benefit, but with

node_modules

node_modules

cache I do not

JJames In my project as an example, I have my site in a separate folder, so I `cd site ...

itsmatteomanf•8/25/23, 8:05 PM

Why aren't you using Pages' root folder option?

Iitsmatteomanf Why aren't you using Pages' root folder option? 🤔

JohnDotAwesome•8/25/23, 8:06 PM

Shenanigans with monorepos and cwd's on Pages

JJohnDotAwesome Shenanigans with monorepos and cwd's on Pages

itsmatteomanf•8/25/23, 8:07 PM

I mean, mine is a monorepo, too...

James•8/25/23, 8:07 PM

yeah I probably could with this project, but root dir changing definitely causes (or caused?) weird issues in the past, so I just don't bother

JJames The issue with caching `node_modules` is that it can break across node versions,...

Jacob•8/25/23, 8:17 PM

Node version is a good callout, thanks!

James•8/25/23, 8:26 PM

How is the cache key determined, and how could I use caching successfully with my example project that does cd site && npm ci && ...cd site && npm ci && ...? If ~/.npm~/.npm was cached, worst case I have to download a couple new dependencies even if my package-lockpackage-lock, etc. changes

James•8/25/23, 8:27 PM

If I didn't have any nested-folder shenanigans, I'd assume updating

package-lock.json

package-lock.json

would invalidate the cache key and mean the entire

node_modules

node_modules

would be re-downloaded?

James•8/25/23, 8:28 PM

Caching ~/.npm~/.npm just seems like a better solution to me and also means that any post-install scripts still run. Let's say I have an npm package that doesn't change version between installs, but has a postinstallpostinstall that runs and pulls updated geoip data or something every time. That's not that uncommon of a use-case.

James•8/25/23, 8:29 PM

By caching

node_modules

node_modules

in its entirety and skipping the install step, that breaks that, vs caching ~/.npm~/.npm and just running npm cinpm ci again which'll use that cache, run any normal hooks/scripts, and everything works as expected. You get the benefit of the cache, and any kinda of postinstall or other weird hooks/scripts run as expected.

Waiting for them to open it up to just being able to configure directories to cache

Similar Threads

Similar Threads

Similar Threads