I thought the build image v2 ran `npm ci` and not `npm install` to give reproducible builds?

I thought the build image v2 ran

npm ci

npm ci

and not

npm install

npm install

to give reproducible builds?

JJames I thought the build image v2 ran `npm ci` and not `npm install` to give reproduc...

JohnDotAwesome•8/25/23, 8:02 PM

With caching, under certain conditions, we know that step can be skipped altogether

JamesOP•8/25/23, 8:03 PM

I strongly suspect you're going to hit issues with caching the

node_modules

node_modules

folder. It's a path well trodden by other CI platforms and folks for years: https://stackoverflow.com/questions/42521884/should-i-have-travis-cache-node-modules-or-home-npm

JamesOP•8/25/23, 8:03 PM

In my project as an example, I have my site in a separate folder, so I

cd site && npm ci && ...

cd site && npm ci && ...

as my build command

JamesOP•8/25/23, 8:04 PM

~/.npm

~/.npm

was cached, I would benefit, but with

node_modules

node_modules

cache I do not

JJames In my project as an example, I have my site in a separate folder, so I `cd site ...

itsmatteomanf•8/25/23, 8:05 PM

Why aren't you using Pages' root folder option?

Iitsmatteomanf Why aren't you using Pages' root folder option? 🤔

JohnDotAwesome•8/25/23, 8:06 PM

Shenanigans with monorepos and cwd's on Pages

JJohnDotAwesome Shenanigans with monorepos and cwd's on Pages

itsmatteomanf•8/25/23, 8:07 PM

I mean, mine is a monorepo, too...

JamesOP•8/25/23, 8:07 PM

yeah I probably could with this project, but root dir changing definitely causes (or caused?) weird issues in the past, so I just don't bother

JJames The issue with caching `node_modules` is that it can break across node versions,...

Jacob•8/25/23, 8:17 PM

Node version is a good callout, thanks!

JamesOP•8/25/23, 8:26 PM

How is the cache key determined, and how could I use caching successfully with my example project that does

cd site && npm ci && ...

cd site && npm ci && ...

? If

~/.npm

~/.npm

was cached, worst case I have to download a couple new dependencies even if my

package-lock

package-lock

, etc. changes

JamesOP•8/25/23, 8:27 PM

If I didn't have any nested-folder shenanigans, I'd assume updating

package-lock.json

package-lock.json

would invalidate the cache key and mean the entire

node_modules

node_modules

would be re-downloaded?

JamesOP•8/25/23, 8:28 PM

Caching

~/.npm

~/.npm

just seems like a better solution to me and also means that any post-install scripts still run. Let's say I have an npm package that doesn't change version between installs, but has a

postinstall

postinstall

that runs and pulls updated geoip data or something every time. That's not that uncommon of a use-case.

JamesOP•8/25/23, 8:29 PM

By caching

node_modules

node_modules

in its entirety and skipping the install step, that breaks that, vs caching

~/.npm

~/.npm

and just running

npm ci

npm ci

again which'll use that cache, run any normal hooks/scripts, and everything works as expected. You get the benefit of the cache, and any kinda of postinstall or other weird hooks/scripts run as expected.

JamesOP•8/25/23, 8:29 PM

Skipping the

npm install

npm install

step entirely seems like a bad idea

itsmatteomanf•8/25/23, 8:31 PM

Uhm, was my project enabled?

22:30:34.197    Uploading to dependency cache
22:30:34.254    Skipping build output cache as it's not supported for your project

22:30:34.197    Uploading to dependency cache
22:30:34.254    Skipping build output cache as it's not supported for your project

JJames Skipping the `npm install` step entirely seems like a bad idea 🤔

JamesOP•8/25/23, 8:31 PM

I would really think heavily about this @Jacob @JohnDotAwesome . My DMs are open for further discussion, but I really encourage you rethink the design with

node_modules

node_modules

here

JJames I would really think heavily about this <@85379843826413568> <@78687712646795264...

Jacob•8/25/23, 8:36 PM

We're discussing internally, thank you for the feedback!

Iitsmatteomanf Uhm, was my project enabled? ``` 22:30:34.197 Uploading to dependency cache 2...

Jacob•8/25/23, 8:37 PM

It's caching dependencies, but not "build output" such as .cache / public for Gatsby. The log messages are pretty rough right now - we will clean them up soon

JJacob It's caching dependencies, but not "build output" such as .cache / public for Ga...

itsmatteomanf•8/25/23, 8:37 PM

Oh, gotcha

It's currently saving me 8s, so I went from 83s to 75s...

itsmatteomanf•8/25/23, 8:37 PM

Of course there is variations between builds

JJacob We're discussing internally, thank you for the feedback!

JamesOP•8/25/23, 8:38 PM

Thanks, please just let me know if I can provide any more info or feedback! The biggest breakage potential here is any kind of

postinstall

postinstall

, etc. scripts, outside of just unnecessary re-downloads. Skipping the

npm install

npm install

(etc.) step is gonna break any workflow like that that relies on something running, and it's not uncommon for

npm

npm

packages to do that kind of thing.

Iitsmatteomanf Oh, gotcha 🙂 It's currently saving me 8s, so I went from 83s to 75s...

itsmatteomanf•8/25/23, 8:39 PM

The actual build is ~6s.

Jacob•8/25/23, 8:43 PM

We have some optimizations left to be made, so hopefully that will improve!

Jacob•8/25/23, 9:29 PM

something I somehow didn't realize before this project is that a seemingly simple project has like 500-700MB of deps

JamesOP•8/25/23, 9:31 PM

Yeahhhh

node_modules

node_modules

bloat is fun. So many libraries end up publishing both build and source files, tests, docs, etc. inflating file size, and optimising output just isn't a priority

JamesOP•8/25/23, 9:32 PM

Simply installing

wrangler

wrangler

and nothing else in a project for example gives me almost 100MB of

node_modules

node_modules

JJames yeah I probably _could_ with this project, but root dir changing definitely caus...

JohnDotAwesome•8/25/23, 9:46 PM

I've got a change up on deck to fix the previous weirdness (in v2 only). If you set a root_dir previously, you couldn't use a package-lock.json (or similar) at the root of your repo because it would just be gone

JamesOP•8/25/23, 9:47 PM

Ah cool to hear, thanks for the info!

Cyb3r-Jak3•8/26/23, 2:42 AM

I don't think I have an projects using npm but if you ever need python/pip cache testing let me know

Yacine Hmito•8/28/23, 7:17 AM

I am not actively working on a project using any of the supported package managers. We're mainly using Deno. Like cyb3rjak3, let me know if/when you enable either Deno support or something customizable.

JJames Thanks, please just let me know if I can provide any more info or feedback! The ...

Jacob•8/29/23, 7:59 PM

We're going to switch to caching npm/yarn global caches. Thanks!

YYacine Hmito I am not actively working on a project using any of the supported package manage...

Jacob•8/29/23, 7:59 PM

How long does your project take to install dependencies with Deno in CI?

JJacob We're going to switch to caching npm/yarn global caches. Thanks!

JamesOP•8/29/23, 8:00 PM

Thanks for hearing my thoughts there

This makes a lot of sense to me and will mean way more use-cases can be supported too, like my weird

cd site && npm ci

cd site && npm ci

stuff.

Jacob•8/29/23, 8:18 PM

We're working on some behind-the-scenes work this week, but should have an updated beta version early next week!

JJacob How long does your project take to install dependencies with Deno in CI?

Yacine Hmito•8/29/23, 9:35 PM

Downloading dependencies without the cache takes approximately 20 seconds, for a 100k LOC project with 600 files.

itsmatteomanf•8/30/23, 1:23 PM

So, looking at timings:

Downloading the cache takes 10 seconds. It'd be 15s to install them (ok, I don't have many, but still).
Node installation is another 15s.
Deployment of 6 files is another 12.
The actual build is 6s.

Jacob•8/30/23, 6:58 PM

Thanks - I’m going to investigate dependency caching performance soon.
It’s tough to save much time for something that already only takes 15 seconds

JJacob Thanks - I’m going to investigate dependency caching performance soon. It’s toug...

itsmatteomanf•8/30/23, 6:59 PM

Oh, yeah, definitely! I guess the annoying part is that it’s wildly inconsistent (I got 1.5s in a subsequent build for the cache download) and the time to download node each time from node’s website.

itsmatteomanf•8/30/23, 6:59 PM

Deployment seems weirdly slow, but that has been the case for a while. It has some sort of overhead.

Iitsmatteomanf Oh, yeah, definitely! I guess the annoying part is that it’s wildly inconsistent...

Jacob•8/30/23, 7:11 PM

Can you send me deployment IDs of a slow + fast one?

JJacob Can you send me deployment IDs of a slow + fast one?

itsmatteomanf•8/30/23, 7:22 PM

Oh, wait. Might be I got confused on one that gets skipped due to dep changes

itsmatteomanf•8/30/23, 7:22 PM

They seem to consistently be around 10s, but let me check a couple more.

itsmatteomanf•8/30/23, 7:23 PM

Yeah 8 to 12, sorry

itsmatteomanf•8/30/23, 7:27 PM

(Might make sense to log a skipping message, to clarify it's actually being skipped)

Jacob•8/30/23, 7:42 PM

Ah yes the skipping would do it. Definitely need log improvements there

Jacob•8/30/23, 7:43 PM

I wonder if we’re hitting disk IO limits

itsmatteomanf•8/30/23, 7:43 PM

Wasn't Node cached in the image itself? I'm seeing it downloading all the time, and installing.

JJacob I wonder if we’re hitting disk IO limits

itsmatteomanf•8/30/23, 7:43 PM

Probably.

itsmatteomanf•8/30/23, 7:43 PM

Locally it's ~220MB

itsmatteomanf•8/30/23, 7:44 PM

Build ID: 29086095-4c4f-4c08-9f44-22e5756a7f4f29086095-4c4f-4c08-9f44-22e5756a7f4f

I thought the build image v2 ran `npm ci` and not `npm install` to give reproducible builds?

Similar Threads

Similar Threads

Similar Threads