Thanks - I’m going to investigate dependency caching performance soon. It’s tough to save much time

Thanks - I’m going to investigate dependency caching performance soon.
It’s tough to save much time for something that already only takes 15 seconds

JJacob Thanks - I’m going to investigate dependency caching performance soon. It’s toug...

itsmatteomanf•8/30/23, 6:59 PM

Oh, yeah, definitely! I guess the annoying part is that it’s wildly inconsistent (I got 1.5s in a subsequent build for the cache download) and the time to download node each time from node’s website.

itsmatteomanf•8/30/23, 6:59 PM

Deployment seems weirdly slow, but that has been the case for a while. It has some sort of overhead.

Iitsmatteomanf Oh, yeah, definitely! I guess the annoying part is that it’s wildly inconsistent...

JacobOP•8/30/23, 7:11 PM

Can you send me deployment IDs of a slow + fast one?

JJacob Can you send me deployment IDs of a slow + fast one?

itsmatteomanf•8/30/23, 7:22 PM

Oh, wait. Might be I got confused on one that gets skipped due to dep changes

itsmatteomanf•8/30/23, 7:22 PM

They seem to consistently be around 10s, but let me check a couple more.

itsmatteomanf•8/30/23, 7:23 PM

Yeah 8 to 12, sorry

itsmatteomanf•8/30/23, 7:27 PM

(Might make sense to log a skipping message, to clarify it's actually being skipped)

JacobOP•8/30/23, 7:42 PM

Ah yes the skipping would do it. Definitely need log improvements there

JacobOP•8/30/23, 7:43 PM

I wonder if we’re hitting disk IO limits

itsmatteomanf•8/30/23, 7:43 PM

Wasn't Node cached in the image itself? I'm seeing it downloading all the time, and installing.

JJacob I wonder if we’re hitting disk IO limits

itsmatteomanf•8/30/23, 7:43 PM

Probably.

itsmatteomanf•8/30/23, 7:43 PM

Locally it's ~220MB

itsmatteomanf•8/30/23, 7:44 PM

Build ID:

29086095-4c4f-4c08-9f44-22e5756a7f4f

29086095-4c4f-4c08-9f44-22e5756a7f4f

Iitsmatteomanf Wasn't Node cached in the image itself? I'm seeing it downloading all the time, ...

JacobOP•8/30/23, 7:46 PM

Do you have NODE_VERSION set?

JJacob Do you have NODE_VERSION set?

itsmatteomanf•8/30/23, 7:46 PM

I set it from

.nvmrc

.nvmrc

JacobOP•8/30/23, 7:50 PM

The version of node 20 that we currently cache in the image is 20.1.0, so any other patches will have to be downloaded

itsmatteomanf•8/30/23, 7:50 PM

Oh, I set it to latest

v20

v20

JacobOP•8/30/23, 7:51 PM

which means you want the latest v20 installed, right? Not the one we cache?

itsmatteomanf•8/30/23, 7:52 PM

I mean, I'd have to pin it to your version to get that, I'd like to not pin it to 2 minor before current

JacobOP•8/30/23, 7:53 PM

Yeah that's why we prefer newer versions rather than what's cached

itsmatteomanf•8/30/23, 7:54 PM

Yeah, but then you get Node acting up like before, I'd assume it might make sense to cache a version once it's downloaded once, or not?

JacobOP•8/30/23, 7:54 PM

what do you mean by node acting up?

itsmatteomanf•8/30/23, 7:54 PM

I mean, it's a simple Worker with an R2 binding, to reduce the load on them and reduce this case.

JJacob what do you mean by node acting up?

itsmatteomanf•8/30/23, 7:54 PM

It often hangs on the download.

JacobOP•8/30/23, 7:55 PM

are you talking about caching with the new build caching, or the version that ships directly in the image?

itsmatteomanf•8/30/23, 7:55 PM

df4f2bc3-acd3-4b5d-8c74-475f3ed2303d

df4f2bc3-acd3-4b5d-8c74-475f3ed2303d

10 minutes to download Node

v20.3.1

v20.3.1

JJacob are you talking about caching with the new build caching, or the version that sh...

itsmatteomanf•8/30/23, 7:56 PM

Caching it in the caching of the dependencies, since it exists now. Either proxy the node files with some caching (or even R2), or put it in the deps cache.

JacobOP•8/30/23, 7:57 PM

We may do that in the future, but tool caching is out of scope for the current sprint. I'll raise this with the team - 10 minutes is a long time

itsmatteomanf•8/30/23, 7:58 PM

Gotcha, hopefully it's resolved soon, I have seen it fail altogether, but less so lately since the changes in node's infra.

James•8/31/23, 10:28 AM

I'm not really seeing that much of a difference in build times with caching personally. It's taking the same amount of time to restore the dependencies from the cache as it was to download the depedencies with pnpm.

itsmatteomanf•8/31/23, 11:05 AM

Yeah, same here but my lists are small, so can’t really conclude much…

JacobOP•8/31/23, 3:55 PM

I think the speedups will probably be most visible in projects that have long build times. A demo Gatsby project with a lot of files and image resizing went from 4.5 min builds to <2.

We’re deff going to try to make dep caching faster, though.

JJacob I think the speedups will probably be most visible in projects that have long bu...

itsmatteomanf•8/31/23, 5:08 PM

I feel like the most benefit will be in ones which cache the output, too. Dependencies are not that much faster. But avoiding image resizing, etc. cuts things down.

JJacob We're working on some behind-the-scenes work this week, but should have an updat...

itsmatteomanf•9/1/23, 11:38 AM

Can’t wait, as I just discovered Astro’s new image pipeline caches the images in node_modules. So it’s gonna speed up my build a lot

Iitsmatteomanf Can’t wait, as I just discovered Astro’s new image pipeline caches the images in...

James•9/1/23, 1:31 PM

Unlikely with the move to caching npm global cache and not node_modules. Caching node_modules in general is a bad idea, but maybe some allowlist of folders in node_modules to be restored after regular install would be good

itsmatteomanf•9/1/23, 1:36 PM

Oh yeah, right, it's the opposite