Cloudflare Developers•3y ago

Next attempt is new commit...

Vero•9/4/23, 12:38 PM

Hmm weird, I've asked internally but no response yet

Iitsmatteomanf Next attempt is new commit...

itsmatteomanfOP•9/4/23, 1:02 PM

Nope, this didn't seem to work either. I hoped when it took 30+ seconds initialize the environment lul

Iitsmatteomanf Nope, this didn't seem to work either. I hoped when it took 30+ seconds initiali...

Vero•9/4/23, 1:15 PM

well, let's wait for John and Jacob to get back to work

Vero•9/4/23, 1:35 PM

@itsmatteomanf Walshy just found the issue, is it working now?

VVero <@487247657736339456> Walshy just found the issue, is it working now?

itsmatteomanfOP•9/4/23, 1:35 PM

Retrying

Walshy, the savior lul

itsmatteomanfOP•9/4/23, 1:36 PM

Yes, shows the attempt to restore from cache!

Vero•9/4/23, 1:39 PM

yaay

itsmatteomanfOP•9/4/23, 1:49 PM

Questions for the team, once they actually get back, no rush:

Are there size limits for the cache?
I am not actually seeing the cache in
```
node_modules
```
```
node_modules
```
being used, so is every folder saved? Are .[folder].[folder] excluded?

Iitsmatteomanf Questions for the team, once they actually get back, no rush: - Are there size l...

Isaac McFadyen•9/4/23, 8:17 PM

AFAIK node_modules isn't cached.

Isaac McFadyen•9/4/23, 8:17 PM

Only ~/.npm is cached.

Isaac McFadyen•9/4/23, 8:17 PM

See James' comments above.

IIsaac McFadyen AFAIK node_modules isn't cached.

itsmatteomanfOP•9/4/23, 8:18 PM

I think it’s currently the opposite.

JJames How is the cache key determined, and how could I use caching successfully with m...

Isaac McFadyen•9/4/23, 8:18 PM

@itsmatteomanf

JJacob We're going to switch to caching npm/yarn global caches. Thanks!

Isaac McFadyen•9/4/23, 8:18 PM

Oh OK, yeah Jacob mentioned they were going to do it here but didn't say if that was current or not.

itsmatteomanfOP•9/4/23, 8:19 PM

Yeah, it’s planned for this week afaik, but we’ll see tomorrow

IIsaac McFadyen Only ~/.npm is cached.

Jacob•9/5/23, 5:48 PM

Working on making this change this week. Current beta still caches node_modules

Luke•9/5/23, 6:20 PM

Hello! I see only Next.js and Gatsby are supported so far. I have a few Vite projects and Nuxt projects I could test with once they're supported

JJacob Working on making this change this week. Current beta still caches node_modules

James•9/6/23, 11:02 AM

Are y’all planning on landing this before adding more folks to the beta? Excited to see this rolling out!

simpson•9/6/23, 7:16 PM

how do we tell if caching is enabled on our account?

Ssimpson how do we tell if caching is enabled on our account?

itsmatteomanfOP•9/6/23, 7:17 PM

There should be a message pretty soon after the build starts, in the logs, that is fetching the cache, and one at the end that is saving it (or skipping it for some reason)

JJames Are y’all planning on landing this before adding more folks to the beta? Excited...

Jacob•9/7/23, 3:37 PM

Adding more users shortly - expecting dependency cache changes by EOW!
Edit: Probably more like early next week. Possibly Astro support soon too

Jacob•9/8/23, 6:56 PM

Just pushed an update - dependency caching now caches global npm/yarn caches. Thanks again for the feedback @James

Jacob•9/8/23, 6:56 PM

Users who have submitted projects the last few days will be enabled in a bit

JJacob Just pushed an update - dependency caching now caches global npm/yarn caches. Th...

James•9/8/23, 7:22 PM

Amazing, thank you for the update and being so receptive to feedback! MeowHeartCloudflare

JJacob Just pushed an update - dependency caching now caches global npm/yarn caches. Th...

JohnDotAwesome•9/8/23, 7:49 PM

Do note that with the change in cache strategy, your previous caches will no longer match

Jacob•9/8/23, 9:27 PM

Astro support is now live party_blob

JJacob Astro support is now live <a:party_blob:818919990635266079>

itsmatteomanfOP•9/8/23, 9:47 PM

And... it works?

Jacob•9/8/23, 9:56 PM

should now!

itsmatteomanfOP•9/8/23, 9:56 PM

Let me retry!

itsmatteomanfOP•9/8/23, 9:57 PM

Looks like it does!

Jacob•9/8/23, 9:59 PM

All users that submitted projects now has build caching enabled for the submitted project. You will need to click

Enable build cache

Enable build cache

in the

Builds and deployments

Builds and deployments

settings.
For the first few users, the build cache toggle is currently ignored. But please enable it, as we will remove the override in the next week or two so that you can turn it on/off as desired.

NOTE: The "Enable build cache" options will show up in all projects, but it will ONLY have any affect on the project(s) you submitted.
Also: The

Clear cache

Clear cache

button won't work until early next week.

Thanks!

simpson•9/9/23, 11:37 AM

Do we have to make any code changes on our side to reference the cache? I enabled it and it looks to work from what I see but it looks like my builds are the same or maybe a little slower

simpson•9/9/23, 11:39 AM

    
07:32:28.237    Success: Finished cloning repository files
07:32:29.372    Restoring from dependencies cache
07:32:29.401    Restoring from build_output cache
07:32:29.945    Successfully restored from dependencies cache
07:32:39.126    Successfully restored from build_output cache
07:32:39.664    Detected the following tools from environment: pnpm@8.6.6, nodejs@18.16.1
07:32:39.669    Installing project dependencies: pnpm install
07:32:41.489    Lockfile is up to date, resolution step is skipped
07:32:41.541    Progress: resolved 1, reused 0, downloaded 0, added 0
07:32:41.697    Packages: +615
07:32:41.698    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
07:32:42.223    Packages are hard linked from the content-addressable store to the virtual store.
07:32:42.223      Content-addressable store is at: /opt/buildhome/.local/share/pnpm/store/v3
07:32:42.223      Virtual store is at:             node_modules/.pnpm

    
07:32:28.237    Success: Finished cloning repository files
07:32:29.372    Restoring from dependencies cache
07:32:29.401    Restoring from build_output cache
07:32:29.945    Successfully restored from dependencies cache
07:32:39.126    Successfully restored from build_output cache
07:32:39.664    Detected the following tools from environment: pnpm@8.6.6, nodejs@18.16.1
07:32:39.669    Installing project dependencies: pnpm install
07:32:41.489    Lockfile is up to date, resolution step is skipped
07:32:41.541    Progress: resolved 1, reused 0, downloaded 0, added 0
07:32:41.697    Packages: +615
07:32:41.698    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
07:32:42.223    Packages are hard linked from the content-addressable store to the virtual store.
07:32:42.223      Content-addressable store is at: /opt/buildhome/.local/share/pnpm/store/v3
07:32:42.223      Virtual store is at:             node_modules/.pnpm

Ssimpson Do we have to make any code changes on our side to reference the cache? I enable...

itsmatteomanfOP•9/9/23, 12:46 PM

Nope, no code changes... the cache takes some time, so for very fast installs it won't change much, unfortunately.

simpson•9/9/23, 12:46 PM

No worries! Just wanted to make sure I wasn’t missing anything

Ssimpson Do we have to make any code changes on our side to reference the cache? I enable...

Jacob•9/11/23, 12:34 PM

Can you send me the deployment ID? I'd like to take a look - our goal is to make sure that at the very least, it doesn't slow down builds

JJacob Can you send me the deployment ID? I'd like to take a look - our goal is to make...

simpson•9/11/23, 6:47 PM

i'll try to grab it later, to be fair it was only like 10 seconds longer

josh•9/14/23, 11:01 PM

@Jacob any plans to support caching bundler/rubygems for Ruby builds in the future?

Jjosh <@85379843826413568> any plans to support caching bundler/rubygems for Ruby buil...

Jacob•9/14/23, 11:23 PM

Deff want to add support for more languages in the future, yes

How much time does your project take cached vs non-cached? (eg. when rebuilding locally)

JJacob Deff want to add support for more languages in the future, yes 🙂 How much time ...

josh•9/14/23, 11:40 PM

latest build is 2m 26s uncached at Cloudflare, 49s cached at Netlify

josh•9/14/23, 11:41 PM

this is a middleman site (quite similar to jekyll, should use the same ruby/node caching approach)

Jacob•9/15/23, 12:25 AM

Thanks!

itsmatteomanfOP•9/19/23, 12:12 PM

@Jacob did you think about the build cache not being invalidated on dependecy updates?

Iitsmatteomanf <@85379843826413568> did you think about the build cache not being invalidated o...

Jacob•9/20/23, 4:50 AM

A bit but nothing planned yet. This first iteration is intended to be very safe for most projects, at the cost of a few more cache misses. How often do you change your dependencies?

Jacob•9/20/23, 4:52 AM

What would you prefer as the cache key?

JJacob A bit but nothing planned yet. This first iteration is intended to be very safe ...

itsmatteomanfOP•9/20/23, 7:43 AM

It’s not the changing of dependencies (it happens often now as I am building the website, but will most likely lower), but the updates. Astro releases an update a week, add other packages and it’s very often.

JJacob What would you prefer as the cache key?

itsmatteomanfOP•9/20/23, 7:45 AM

Do we need a cache key for the output cache? I get dependencies, might make sense, although I am not sure what Netlify does, for example… but the cache output is handled by each package.

Iitsmatteomanf Do we need a cache key for the output cache? I get dependencies, might make sens...

Jacob•9/20/23, 1:52 PM

What if you have a different branch with vastly different settings? Maybe we could use the branch as a cache key instead? What if you change frameworks? (I guess we could rely on you hitting the Clear Cache button for that?)

Jacob•9/20/23, 1:56 PM

We were trying to avoid users having to hit Clear Cache and having things just work.

Jacob•9/20/23, 1:56 PM

But we weren't thinking about users updating deps every week

07:32:28.237 Success: Finished cloning repository files 07:32:29.372 Restoring from dependencies cache 07:32:29.401 Restoring from build_output cache 07:32:29.945 Successfully restored from dependencies cache 07:32:39.126 Successfully restored from build_output cache 07:32:39.664 Detected the following tools from environment: pnpm@8.6.6, nodejs@18.16.1 07:32:39.669 Installing project dependencies: pnpm install 07:32:41.489 Lockfile is up to date, resolution step is skipped 07:32:41.541 Progress: resolved 1, reused 0, downloaded 0, added 0 07:32:41.697 Packages: +615 07:32:41.698 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 07:32:42.223 Packages are hard linked from the content-addressable store to the virtual store. 07:32:42.223 Content-addressable store is at: /opt/buildhome/.local/share/pnpm/store/v3 07:32:42.223 Virtual store is at: node_modules/.pnpm

Next attempt is new commit...

Similar Threads

Similar Threads

Similar Threads