Astro build cache not saving/restoring

Hi hi hello :)

I host a personal image gallery built with Astro on Pages, which naturally contains a bunch of images that get optimized into a few different variants during the build stage, using the built-in

getImage

getImage

getImage

getImage utility. Astro caches these files in the

node_modules/.astro/assets

node_modules/.astro/assets

node_modules/.astro/assets

node_modules/.astro/assets directory so subsequent builds don't need to re-optimize everything from scratch.

Pages is supposed to upload and restore the

node_modules/.astro

node_modules/.astro

node_modules/.astro

node_modules/.astro directory as long as build caching is enabled, you're using v2, etc. and also if it detects "astro" as a dependency (or something like that). However, in my case, it is not doing this, resulting in build times of ~17 minutes every time.

There is another ticket from late last year about this problem but it seems to be a separate issue: build-caching-betaAstro build caching

cowslawOP•6/24/24, 12:05 AM

I have confirmed that the images are properly being saved to the Astro cache directory by changing the build command:

# first build
# after this build, node_modules/.astro/assets contains the cached images
astro build && ls -lhR node_modules/.astro

# another build (tweaked pnpm run build command)
# the ls fails, because the files are either not being uploaded or restored
ls -lhR node_modules/.astro && astro build

# first build
# after this build, node_modules/.astro/assets contains the cached images
astro build && ls -lhR node_modules/.astro

# another build (tweaked pnpm run build command)
# the ls fails, because the files are either not being uploaded or restored
ls -lhR node_modules/.astro && astro build

So despite them being saved to the astro cache dir, they aren't being saved between builds. The pages buildbot does not say which or how many files it's caching, it only says the following after the build completes:

2024-06-23T22:23:56.751708Z    Success: Dependencies uploaded to build cache.
2024-06-23T22:23:56.835813Z    Success: Build output uploaded to build cache.

2024-06-23T22:23:56.751708Z    Success: Dependencies uploaded to build cache.
2024-06-23T22:23:56.835813Z    Success: Build output uploaded to build cache.

...which isn't very helpful, so I can't tell if they are not being uploaded in the first place, or if they are being uploaded but not downloaded correctly. I should note that the caching does work locally on two computers and my VPS, so I don't think it's an Astro thing as far as I can tell. e.g. building locally with the cache takes only 8 seconds vs 17 minutes without the cache.

I do have full build logs available that I can send via DM, as well as any other info/files that might help debug. But I am just curious if there's anything I'm missing, because Pages build cache for Astro projects has certainly worked for me before. Thanks in advance!

cowslawOP•7/2/24, 7:29 PM

Is it rude to bump a thread?

lol

Vero•7/3/24, 2:28 PM

Hey. Can you please share the logs and deployment ID?

Vero•7/3/24, 2:28 PM

?pages-deployment-id

Flare•7/3/24, 2:28 PM

The Pages deployment ID is a unique build identifier.
It's the UUID in the browser bar (for example, a URL would be dash.cloudflare.com/ACCOUNT_ID/pages/view/PROJECT/DEPLOYMENT_IDdash.cloudflare.com/ACCOUNT_ID/pages/view/PROJECT/DEPLOYMENT_ID where the deployment ID looks something like a398d794-7322-4c97-96d9-40b5140a8d9ba398d794-7322-4c97-96d9-40b5140a8d9b).
This ID can help troubleshoot some issues with Pages builds so if you have a failing build make sure you grab that ID for the team to use.

cowslawOP•7/7/24, 3:36 PM

Ah sorry I didn't get the notif for this, here is the deployment ID of a build where I am calling ls -lhR node_modules/.astrols -lhR node_modules/.astro after the build step (the directory that is supposed to be cached):

ebd18526-afcf-40f6-904a-88d69779028bebd18526-afcf-40f6-904a-88d69779028b

cowslawOP•7/7/24, 3:37 PM

I hope you can see the logs in there, otherwise I can DM them to you

Vero•7/8/24, 9:17 AM

I've just escalated your issue. A support engineer will look into it

Luís C•7/8/24, 9:56 AM

Hello @Vero

do we have an internal ticket ID for this case? If so, can you DM me that?

LLuís C Hello @Vero 🐙 do we have an internal ticket ID for this case? If so, can you DM...

Vero•7/8/24, 9:57 AM

No, we don't

Luís C•7/8/24, 10:08 AM

Hello @cowslaw, can you please DM me the ticket ID from late last year about this?

cowslawOP•7/8/24, 12:04 PM

I linked to the Discord thread above, but it wasn’t my ticket so I don’t have the ID for you

Walshy•7/15/24, 4:40 PM

We're rolling out some debugging right now @cowslaw

Walshy•7/15/24, 4:40 PM

You still seeing the issue?

cowslawOP•7/15/24, 4:59 PM

I last deployed on the 11th and it was still not working, gonna try again right now! Will I just be able to see what files it’s caching?

cowslawOP•7/15/24, 5:11 PM

aahh guessing it's not out just yet? Output still looks the same and it hasn't cached anything

Walshy•7/15/24, 5:18 PM

they're internal debug logs so you won't see them but yeah not fully out yet, I'll ask you to run a new build when they are

Walshy•7/15/24, 5:18 PM

cc: @Dario who is doing the rollout

WWalshy they're internal debug logs so you won't see them but yeah not fully out yet, I'...

cowslawOP•7/15/24, 5:21 PM

ah gotcha

cowslawOP•7/15/24, 5:35 PM

I was able to repro this issue using a fresh astro project in case you guys needed some extra info too: https://github.com/zaknesler/cloudflare-astro-assets

GitHub

GitHub - zaknesler/cloudflare-astro-assets: Demo of Astro assets no...

Demo of Astro assets not being saved/restored using Cloudflare Pages. - zaknesler/cloudflare-astro-assets

cowslawOP•7/15/24, 5:35 PM

But I will be here to retry deployments, no problem at all

Walshy•7/15/24, 8:10 PM

@cowslaw this is now fully rolled out if you can retry

cowslawOP•7/15/24, 8:27 PM

sweet, it'll finish in a minute but here is the deployment ID: 818f110e-d83b-4c12-bf3a-7b93b3c5fc80818f110e-d83b-4c12-bf3a-7b93b3c5fc80

Walshy•7/15/24, 8:52 PM

Ok debug logs gave us more info

Walshy•7/15/24, 8:53 PM

Essentially, it comes down to this line:

WARN Ignoring not compatible lockfile at /opt/buildhome/repo/pnpm-lock.yaml

pnpm is then very likely changing the lockfile to be compatible with the version installed (8.7.1) and because we key on the lockfile, it's chaning the cache key

Walshy•7/15/24, 8:54 PM

From https://github.com/zaknesler/cloudflare-astro-assets/blob/main/pnpm-lock.yaml seems like it was made with pnpm v9

cowslawOP•7/15/24, 8:54 PM

y'know deep down I felt it was pnpm related

Walshy•7/15/24, 8:54 PM

but 8.7.1 is in use

cowslawOP•7/15/24, 8:57 PM

just pushed with 8.7.1 gonna try that

cowslawOP•7/15/24, 8:58 PM

95e7ce3b-7fc5-4de8-ac3b-f15067c0ebbd95e7ce3b-7fc5-4de8-ac3b-f15067c0ebbd lol that worked, the deployment finished before I could copy and paste the uuid

Dario•7/15/24, 8:59 PM

nice!

cowslawOP•7/15/24, 8:59 PM

ok well I'm swearing off of pnpm

Dario•7/15/24, 9:00 PM

@Walshy | Deploying idea! maybe we should add a (public) warning to the logs, if you're opted in for the build caching and the lock file is not recognized we could clearly log that this will make caching not work, WDYT?

Walshy•7/15/24, 9:00 PM

haha glad it's resolved and glad we have more info here now -- this is the first case we've needed to actually debug build caching so far

DDario @Walshy | Deploying idea! maybe we should add a (public) warning to the logs, if...

Walshy•7/15/24, 9:01 PM

the issue is we didn't know pnpm would ignore this, we just saw the lockfile and was like "oh yay"

but maybe we should have a warning if the hash changes in the build

WWalshy the issue is we didn't know pnpm would ignore this, we just saw the lockfile and...

Dario•7/15/24, 9:03 PM

maybe.... I guess I don't know what the behavior is with all the various PMs, if all just override the lockfile in case it's invalid then I think a warning there makes more sense (+ a warning when the lock file is not present at all of course)

Walshy•7/15/24, 9:03 PM

yarn definitely overrides, npm overrides as well afaik

Walshy•7/15/24, 9:04 PM

no idea about bun

Dario•7/15/24, 9:05 PM

ok other idea....

Dario•7/15/24, 9:05 PM

kinda combining both....

Dario•7/15/24, 9:05 PM

we can check the hash before and after the install command right? and use that as an indication that the install has overridden the lock file

Walshy•7/15/24, 9:07 PM

yep, my thought was put it at the end just so it's seen - logs are noisy no one looks at the middle :LuL:

Walshy•7/15/24, 9:07 PM

but i'm not suepr fussed about it, just that's where my head goes to

Dario•7/15/24, 9:08 PM

ah I see makes sense

WWalshy yep, my thought was put it at the end just so it's seen - logs are noisy no one ...

Dario•7/15/24, 9:08 PM

yes!

Walshy•7/15/24, 9:09 PM

wonder if it's worth exposing the cache key as well

Walshy•7/15/24, 9:09 PM

and documenting how it's generated

Walshy•7/15/24, 9:09 PM

so people can self-debug

cowslawOP•7/15/24, 9:15 PM

Well I'm just gonna go back to npm or use bun instead, thank you both so much for helping me out

cowslawOP•7/15/24, 9:19 PM

hnnng it went from 17 minutes to less than 2 mins, and most of that is pulling from LFS, I am so happy now lol

Astro build cache not saving/restoring

Similar Threads

Similar Threads

Similar Threads