Edge function workers failing to boot, Typescript error
We are seeing a major problem with about half of our edge function workers failing to initialize due to a typescript error. Our production code hasn't been upated in a month.
event loop error: TypeError: Cannot read properties of undefined (reading 'wasm') at Object../src/env.js (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.9.0:67:467) at webpack_require (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.9.0:106:9092) at Object../src/utils/hub.js (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.9.0:97:191) at webpack_require (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.9.0:106:9092) at Object../src/tokenizers.js (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.9.0:79:1019) at webpack_require (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.9.0:106:9092) at Object../src/pipelines.js (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.9.0:73:801) at webpack_require (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.9.0:106:9092) at Object../src/transformers.js (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.9.0:82:16514) at webpack_require (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.9.0:106:9092)
33 Replies
This just started happening at midnight last night - super frustrating
We found a workaround by disabling the xenova import and disabling that function temporarily, but need some insight into how and why this happened so we can reintroduce the feature and mitigate this intermittent worker boot problem
No one else reporting anything.
There was a runtime change last week at supabase/edge-runtime that maybe hit the servers.
Also you are running on almost 2 year old code version? But at least it is a fixed version so would not change under you.
Have you searched that error message on the web? Is see hits, but have not looked at details.
Generate an issue in supabase/edge-runtime or file a support request if it seems like the runtime change is the issue... https://github.com/supabase/edge-runtime/releases/tag/v1.68.3
GitHub
Release v1.68.3 · supabase/edge-runtime
1.68.3 (2025-07-29)
Bug Fixes
memory leaks could occur due to ModuleLoader (main) (#576) (8658638)
It turns out it is failing some of the time on importing one of our dependencies, just some of the time. Starting last night at midnight ET.
I have a support request with no response yet after a few hours - we have a Teams license and this has us down, but no reply yet.
Could it be the source of the dependency having issues?
Did you at least get the email with the ticket number back so you know the email is working?
I did get an email, yes
This is a ... maybe? Why would it be failing half of the time?
I don't know enough about it. But CDN's have issues if that is where they are coming from. Could be different routes from different edge functions based on what region they run it?
I am having a similar problem with wasm loading on an edge function that stopped working overnight with no deployment changes. In my case it has started reporting file not found for the wasm file it's loading, even though I've verified the wasm files are being uploaded (and, as I said, it broke without any new deployment). I notice in the logs it now says " "served_by": "supabase-edge-runtime-1.68.0-develop.31 (compatible with Deno v2.1.4)"," so I thinkj they've turned on Deno 2.1 for my function and that broke it. My app is basically broken for signups right now and I've had no response from my support request
@Law @jemmyw Sorry, as you have already spotted, we've been updating several regions to use deno 2.1 release. It's likely your functions were affected by those changes.
For now can you use regional invocations (https://supabase.com/docs/guides/functions/regional-invocation) and point your functions to us-east-1, us-west-1, and eu-central-1? Those regions still runs Deno 1.45 and functions should work as before without any changes from your end.
Meanwhile, we'll try to reproduce the issue and deploy a fix as soon as possible.
For @Law would it matter that the version of their npm dependency is 1.7 years old?
@jemmyw can you DM me a link to your support ticket? We might need some more info on why it started failing
can confirm regional invocation works. that's painful for me though, native app so I'll have to build and distribute
@Law Can you also DM me your project ref and function name?
A quick update: it appears both @jemmyw 's and @Law 's issues are related. It has to do with loading wasm files. We've reproduced the issue and figuring out potential fixes.
we merged a fix for it https://github.com/supabase/edge-runtime/pull/579
GitHub
fix: runtime created with the old eszip cannot reference static fil...
What kind of change does this PR introduce?
Bug fix
Description
All scripts initialized in the runtime created by old eszip were specified based on the alternative path reported by maybe_entrypoint...
will probably take 2-3 hours to build and fully roll-out in all regions
thanks @laktek, I notice the PR mentions "old eszip" would upgrading to deno 2 locally have also fixed it?
yeah, in retrospect switching to deno 2 locally and redeploying via CLI would have fixed it too
@jemmyw @Law We've deployed the fix to the affected regions. Can you try and see if your functions work now?
@laktek fixed for me, thank you
As a follow-up to this, it appears it was failing half of the time because traffic was getting routed to servers in different regions, and this 2.1 rollout only happened in some regions.
@laktek this is broken again, and this is really not OK
We are seeing the same failure on new sets of edge functions, "event loop error: TypeError: Cannot read properties of undefined (reading 'wasm')\n at Object../src/env.js
we are down again - this is really, really not fun
This is happening in functions that we did not patch, we got updates that a fix was applied on the server side and we didn't need to do anything, but now customers are reporting they are down again.
Make sure you update your ticket. No guarantee he will see this or what his hours are. I had flagged them with the 2nd issue here otherwise it would eventually have been found in the support queue. Seems like maybe you upgraded to Team just recently or after the request based on the comments I saw as they were searching for your ticket. But it did not get bumped like a team request would have it seemed.
I did update the ticket a half hour ago before posting here, and have been digging through logs to figure out which functions are failing.
I thought a comment here was you could redeploy on an older runtime from the CLI.... but maybe misread that.
I am very leery of hacking around our deployment process to try and keep our production server afloat
this seemed to be resolved earlier today but is broken again, at least for our West Coast US users.
I am carefully patching one edge function at a time to remove features/dependencies that are failing to import and triggering this issue
@Law from the logs can you find in which regions you're still getting the errors?
sorry I didn't get a notification on a reply to your ticket
"region": "us-west-2",
"served_by": "supabase-edge-runtime-1.68.0-develop.32 (compatible with Deno v2.1.4)",
that's actually running the fixed version
Still seeing these an hour ago
is it possible for you to use regional invocations and re-route requests for now?
I'll check again why is it stil failing for your case
we can update some of our client calls, but a lot of our edge functions are webhooks that have to be validated and approved by 3rd parties.
I can't change the queries or header information for those
for webhooks, you may be able to update the URLs and add the following query-parameter:
?forceFunctionRegion=us-west-1that's what I'm saying - I cannot update all of them
gotcha! make sense
will try to see what's the cause and apply another fix. But realistically it would take 4-6 hours to get it resolved from our end
yeah, doing what I can here as well
what's the name of the function you still see these errors?
I will DM you for privacy. Got few questions regarding the functions that are still giving errors