since 2 days ago I'm getting a ton of

since 2 days ago I'm getting a ton of internal error and Network connection lost. errors when calling my durable objects. they don't have particularly high load and it happens to many different instances, not just one there's no useful error to help me fix it unfortunately
UU
Unknown User27d ago
V
Vincent27d ago
✅ Done!
D
DaniFoldi27d ago
V
Vincent27d ago
@Frederik Just sent you all the details. For us, it's mostly the internal error, not the Network connection lost thing.
C
chronark27d ago
sure, 1s
V
Vincent27d ago
@Frederik Small update here: we investigated a bit more today, and still have no good sense of where those internal error errors are coming from, but… we have noticed that they all don't carry the remote: true field, meaning it's unlikely that they're actually coming from transactional storage APIs—contrary to the initial hunch we had before. If so, they would have been thrown by the Durable Object, in which case we would see these errors having a remote: true property when caught in the worker. So it's more likely that they have a different origin, and maybe the focus on transactional storage was a red herring.
V
Vincent26d ago
New update from today. Yesterday we tried to wrap the .fetch() calls we're making from our Worker to the Durable Object in a retry loop in an attempt to mitigate these intermittant failures. See the code snippet we're using. However, this isn't having any effect. We ran into another few of these internal error today, but retrying them will always fail in the identical way. Questions for @Frederik: 1. Should we also be obtaining a new stub instance if we want to do retries like this? 2. Should we add a delay before attempting these retries? Any guidance in this area would help. 3. Do you have an idea already about where these are coming from by any chance?
No description
UU
Unknown User25d ago
V
Vincent24d ago
Thanks, @Frederik — that's good information, will try that next.
C
chronark24d ago
I replaced out code to create a new stub for the retries, but it didn't seem to help, we're still getting a lot of "internal error"
V
Vincent23d ago
@Frederik We just rolled out the change that grabs a new stub on every retry and that seem to have done the trick indeed—thanks! 🙏 This morning, we're still getting these "internal errors" but now whenever we get them, the retry will work (typically on the first retry), which makes this issue a lot less urgent for us. We'd still like to not get those "internal errors" of course, but at least it won't be causing issues for us anymore. I spoke a little too soon. Although in most cases retry works, we're still encountering some internal errors where retrying does not have an effect. So our issues aren't completely resolved yet.
UU
Unknown User23d ago
V
Vincent23d ago
@Frederik Super appreciated! 🙏
C
chronark2d ago
we're also still getting errors about once every 5 min, even with retries with new stubs hey guys, could you share an update with us? it's still happening unfortunately @Frederik have you found a solution yet?
Want results from more Discord servers?
Add your server
More Posts
Exception with no exceptionsHello, I have a weird issue. My worker is throwing ends with an exception, but there is no logs or help with hosting website please.im getting 403 forbidden on my website, ive gotten my domain threw cloudflare and am using hostingerCheck if Presigned URL file is uploadedHow can I check if a file is uploaded using presigned url without workers (I have 100 buckets).Creating and Declaring Durable Object Instance & Alert in SvelteKitI wonder, in what file in SvelteKit should I put this Durable Object instance https://developers.cCannot get image transformations to work with S3 proxy.I'm having an issue getting Image Transformations working, and I'm not 100% why. I have a worker thRestricting worker access?Hi, is it possible to restrict the domain that workers are called from?how return dns type AHowdy! I would like to know if any of you know how to recover the dns type A or AAAA. I created a prcloudflare for setting up domains for cold emailing, how is it in comparison to Godaddy?Been hearing they allocate a different IP for every domain.Is there a possibility to create a subdomain for all workers?As the title says, is there a possibility to have a subdomain like workers.example.com, and then useI have Zaraz all set up for our websiteI have Zaraz all set up for our website but run into two issues, which seems bugs on Cloudflare's siNeed help with 020Our previous security professional put us on 020 to help with attacks on our Shopify store. It workeDurableObjectStub is missing all methodsHey, I'm having a little issue. I'm trying my hand at Durable Objects for the first time and I've glive stream via workersHow can I make a live stream with a direct link through workers?Strange issue with `console.log()` omitting members of objectsI'm using Itty Router with CF Workers. For some reason, when working locally and I run `console.log(No images and internal linkshttps://test0-5la.pages.dev/ fe15e1a9a809cda9a9fbd120dd546cc2 I restored a website from wayback macDeploy preview with wrangler cliHello everyone, Is it possible to deploy a preview with the wrangler cli without connecting github Multi tenant pagesIs it possible to create a wildcard and have a multi tenant pages project that displays different coPostgresql Connection (Node) erroring: No such module "cloudflare:sockets".I've been trying to test out a CF worker with a basic DB connection and query and tried everything tAudit Logs and Old Values Not Displayed#general-help I checked the audit logs and the old values aren't there. If I upgrade to a paid vpnpm is suddenly on v9.0.1``` pages.dev: https://site-3vg.pages.dev/ deployment ID: 35f36db7-b65d-478b-86e7-70b3c9383ee3 accou