JSad DX has to be such that we need to build things like proxy-fetch.
We either use waitUntil, queues or workflows to avoid wall-time specially for inference call scenarios. C'est la vie.
We need an out-of-the-box solution.
We either use waitUntil, queues or workflows to avoid wall-time specially for inference call scenarios. C'est la vie.
We need an out-of-the-box solution.
1What's the delay from sending off the proxy fetch to it being returned? My issue with Queues was always that few seconds delay while the queue is receiving the event
LIt varies wildly, but with this Queues-based variety, you should count on 2-3 seconds p90. As @Jehu wrote, just making the fetch from the DO is often the right choice. It could turn out that the diagram might be the most valuable thing about pushing this out, but maybe someone will find it useful.
We're also working on the docs for a variety that's based upon Durable Objects. Its latency depends upon where in the world the proxy DO is in relation to the DO that makes the fetch. I don't have numbers, but my guess is that same co-lo, you'd be talking no more than 20ms latency, on top of the fetch itself of course. You can stand up an instance of it in each region, which will cost ~$4/month each from Cloudflare but we're hoping to also offer it as a service for about the same, except we take care of scaling and world-wide co-lo. Sad part is, we're essentially re-inventing Queues.
That version also does streams and SSE but we are struggling to live test demo those features and may only release that as a service.
We're also working on the docs for a variety that's based upon Durable Objects. Its latency depends upon where in the world the proxy DO is in relation to the DO that makes the fetch. I don't have numbers, but my guess is that same co-lo, you'd be talking no more than 20ms latency, on top of the fetch itself of course. You can stand up an instance of it in each region, which will cost ~$4/month each from Cloudflare but we're hoping to also offer it as a service for about the same, except we take care of scaling and world-wide co-lo. Sad part is, we're essentially re-inventing Queues.
That version also does streams and SSE but we are struggling to live test demo those features and may only release that as a service.
1How do I tag mods or is there a ? command for spam like this
LThere is a report post feature in the Discord dropdown. I reported as soon as I saw it. Someone at Cloudflare deleted soon after my report.
LWhat's the error you'll get if you exceed the pending outgoing fetch limit? I know the current in flight limit is 6, but I've tried getting 200 in flight at the same time and I haven't exceeded that limit or gotten an error. This is in vitest-pool-workers so maybe I need to experiment in prod?
1Definitely wanna test in prod first
LYou don't get an error when you have more than 6, they just get queued and once one completes another one runs. The error is when you are more than 1000 by default I think.
LThanks. What are the details of the error so I can catch it specifically? Or should I just send 2000 and see?
C2 [ sse / durable object ] questions please. 1) There is a lovely blog about hibernating ws's, is this something we should also do for sse's? 2) How does pricing for sse's work please?
ty!!!
ty!!!
HHibernation is only available for WebSockets at the moment
CThanks for the info! Do we get charged for like amount of time writers exist in do's, here's the simple prototype that I'd like to build up, unless it'll we are charged for each moment the do exists!
worker:
html:
worker:
html:
Cdurable object:
LI have been pestering the agents team about this since I become a member of Anthropic's MCP Transports Working Group. We either need hibernating SSE's or we need to get my WebSocket SEP approved.
LFor now, I'm working on productionalizing my proxyFetch. I have the Queue based one out. The SSE one is coming soon. https://lumenize.com/docs/proxy-fetch/
Cost-effective external API calls from Durable Objects using Cloudflare Queues
1I had this issue just last week, the ID will always spawn in WEUR regardless of deletion of storage or anything. Best suggestion has been to use a different ID I guess (which might not work depending on use case)
1I was getting DOs spawning in Perth Australia instead of Melbourne where I live or Sydney, so I was getting hit with 100s of ms extra request time
Cwell the DO won't move once it's created, and even deleting the storage it still sticks around for a while, at least a few hours last I knew, not sure if there's any better docs/knowledge on that but afaik it's all heavily cached once created.
I think here some location hints might be more helpful if your users are from a specific area, to force it consistently
I think here some location hints might be more helpful if your users are from a specific area, to force it consistently
Cfwiw there was a decent bit of request rerouting from north america -> europe earlier today (requests being sent in North America being handled in Europe usually due to capacity reasons), like over 50% going to Europe for 3 hours or so peak US time (for free plan, slightly less for pro/biz/etc, going up the plans). It's calmed down now, but if your request gets bounced over there, Worker runs over there, placement of the DO will be near there
Cyea it's still happening but has just gone down, if the primary concern of yours is this rerouting it's not very often it's done, only recently there's been some of it. It's pretty transient but yea annoying to work around for still. Ultimately during the rerouting the placing of the object in Europe is good placement, since going back to NA would be further, it's just after it gets messy
Clast major time it was happening was August (for North America), and then nothing for the months before, going to vary depending on the exact location you're hitting though
CI was trying to find a good reference and as far as I know and can find, the ID and the location exists forever even if you delete all storage, won't be re-placed
durable-objects
There's some downstream constraints from this like re-creating R2 buckets with the same name (which use DOs) never change regions, I imagine from the same reason...
durable-objects
Storage gone, the object lives forever thoughI have some DOs at least a month old which say they have no storage that still are "around" in CF's Durable Objects Objects list api.
There's some downstream constraints from this like re-creating R2 buckets with the same name (which use DOs) never change regions, I imagine from the same reason...
LWe release the Queues based variant of proxy-fetch yesterday. Today we release the Durable Objects based one. latency of 50-100ms compared to 200ms to 3 seconds for the Queue based one. Stay tuned for the SSE one next. See: https://lumenize.com/docs/proxy-fetch/durable-object
Use a dedicated proxy DO to handle external fetches
L
1The idea is that it would be dealing with only so many requests at once so that the 6 outgoing limit isn't an issue. If you needed more concurrency you could spin up more ProxyFetchDO instances
1This is great. I have a bunch of use cases for this already. I had planned on implementing Queues in some way but the variable delay for consuming was making me weary. ProxyFetch should be a great solution, now you just need to port the Schedule API from Agents so we can easily add delays to the proxy-fetch/callback firing
TWhen a Frontend Worker calls a DO which is in a different datacenter, does the request go over public internet or across CFs own fiber?
LOur own internal networking.
1Is that the case for all requests from Cloudflare assets/resources to other Cloudflare assets/resources? Like if a DO requests a file from R2 is that also internal networking?
LIn theory it should, yes. I cannot answer for "all" Cloudflare products with certainty though! If it's an internal dev platform product though, it usually stays within our network as much as possible.
Tthanks
1That's cool thank you, always like getting more info about the inner workings. Helps me to be able to appropriately build stuff to take advantage of the platform where possible
LAs said, the DO will move towards the original location/colo if that is healthy for it to run. See durable-objects
LNetwork routing has MANY complexities and edge cases, colos going down, peers going down, cables being bitten by sharks or cut by people, so my suggestion is to benchmark first and not base your assumptions on theoritical statements. 
LStay tuned. The next variant will have ProxyFetchDO spin up Workers to do the fetches. That might be next week though, because I have to do a deep dive on the differences between inheriting from RpcTarget and DurableObject. That may result in optimization of proxy-fetch.
When I'm done with all the variants, I'll upgrade my measurment framework (current 3x more effort than the implementations themselves) and do a big writeup of what I learned.
That said, the theoretical cost savings of this variant is 6x, so not bad for 10s of ms latency add. Think of a chat use case. 95% of the time a post doesn't require an external round trip, but what if your policy was to check links against a pornography or scam listing server before you allow links to go into the thread. Say you have 1000 rooms, you'll have 10s of these per second happen and it won't be all that bursty.
When I'm done with all the variants, I'll upgrade my measurment framework (current 3x more effort than the implementations themselves) and do a big writeup of what I learned.
That said, the theoretical cost savings of this variant is 6x, so not bad for 10s of ms latency add. Think of a chat use case. 95% of the time a post doesn't require an external round trip, but what if your policy was to check links against a pornography or scam listing server before you allow links to go into the thread. Say you have 1000 rooms, you'll have 10s of these per second happen and it won't be all that bursty.
Lport the Schedule API from Agents so we can easily add delays to the proxy-fetch/callback firingCan you elaborate?
LBTW, it handles spikes shockingly well. The DO lets you queue 1000 or so outgoing fetches. Sure, they take a while to all resolve because there are only 6 in flight at a time, but it doesn't crash.
THi, I'm new here and from Malaysia. My timing will be off from you guys but I do have some questions in mind that may sound dumb (i'm only learning BE this year), but I've been trying to understand and use Agent SDK.
I'm trying to build a websocket connection using the Agent SDK framework. I was concern on how is the lifecycle of the agent is going to be? because when reading through the documentation there are 2 types of Websockets connection and was suggesting to use the Hibernation Websocket.
With the Agent SDK basically handling everything even on the frontend with useAgent() and backend handling all the routing to the unique agent I will be creating, does the durable object goes into hibernation / sleep when the the agent websocket is onClose() ? Or it will be a default durable object behavior where it still wait for the 10s before hibernation?
I'm trying to build a websocket connection using the Agent SDK framework. I was concern on how is the lifecycle of the agent is going to be? because when reading through the documentation there are 2 types of Websockets connection and was suggesting to use the Hibernation Websocket.
With the Agent SDK basically handling everything even on the frontend with useAgent() and backend handling all the routing to the unique agent I will be creating, does the durable object goes into hibernation / sleep when the the agent websocket is onClose() ? Or it will be a default durable object behavior where it still wait for the 10s before hibernation?
LWhile it is an option in PartyKit, upon which Agents is built, Agents always sets up a hibernating WebSocket.
PJust updated
wranglerHow can I DELETE a durable object instance?
LWhat do you expect to see? You cannot interact with individual Durable Objects outside of a worker (in addition to the just launched Studio in the dashboard).
LYou don't delete DOs. You can clear their storage by calling the method, and
deleteAlarm()Hmm, trying to see if I can use Cloudflare containers for single use jobs, but their are tied to DOs, so I want to delete the container instance after use. Maybe I need to use the storage.deleteAll() and deleteAlam() to clear the DO context then I can reuse that slot?
PI was trying to figure out how to delete a durable object instance, because I have container images that are just piling up taking tons of space (probably), I actaully don't know, cant see that information.