I need to run async things like vectorize, openai interleaved with sql queries though? transactions

I need to run async things like vectorize, openai interleaved with sql queries though? transactions only work if its all sync right?

Iggy•2/25/25, 3:19 AM

Is the 50GB SQLite storage per account a hard limit or soft? If it’s soft, what range would enterprise expect to be able to get to?

HowlOftheSun•2/25/25, 4:25 AM

If I use a decorator on a function in a do would the decorator be able to write to storage in the do? Would I have to pass the ctx?

Ccloudkite I need to run async things like vectorize, openai interleaved with sql queries t...

lambrospetrou•2/25/25, 8:04 AM

I don't follow. Why would you be doing SQL data migrations while you invoke remote APIs? You can do async transactions too, BTW, but it's going to be horribly slow if you mix many migrations with remote APIs.
Anyway, depends what you are doing, but there are async transactions: https://developers.cloudflare.com/durable-objects/api/storage-api/#transaction

Llambrospetrou I don't follow. Why would you be doing SQL data migrations while you invoke remo...

cloudkiteOP•2/25/25, 8:18 AM

Sorry meant that doing inserts from results of remote API

cloudkiteOP•2/25/25, 8:21 AM

So let’s assume I wrap everything in transaction, if a code deploy happens, error is thrown and everything rolls back. So now I need to retry? How do I do that since this is all happening in the websocketmessage handler?

cloudkiteOP•2/25/25, 8:22 AM

client may have left/disconnect, but I still need that state to be recorded

Ccloudkite So let’s assume I wrap everything in transaction, if a code deploy happens, erro...

lambrospetrou•2/25/25, 8:40 AM

This is as any other database really. Ideally, you should not be doing remote calls within your transactions, since it slows down everything. Do your remote call to fetch something, and then start a transaction to do your writes. If the server crashes the writes won't be partially done.

How is this use-case different than any traditional application running on a server and crashing in the middle of your transaction? The database transactions only guarantee consistency within the database, not with the remote calls you make, so retries and all that depends entirely on what the application is doing.

We cannot blindly retry database queries for you since we don't know if it's safe to do so. You are the one knowing that.

Llambrospetrou This is as any other database really. Ideally, you should not be doing remote ca...

cloudkiteOP•2/25/25, 8:56 AM

That’s not really what I was asking. How can I retry since calling any storage call inside the stale DO will error? If I was calling worker -> DO then I presumably would recreate stub and call RPC method or fetch again. However I’m calling websocket -> DO

Ccloudkite That’s not really what I was asking. How can I retry since calling any storage c...

lambrospetrou•2/25/25, 9:07 AM

If the DO crashes/stops running, your websocket will close, so your client needs to reconnect anyway.

Llambrospetrou If the DO crashes/stops running, your websocket will close, so your client needs...

cloudkiteOP•2/25/25, 9:08 AM

Yeh but they may have left/exited in meantime and I still need that state to be saved

cloudkiteOP•2/25/25, 9:16 AM

I wouldn’t have this problem if I use D1/Postgres etc though right? since if I deploy a new version of the DO then the stale DO with inflight request won’t throw error on call to D1/postgres?

WWalshy <https://github.com/cloudflare/workerd/issues/2240>

Ashley•2/25/25, 10:14 AM

Thank you, glad it wasn’t just me!

Ccloudkite Yeh but they may have left/exited in meantime and I still need that state to be ...

James•2/25/25, 3:48 PM

system design should always consider recoverability from network failures regardless of initiator, even if you ran managed postgres on the same machine in some other VPS and the db craps out due to some memory error, your client will need to know how to handle the error and possibly retry the transaction

JJames system design should always consider recoverability from network failures regard...

cloudkiteOP•2/25/25, 5:02 PM

You’re missing the point. Yes I can if it’s a normal error. But not if it’s an error due to stale DO

Ccloudkite You’re missing the point. Yes I can if it’s a normal error. But not if it’s an e...

James•2/25/25, 5:03 PM

maybe im missing something, but doesnt the client get thrown an error regardless of the condition, such as stale DO?

JJames maybe im missing something, but doesnt the client get thrown an error regardless...

cloudkiteOP•2/25/25, 5:05 PM

I can’t rely on client to retry as they may have left/disconnected. if normal error is thrown can retry in the DO

James•2/25/25, 5:07 PM

so you're saying the same code update kills both the client and DO and you're left with the front-end as the final point of recoverability?

cloudkiteOP•2/25/25, 5:08 PM

No saying that can’t rely on client to retry as they may have close browser. If they still there can reconnect and retry. But no guarantee that they are still around

James•2/25/25, 5:12 PM

I don't see this as any different than other types of transactional processing and edge case mgmt. There's a reason half the ecommerce sites up there have a big banner saying 'dont close or refresh this page or else!'. Somewhere along the line we as system designed have to design around failure or data inconsistency, regardless where in the network it happens.

I agree it could be better, but I don't think it's too unusual. If I were concerned about this on my app, i'd probably just try to isolate critical transaction paths to recoverable workflows.

cloudkiteOP•2/25/25, 5:17 PM

By definition it’s unusual because if you used different DB like D1/postgres you wouldn’t have this specific issue

Ccloudkite By definition it’s unusual because if you used different DB like D1/postgres you...

James•2/25/25, 5:19 PM

tradeoffs... your postgres can die for all kinds of reasons

also D1 is no different than DO in that regard, just happens to not be tied to your code base, but then again you could just use DO storage in a different binding to de-couple your websocket handler from storage similar to the D1 or postgres architecture

Ccloudkite By definition it’s unusual because if you used different DB like D1/postgres you...

lambrospetrou•2/25/25, 10:55 PM

You keep mentioning D1/Postgres, but it's not the same. In the scenario you bring up you mention webscockets where the client disconnected but you want the last updates they sent to be persisted guaranteed. Other databases, including D1/Postgres do not guarantee this at all.
Firstly, if the client disconnects, most probably at some point the connection will close too, regardless if the websocket server dies or not. If the client disconnected before their updates were acknowledged, then they are not guaranteed to be persisted and processed. This is not DO specific, it's general systems.

In general, if the client does not wait for ACK, then nothing is guaranteed. No matter the system. Unless you put the update in a queue and process it asynchronously in the background in the server.

Llambrospetrou You keep mentioning D1/Postgres, but it's not the same. In the scenario you brin...

cloudkiteOP•2/26/25, 12:13 AM

I think the confusion is that I’m talking specifically about stale DO from a code update scenario inserting into this.ctx.storage.sql where as it seems you are talking about general errors/disconnects which I do have covered

cloudkiteOP•2/26/25, 12:28 AM

I really just wanted to know if what the cloudflare team member suggested (https://community.cloudflare.com/t/durable-objects-state-during-lifecycle-events/319932/4) was ever implemented:

"I could imagine some users would rather old requests have a chance to complete (including storage operations), and new ones would have to wait on the old object to complete work before they began processing."

sounds like it hasnt, so that answers my question, happy to put this to bed.

King Kazu•2/26/25, 5:47 AM

since dos are single threaded, is it recommended to run AI functions inside a do or should it be done from inside a worker then push updates to the do datastore?

KKing Kazu since dos are single threaded, is it recommended to run AI functions inside a do...

1984 Ford Laser•2/26/25, 7:38 AM

Run in Worker 100%

Nimesh•2/26/25, 2:41 PM

Hi, I asked a question in #workers-ai but think it might be more appropriate to share here. I checked the source code of the new agents-sdkagents-sdk (congrulations on the launch) and it looks like it’s based on Durable Objects and one of the examples makes a fetch request to an AI provider, but aren’t Durable Objects charged based on wall time? I was curious if it makes sense to make these these long running AI requests from a DO from a pricing point of view. Is this expected to change anytime soon?

KKing Kazu since dos are single threaded, is it recommended to run AI functions inside a do...

James•2/26/25, 4:23 PM

you pay for wall time on a DO, so yeah like he said, use a worker for API calls

Super Morris•2/26/25, 7:32 PM

I have one very important question:

We tried to use DO+SQLite and built a test worker. Ever first request from around the world is about ~1 second. then its very fast ~60ms. after a minute or so its back to 1 second.

We will have a lot instances for each users app and having "coldstarts" of 1s for every DO instance for every worker location is very very bad for us.... Is there any workaround/setting ?

Please help :NotLikeThis:

Fernando Barroso•2/26/25, 8:28 PM

Hello! Does anyone know how long limit increase request takes? I submitted a request to increase SQLite storage size using this form like 2 weeks ago and I haven’t heard anything:

https://forms.gle/ukpeZVLWLnKeixDu7

Google Docs

Cloudflare Developer Platform: Limit Increase Request Form

Want to increase the limits of your current plan? Let us know what your requirements are here.

FFernando Barroso Hello! Does anyone know how long limit increase request takes? I submitted a req...

sathoro•2/27/25, 1:08 AM

it is a black hole.

SSuper Morris I have one very important question: We tried to use DO+SQLite and built a test ...

Super Morris•2/27/25, 11:20 AM

Why is there a cold start of 1+ seconds after a few minutes of idle? (DO + SQLite)

SSuper Morris Why is there a cold start of 1+ seconds after a few minutes of idle? (DO + SQLit...

1984 Ford Laser•2/27/25, 11:28 AM

Because instances need to be re-initialized as they will get ejected after ~5 seconds or so of inactivity. Maybe you need to rethink your approach to implementing DOs?

Super Morris•2/27/25, 12:03 PM

@1984 Ford Laser Thank you!
But that is for every worker location + durable object combination... no way to prevent this?
I mean what is the point of a very fast vertical and horizontal scaleble DB that needs 1s+ cold start on every 280+ server for every DB, every 5 seconds? wtf...

FFernando Barroso Hello! Does anyone know how long limit increase request takes? I submitted a req...

DaniFoldi•2/27/25, 12:55 PM

I don't think that specifically the sqlite size limit can be increased today, sorry

DDaniFoldi I don't think that specifically the sqlite size limit can be increased today, so...

boop•2/27/25, 1:59 PM

I’m going to assume that he means the account storage limit

Good•2/27/25, 2:05 PM

can i have some durable object limit just to test things out something its for a school i would appreciate if you guys can help out

Original message was deleted

Good•2/27/25, 3:01 PM

imagine using paid worker program for testing out starbase db and testing out stuff

Good•2/27/25, 3:01 PM

i am not even sure will be using do but it seem great product

Original message was deleted

Good•2/27/25, 3:53 PM

it is but you can atleast set some minimum limits for dev purposes though very helpful

Good•2/27/25, 3:54 PM

i can just request / suggest but you have to try if you do not try you won't know

Original message was deleted

Good•2/27/25, 3:56 PM

can you please look into this https://starbasedb.com/ and confirm i can test this out locally

Good•2/27/25, 3:56 PM

i couldn't find anything the main page is just deploy and connect

SSuper Morris @1984 Ford Laser Thank you! But that is for every worker location + durable obj...

1984 Ford Laser•2/27/25, 8:47 PM

What? They don't restart 280+ instances because each DO instance is only located in 1 colo at a time. You severely misunderstand the platform

11984 Ford Laser What? They don't restart 280+ instances because each DO instance is only located...

Super Morris•2/27/25, 9:05 PM

while testing yesterday we requested a worker from different locations subsequently and there where cold start times of 1+ seconds from each location. might have been bad timing... today we saw that it only needs to cold start once for each DO instance globally, however about 30seconds later the same thing... so still 1 second of wait for each DO that is idle for more than 30 seconds... thats just stupid for our use-case.
we will use neon + hyperdrive for now.... 15-30 ms read and write. both from us and eu (for a eu hosted db)... no cold starts

SSuper Morris while testing yesterday we requested a worker from different locations subsequen...

1984 Ford Laser•2/27/25, 9:12 PM

Sounds like you are better suited to a single DB versus lots of smaller DBs via DO, not every backend architecture will be suitable for DOs

Iggy•2/27/25, 9:43 PM

I have a very basic worker + sqlite backed DO that is reading 10 rows. I'm getting ~150ms in round trip time from it (the SQL duration is <1ms), and coworkers across the country are getting >=600ms consistently, even when hitting local data centers. Shouldn't they all be roughly the same time? What's causing the huge discrepancy?

IIggy I have a very basic worker + sqlite backed DO that is reading 10 rows. I'm getti...

1984 Ford Laser•2/27/25, 9:47 PM

Are your coworkers hitting the same DO instance as you? Instances exist only in 1 colo so maybe they are seeing an unfavourable route to the instance, maybe get them to request a new instance, provide a location hint, and see if they get faster times

Iggy•2/27/25, 9:48 PM

We're both executing the exact same curl if that's what you're asking?

I need to run async things like vectorize, openai interleaved with sql queries though? transactions

Similar Threads