AFAIK, no, but it doesn’t have to, since the storage is synchronous anyway

HHello, I’m Allie!AFAIK, no, but it doesn’t have to, since the storage is synchronous anyway

Larry•10/2/24, 12:44 AM

I'm concerned about consistency. As a made up example, let's say I have to update the account balance which is a row in the accounts table whenever there is a withdrawl which is a row in the withdrawls table. If I do the one and something crashes before I can complete the other, does the 1st get rolled back?

LLarry I'm concerned about consistency. As a made up example, let's say I have to updat...

Isaac McFadyen•10/2/24, 12:57 AM

Theoretically this should be done in a batch (where a batch using the DO SQLite API is just semicolon-seperated statements I believe), and if it's done in a batch then yes it's automatically done as part of a SQLite transaction and would be rolled back.

IIsaac McFadyen Theoretically this should be done in a batch (where a batch using the DO SQLite ...

lambrospetrou•10/2/24, 9:33 AM

Yes, using the

.batch([query_1, query_2])

.batch([query_1, query_2])

function will use SQLite transactions under the hood so you can make sure both queries either commit or rollback. https://developers.cloudflare.com/d1/build-with-d1/d1-client-api/#dbbatch
edit: Oops, sorry, I thought you were talking about D1. But the same works for the DO by what Isaac mentioned.

Llambrospetrou Yes, using the `.batch([query_1, query_2])` function will use SQLite transactio...

Hello, I’m Allie!OP•10/2/24, 9:45 AM

Actually, haven't checked yet, but does

storage.transaction

storage.transaction

work with SQLite-engine DOs?

HHello, I’m Allie!Actually, haven't checked yet, but does `storage.transaction` work with SQLite-e...

Larry•10/2/24, 12:17 PM

storage.transaction

storage.transaction

working for "SQLite-engine DOs" is great! However, is that just for the old

get()

get()

put()

put()

, etc. calls or does it work for SQL statements also?

LLarry `storage.transaction` working for "SQLite-engine DOs" is great! However, is that...

Hello, I’m Allie!OP•10/2/24, 12:18 PM

Uh... I phrased that weirdly... I hope Lambros meant both?

Hello, I’m Allie!OP•10/2/24, 12:18 PM

Though not sure, might be a case of try it and see?

LLarry `storage.transaction` working for "SQLite-engine DOs" is great! However, is that...

lambrospetrou•10/2/24, 2:14 PM

It should work for everything. In SQLite DOs, the KV API is actually using the same SQLite database with a "secret-hidden-not-so-much" table automatically created in the same SQLite database. So, both the KV and SQL API are consistent with each other.
I wouldn't suggest mixing them arbitrarily though since I am not sure how multiple transactions would work, e.g. nesting them.

Also, as long as your are doing storage writes without any IO inbetween (e.g.

fetch

fetch

), you shouldn't even need explicitly making transactions. But anyway, without concrete examples, this is mostly theory so I will stop here

Llambrospetrou It should work for everything. In SQLite DOs, the KV API is actually using the s...

Hello, I’m Allie!OP•10/2/24, 2:20 PM

I think the idea was more that you have rollback support? Not sure if I've missed something, but don't think the SQLite API itself has anything for that?

Hhannes short question: D1 docs say "use prepare() for perf and defend against SQL injec...

lambrospetrou•10/2/24, 3:56 PM

The

.exec()

.exec()

method does the prepare statements for you. Soon they will be cached automatically as well.

Llambrospetrou It should work for everything. In SQLite DOs, the KV API is actually using the s...

Larry•10/2/24, 8:43 PM

It's going to be impossible for me to confirm this experimentially, so @kenton , does this hold true for

.sql.exec()

.sql.exec()

If you invoke put() (or delete()) multiple times without performing any await in the meantime, the operations will automatically be combined and submitted atomically. In case of a machine failure, either all of the writes will have been stored to disk or none of the writes will have been stored to disk.

xeon06•10/2/24, 8:58 PM

Hey folks, I'm confused about durable object bindings for Pages. I'm trying to setup the binding but there are no durable objects I can pick in the list, and when I go to the durable object section of the dashboard there's no obvious way to create them?

Xxeon06 Hey folks, I'm confused about durable object bindings for Pages. I'm trying to s...

Larry•10/2/24, 9:00 PM

There was talk of being able to create DOs in a Pages project and that may have been implemented since I last paid close attention, but if not, you need a plain old Workers project (with an empty Worker, if you wish) to deploy DOs.

xeon06•10/2/24, 9:02 PM

Wow I'm not sure how I completely missed that it wasn't available on Pages

xeon06•10/2/24, 9:03 PM

I am not seeing any mention of Pages in the docs now that you mention it, only Workers

Larry•10/2/24, 9:04 PM

One of the complexities had to do with the fact that DOs must specify migrations in the .toml file and .toml files weren't supported by Pages back then. I think I saw where that changed at some point, but that could also just be something I heard was planned.

xeon06•10/2/24, 9:06 PM

I believe toml files are now supported but am similarly not sure of how that impacts things here. Thanks for the comments though!

xeon06•10/2/24, 9:08 PM

https://blog.cloudflare.com/pages-workers-integrations-monorepos-nextjs-wrangler/

The Cloudflare Blog

Announcing Pages support for monorepos, wrangler.toml, database int...

Today, we’re launching four improvements to Pages that bring functionality previously restricted to Workers, with the goal of unifying the development experience between the two. Support for monorepos, wrangler.toml, new additions to Next.js support and database integrations!

xeon06•10/2/24, 9:09 PM

Maybe I just need to recheck my deploy script, it might just be using the old "way"

LLarry One of the complexities had to do with the fact that DOs must specify migrations...

DaniFoldi•10/2/24, 9:23 PM

That wasn't the complexity with Pages, and the compatibility matrix shows that defining DOs is still not supported in Pages: https://developers.cloudflare.com/workers/static-assets/compatibility-matrix/#_top

Cloudflare Docs

Workers vs. Pages (compatibility matrix) | Cloudflare Workers docs

Compatibility matrix for asset hosting on Cloudflare Workers and Pages.

LLarry It's going to be impossible for me to confirm this experimentially, so @kenton ,...

kenton•10/2/24, 9:40 PM

yes, this applies to sql.exec()sql.exec().

DDaniFoldi That wasn't the complexity with Pages, and the compatibility matrix shows that _...

xeon06•10/2/24, 10:32 PM

Implying they can be defined in workers and used in pages? Do I just need to add them to a worker's wrangler.toml?

Larry•10/2/24, 10:35 PM

Yes, they can be defined in a Worker, then bound to the Pages app in the console (but maybe that can now be done in the .toml, I don't know), then called from the Pages app.

xeon06•10/2/24, 10:48 PM

Ahh I see, it looks like I can probably bind it directly in the UI in Pages now, just didn't know how to "define" the worker!

Larry•10/2/24, 10:50 PM

That's well covered in the getting started for DOs

Benjamín Vicente 🇨🇱•10/2/24, 11:15 PM

Is https://github.com/evanderkoogh/do-taskmanager still the recommended way to do tasks?

For context, I had planed to use setTimeout with no more that 10s to run tasks after a RPC call (that could work perfectly to the 30s per request limit right now), but in a testing environment, the timeout seems to be cleared/evicted immediately.

Edit with more context: What I am building requires debouncing a task.
For example, a real time editor that generates a PDF from the edited content. Generating a PDF on each change doesn't make since when multiples changes are made per second and generating a PDF takes multiple seconds. So I want to generate a PDF after X seconds of each change, and cancel the previous task if a change is made (ideally, with access to an abort controller to cancel internal steps).
My initial idea is that a DO holds multiple text editors (like a "workspace") and multiple clients (web browsers via WS). A workers call the DO to connect via WS, and the DO handles text change.

GitHub

GitHub - evanderkoogh/do-taskmanager: A TaskManager for Durable Obj...

A TaskManager for Durable Objects, simplifying the use of Alarms - evanderkoogh/do-taskmanager

Mmurlax Do we have any workaround for moving DO locations given that it is not natively ...

BadMatter•10/3/24, 4:08 PM

I also thought about this as a workaround for automatic relocation. I would like to check for the current location and if it is not optimal recreate the DO. Is this somehow possible? Help would be much appreciated.

BBadMatter I also thought about this as a workaround for automatic relocation. I would like...

Hello, I’m Allie!OP•10/3/24, 4:16 PM

Sort of? The biggest question is how you propagate the new ID

zX0•10/3/24, 6:19 PM

Quick note to the crew working on DO . . . this SQLite drop is rad , thank you!

SSibi We built DO relocation ourselves by checking the distance between the datacenter...

Jehu•10/4/24, 4:16 PM

It’d be great to see this open source( code or design)

SSibi Unfortunately this functionality is quite deeply baked into our services. But I ...

Jehu•10/4/24, 4:24 PM

Hi level pattern, dos and don’ts would be fantastic for us newbs!

James•10/4/24, 4:25 PM

just to confirm sql.exec and batching 'transactions'. If I have something like a big fat DDL migration sql script run in a single exec. lets say the first two statements pass (create table a,b) but I fail the 3rd due to bad sql or DDL failure (table C already exists), then the whole thing rolls back?

Ashish Rawat•10/5/24, 5:49 AM

So, I want to know because durable object has 1GB limit per object. Let's say if I store a user data per object in durable object, then later if I want to fetch let's say 20 user data all at once (for listing on frontend). How to do that

AAshish Rawat So, I want to know because durable object has 1GB limit per object. Let's say if...

James•10/5/24, 5:51 AM

you need a deterministic way of accessing your durable objects, likely a central db keeping track of IDs, you'd have to hit every stub with whatever query you want and join the results

Ashish Rawat•10/5/24, 5:52 AM

so basically aggregate and manual sharding via timestamp

Ashish Rawat•10/5/24, 5:53 AM

then I'll wait for durable object with sqlite to get more mature until sharding is automatic. The same is the case with D1.

Ashish Rawat•10/5/24, 5:54 AM

This is very confusing for sqlite

Screenshot_2024-10-05_at_11.24.17_AM.png

matt•10/5/24, 6:39 AM

can you test a durable object locally without running workerd? perhaps just a plain JS implementation of the parent DurableObject class?

Mmatt can you test a durable object locally without running workerd? perhaps just a pl...

Hello, I’m Allie!OP•10/5/24, 6:49 AM

You can run Miniflare/wrangler yourself, but it still uses workerd under the hood

Hello, I’m Allie!OP•10/5/24, 6:50 AM

May I ask why specifically you want to avoid workerd?

matt•10/5/24, 6:54 AM

im migrating to bun and normally tests run so fast you can enable save on keypress in vscode and watch your tests run as you're typing

matt•10/5/24, 6:55 AM

but you wouldn't want any external libs (workerd) spinning up on each keypress. i think the way i'll get around it is taking all the business logic out of the DO into a regular class and just test that

AFAIK, no, but it doesn’t have to, since the storage is synchronous anyway

Similar Threads

Similar Threads

Similar Threads