Cloudflare Developers•13mo ago

Yeah, for now it has to be declared separately

phil•12/28/24, 6:41 PM

Is there anyway to get a dump of the SQLite db of a DO?

Pphil Is there anyway to get a dump of the SQLite db of a DO?

Brayden•12/29/24, 3:45 AM

You could try copying logic from this repo that handles data dump from DO’s, or connect your DO up to a deployment of this and have access to it via an HTTP route.

Most of the goodies of interacting with the data layer of a durable object are left for us to build on. D1 has some of the prebuilt interactions like dumps built out for it.

https://github.com/outerbase/starbasedb/tree/main/src/export

GitHub

starbasedb/src/export at main · outerbase/starbasedb

HTTP SQLite scale-to-zero database on the edge built on Cloudflare Durable Objects. - outerbase/starbasedb

peachneo•12/29/24, 6:09 AM

I take it the idFromName still requires the provided string to be universally unique? hashes it to some DO compatible id? Or is it hashing it with some additional data to ensure global uniqueuness?

Ppeachneo I take it the idFromName still requires the provided string to be universally un...

Hello, I’m Allie!OP•12/29/24, 6:30 AM

It doesn’t have to be unique, but if you use the same Name twice then you hit the same object

HHello, I’m Allie!It doesn’t *have* to be unique, but if you use the same Name twice then you hit ...

peachneo•12/29/24, 7:22 AM

But it's isolated per "tenant" / "account" / "zone" right?

Ppeachneo But it's isolated per "tenant" / "account" / "zone" right?

Hello, I’m Allie!OP•12/29/24, 7:38 AM

Per DO Namespace

BBrayden You could try copying logic from this repo that handles data dump from DO’s, or ...

Burrito•12/29/24, 10:52 AM

Does this work with large databases? This seems to construct the entire dump in memory.

HHello, I’m Allie!Yeah, for now it has to be declared separately

Isak•12/29/24, 11:39 AM

Any workaround for this atm? I'm thinking of a separate worker during dev with a temporary main file that exports the dos.

BBurrito Does this work with large databases? This seems to construct the entire dump in ...

Brayden•12/29/24, 1:16 PM

For large databases we’d need to construct some sort of streaming response which shouldn’t be too difficult. I can add it to my list to add in.

BumblebeeSquare•12/29/24, 6:34 PM

hi all, is there a handy function i can call to destory the DO ? sometime i use DO for a quick task then i need to destroy it to avoid charge ... clearing all key value storage & sqlite myself is annoying..

BBumblebeeSquare hi all, is there a handy function i can call to destory the DO ? sometime i use ...

lucidplot•12/29/24, 6:41 PM

deleteAll
deleteAll(options Object optional): Promise

Deletes all stored data, effectively deallocating all storage used by the Durable Object. For Durable Objects with a key-value storage backend, deleteAll() removes all keys and associated values for an individual Durable Object. For Durable Objects with a SQLite storage backend, deleteAll() removes the entire contents of a Durable Object's private SQLite database, including both SQL data and key-value data.

https://developers.cloudflare.com/durable-objects/api/storage-api/#deleteall

Cloudflare Docs

Durable Object Storage · Cloudflare Durable Objects docs

The Durable Object Storage API allows Durable Objects to access transactional and strongly consistent storage. A Durable Object's attached storage is private to its unique instance and cannot be accessed by other objects.

peachneo•12/29/24, 11:56 PM

Given two different durable object classes DOA and DOB, and a third DOC.

DOA and DOB share a common interface via a method (let's say receiveEvent), otherwise are different.

DOC only cares about that interface and wants to store references to the objects in a list and call the receiveEvent method on them.

It seems to me this isn't particularly straightforward because classes are tied to namespaces.
The references stored in the list would have to be ID + namespace identifier, and because namespaces are objects I will require some mapping somewhere i.e. "MY_NAMESPACE_STRING" -> env.MY_NAMESPACE

peachneo•12/29/24, 11:58 PM

Alternatively I can model all my DOs under a single class - eg "Actor", and then effectively re-implement my own RPC layer using fetch or layered on top of CFs RPC, but then that feels wrong to me

Ppeachneo Given two different durable object classes DOA and DOB, and a third DOC. DOA an...

James•12/30/24, 1:24 AM

Use a deterministic ID.

Ppeachneo Alternatively I can model all my DOs under a single class - eg "Actor", and then...

Larry•12/30/24, 1:41 AM

If you decide to go the single DO route, keep in mind that different instances of the same DO class can be loaded in the same memory, and if that happens, they will have to share a single pool of 128MB.

LLarry If you decide to go the single DO route, keep in mind that different instances o...

peachneo•12/30/24, 1:47 AM

Hmm is there some documentation detailing this?
If we can't make reliable assumptions about memory availability with > 1 DO namespace instance that seems problematic.

peachneo•12/30/24, 5:16 AM

If I have multiple DDL statements in a call to "sql.exec()" is that wrapped in a transaction automatically?

Ggnomesgames From my understanding, if you use SQL in DO, then call to the underlying SQL DB ...

Larry•12/31/24, 4:40 PM

It’s isolated (the “I” in ACID) from other transactions because as you say it’s essentially single threaded. However, if a later separate sql.exec() fails for some reason, it won’t roll back the earlier ones. Failing the “A” (atomic) in ACID. You have to explicitly wrap it in a transaction to achieve that.

Ppeachneo If I have multiple DDL statements in a call to "sql.exec()" is that wrapped in a...

Larry•12/31/24, 4:48 PM

Not completely. See durable-objects

LLarry Not completely. See https://discord.com/channels/595317990191398933/77321944391...

Brayden•12/31/24, 4:49 PM

Here's a further example in action to help demonstrate as well:

public executeTransaction(
    queries: { sql: string; params?: unknown[] }[]
): unknown[] {
    return this.storage.transactionSync(() => {
        const results = []

        try {
            for (const queryObj of queries) {
                const { sql, params } = queryObj
                const result = this.executeQuery({ sql, params })
                results.push(result)
            }

            return results
        } catch (error) {
            console.error('Transaction Execution Error:', error)
            throw error
        }
    })
}

public executeTransaction(
    queries: { sql: string; params?: unknown[] }[]
): unknown[] {
    return this.storage.transactionSync(() => {
        const results = []

        try {
            for (const queryObj of queries) {
                const { sql, params } = queryObj
                const result = this.executeQuery({ sql, params })
                results.push(result)
            }

            return results
        } catch (error) {
            console.error('Transaction Execution Error:', error)
            throw error
        }
    })
}

HHello, I’m Allie!Yeah, for now it has to be declared separately

Isak•1/1/25, 12:21 PM

Hey again is there some issue or pr about this that I can track?

Jacob Wright•1/1/25, 9:16 PM

I'm interested in using the new SQLite DO, and I was wondering if there is a serialize/deserialize package to store unstructured data in a SQLite blob more efficiently than a json text field.

peachneo•1/2/25, 1:31 AM

if I call .get() on a DO namespace to get a stub, as I understand it - this will awake the object, but eventually the object should be cleaned up as nothing is committed to storage? (i.e I won't end up with dangling objects/ no need for an explicit clean up unless a call to storage is made?)

Ppeachneo if I call .get() on a DO namespace to get a stub, as I understand it - this will...

Larry•1/2/25, 2:18 AM

I don't know if the awakening (instantiating) occurs when you get the stub. My gut says that happens when you 1st call it. Regardless, your understanding about not needing to clean anything up unless something is saved to storage is correct.

LLarry I don't know if the awakening (instantiating) occurs when you get the stub. My g...

peachneo•1/2/25, 2:19 AM

great, thank you

anirudh•1/2/25, 6:42 AM

Is there a way to prevent a durable object from spawning in a certain location like Hong Kong? Reason - we are using Anthropic API (LLM) and they don't seem to allow API access from that region. I've looked at the docs and it doesn't seem to be possible.

Aanirudh Is there a way to prevent a durable object from spawning in a certain location l...

Max (@rozenmd)•1/2/25, 11:57 AM

not that I know of, but you can fetch

https://cloudflare.com/cdn-cgi/trace

https://cloudflare.com/cdn-cgi/trace

from the DO to work out where it is and return early if in HKG

Bbanny`Hello 👋 , are there any guides/examples on how one might do an `transfer` migra...

Beefp•1/2/25, 5:14 PM

sorry for the bump but did you find a solution to this?

ᴀ

ᴀkqj•1/2/25, 10:58 PM

i'm trying to use DO with sqlite

wrangler dev --remote

wrangler dev --remote

, to connect with KV

[[durable_objects.bindings]]
name = "MY_DURABLE1"
class_name = "DO1"

[[durable_objects.bindings]]
name = "MY_DURABLE2"
class_name = "DO2"

[[migrations]]
tag = "v2"
new_sqlite_classes = ["DO1", "DO2"]

[[durable_objects.bindings]]
name = "MY_DURABLE1"
class_name = "DO1"

[[durable_objects.bindings]]
name = "MY_DURABLE2"
class_name = "DO2"

[[migrations]]
tag = "v2"
new_sqlite_classes = ["DO1", "DO2"]

but running into

✘ [ERROR] Uncaught (in promise) Error: Error: This Durable Object is not backed by SQLite storage, so the SQL API is not available. SQL can be enabled on a new Durable Object class by using the `new_sqlite_classes` instead of `new_classes` under `[[migrations]]` in your wrangler.toml, but an already-deployed class cannot be converted to SQLite (except by deleting the existing data).

✘ [ERROR] Uncaught (in promise) Error: Error: This Durable Object is not backed by SQLite storage, so the SQL API is not available. SQL can be enabled on a new Durable Object class by using the `new_sqlite_classes` instead of `new_classes` under `[[migrations]]` in your wrangler.toml, but an already-deployed class cannot be converted to SQLite (except by deleting the existing data).

is there a way to delete the DO since I don't see any DO in dashboard or how to mitigate this error?

Fernando Barroso•1/3/25, 7:29 PM

Hello all! Does anyone know if there’s a way to pass W3C Trace Context headers to a durable object when calling it via RPC? Or would it be necessary to use vanilla fetch with HTTP Headers for that?

Jürgen Leschner•1/3/25, 9:05 PM

FYI - this is just a heads up regarding Astro and Durable Objects
It appears that the current astro/cloudflare adapter is unable to build a worker which exports classes for durable objects
This means that even if you use workers assets with Astro, you still can't combine your durable object code with other Astro SSR code in one worker.
I have filed https://github.com/withastro/adapters/issues/494
Repro in https://github.com/jldec/astro-do/pull/1

GitHub

Cloudflare adapter is unable to export Durable Object classes · Iss...

Astro Info Astro v5.1.2 Node v22.11.0 System macOS (arm64) Package Manager pnpm Output server Adapter @astrojs/cloudflare Integrations @astrojs/tailwind Describe the Bug When attempting to use Dura...

GitHub

attempt to add durable object - not working by jldec · Pull Request...

pnpm dev currently fails with
20:15:59 [WARN] [config] The adapter @astrojs/cloudflare has limited support for "sharp". Certain features may not work as expected.
20:15:59 [types]...

Kalu•1/4/25, 9:03 PM

Hello,
I don't understand the size limit of the store, I'd like to make a figma-like, so I have a big json from 100k to 3Mo that I need to be able to mutate, as main backup I use R2, for use with hibernation websocket I need to put it in the store but from what I see you can only put value a 128 Ko.
Is that really the limit? Is Durable object really usable for this situation?

KKalu Hello, I don't understand the size limit of the store, I'd like to make a figma-...

1984 Ford Laser•1/4/25, 10:11 PM

Can you split big json into multiple parts?

Kalu•1/4/25, 10:15 PM

This could make the coding too complicated

christianklotz•1/4/25, 10:53 PM

First time playing with DOs here but in the context of workers for platforms.

Do I understand this correctly that DOs cannot be deployed with (an essentially untrusted) user script upload but they have to be defined as part of the main app?

So if I want to give a user script access to DOs, this would be done via bindings somehow?

Trying to wrap my head around this. Is there an example of how a

sayHello

sayHello

RCP or the alarm API could be called from a WFP user script?

Ggnomesgames It does! You can use the `script_name` field of your DO binding to point to an e...

WonderPandaDev•1/5/25, 9:38 PM

Hey I've been trying to get this to work for a while now. I have two different workers and I want to be able to connect to a Durable Object defined in Worker B from Worker A. I've got both the script_name set as well as the service binding defined but when I run wrangler for for both separate projects the DO bindings always shows as [not connected]

WonderPandaDev•1/5/25, 9:38 PM

shubhamvscode•1/6/25, 8:37 AM

Hey,

Is there any timelimit for a durable object ?

Like i want to do something like:
sending email after 5 min of user signup so it's not working but if i do 10sec it works

Sshubhamvscode Hey, Is there any timelimit for a durable object ? Like i want to do something...

lucidplot•1/6/25, 8:56 AM

you can use a DO alarm or a queue

Fernando Barroso•1/6/25, 4:32 PM

hello all! In all examples I see alarms being scheduled for several seconds in the future. If I wanted to schedule immediate async work (i.e.

await this.ctx.storage.setAlarm(Date.now());

await this.ctx.storage.setAlarm(Date.now());

) could I run into any issues? Are there any gotchas here? I have been using this and it works, however, I have encountered (what seems to be) race conditions with this async work so I wonder if that immediate alarm has anything to do with this.

WWonderPandaDev Hey I've been trying to get this to work for a while now. I have two different w...

James•1/6/25, 4:35 PM

for local dev, route DO requests to the DO fetch worker instead of the direct binding since multiple DO bindings local doesn't really work

FFernando Barroso hello all! In all examples I see alarms being scheduled for several seconds in t...

James•1/6/25, 4:40 PM

maybe, since the whole DO is single threaded, you could run into scaling issues with that vs using a queue and worker instead, also DO CPU is more expensive than a worker

danjenkins•1/6/25, 11:20 PM

I swear there used to be a way to assign metadata to a websocket in durable objects and then find all the websockets that had that meta data? I can't find any docs on it now?

danjenkins•1/6/25, 11:22 PM

ah just like that found it

JJames for local dev, route DO requests to the DO fetch worker instead of the direct bi...

WonderPandaDev•1/7/25, 3:37 AM

Hey thanks for the tip I'll try this out! Is this documented anywhere? Still trying to ramp up on the expected experience when developing locally. Just to make sure I understand, are you saying I should keep the service binding and use that to forward the Request to the worker running in a separate wrangler dev process and then that can pass the request to the DO?

James•1/7/25, 4:18 AM

I made a proxy obj in front of the binding in my main app. If I’m running local, the proxy sends a fetch to localhost separate wrangler running the DO with the same payload and response signature. If not local, the proxy just calls the binding directly.

The fetch handler on the DO only runs locally, and instantiates the stub directly based on the request body.

Ddanjenkins I swear there used to be a way to assign metadata to a websocket in durable obje...

1984 Ford Laser•1/7/25, 6:56 AM

for anyone looking for this in the future, you do

this.state.acceptWebSocket(socket, topic)

this.state.acceptWebSocket(socket, topic)

, then to find connections by

topic

topic

you would do

this.state.getWebSockets(topic)

this.state.getWebSockets(topic)

11984 Ford Laser for anyone looking for this in the future, you do `this.state.acceptWebSocket(so...

danjenkins•1/7/25, 1:04 PM

my bad, in my haste I forgot to mention how its done!

BBrayden Here's a further example in action to help demonstrate as well: ``` public exec...

Larry•1/7/25, 1:59 PM

Thanks for giving that example. It's perfect for when you have an array of SQL statements you want to execute in a transaction. You can also take advantage of my favorite thing about the DO concurrency model by intermixing queries and code using

this.storage.transactionSync(() => {...})

this.storage.transactionSync(() => {...})

directly. In the callback, you can do a query, process it with code, do another query, etc. Then, every storage operation in that callback will either all-succeed or all-fail.

public executeTransaction( queries: { sql: string; params?: unknown[] }[] ): unknown[] { return this.storage.transactionSync(() => { const results = [] try { for (const queryObj of queries) { const { sql, params } = queryObj const result = this.executeQuery({ sql, params }) results.push(result) } return results } catch (error) { console.error('Transaction Execution Error:', error) throw error } }) }

[[durable_objects.bindings]] name = "MY_DURABLE1" class_name = "DO1" [[durable_objects.bindings]] name = "MY_DURABLE2" class_name = "DO2" [[migrations]] tag = "v2" new_sqlite_classes = ["DO1", "DO2"]

✘ [ERROR] Uncaught (in promise) Error: Error: This Durable Object is not backed by SQLite storage, so the SQL API is not available. SQL can be enabled on a new Durable Object class by using the `new_sqlite_classes` instead of `new_classes` under `[[migrations]]` in your wrangler.toml, but an already-deployed class cannot be converted to SQLite (except by deleting the existing data).

Yeah, for now it has to be declared separately

Similar Threads