can someone provide an example on this topic: How do I insert and then append a json object (not a s

can someone provide an example on this topic: How do I insert and then append a json object (not a scalar,) but a proper json object using json_insert at the end of an existing json array in D1? My objective is to store data like this in one text column: history: [{object1 with key and values}, {object2 with keys and values}]

Elodere•11/27/23, 10:12 AM

Does D1 support UUIDv7 or should I go with ULID for now?

EElodere Does D1 support UUIDv7 or should I go with ULID for now?

Rob M.•11/27/23, 10:29 AM

Why? UUIDv7 is still in a WIP state itself. Is it a game changer?

Elodere•11/27/23, 10:33 AM

UUIDv7 from what I've glanced of the spec is supposed to be a lot better than v6 (generally recommended to use v7 whenever possible basically)

RRob M.Why? UUIDv7 is still in a WIP state itself. Is it a game changer?

Elodere•11/27/23, 10:33 AM

I don't know enough to claim that it's a game changer, but it's definitely a significant improvement over previous versions

Elodere•11/27/23, 10:34 AM

hence why some people are looking at UUIDv7 as a possible replacement for using ULID

EElodere UUIDv7 from what I've glanced of the spec is supposed to be a lot better than v6...

Elodere•11/27/23, 10:44 AM

Also something I've realized is that the dashboard only seems to allow for string, int, blob, and real datatypes

But because D1 runs on SQL, I assume it should be fine to use datatypes like UUID right?

Elodere•11/27/23, 10:46 AM

Thank you I didn't realize it was SQLite

Elodere•11/27/23, 10:47 AM

Yeah then I was right in just using string as the type

EElodere I don't know enough to claim that it's a game changer, but it's definitely a sig...

Rob M.•11/27/23, 10:48 AM

D1 is a game changer dogekek

Elodere•11/27/23, 10:49 AM

Yeah the only thing I'm worried over is that because cloudflare pages is an edge based solution, that I'll have to think of distributed data from the get go

Then again, I did see it mentioned somewhere that D1 is transaction based, so race conditions will thankfully not happen

Stone•11/27/23, 10:49 AM

Does someone know if I use the FTS5 (full text search) extension in D1?

SStone Does someone know if I use the FTS5 (full text search) extension in D1?

Max (@rozenmd)•11/27/23, 11:31 AM

FTS5 is supported, yes

EElodere Yeah the only thing I'm worried over is that because cloudflare pages is an edge...

Max (@rozenmd)•11/27/23, 11:31 AM

Yep, we run every query in a transaction

Stone•11/27/23, 11:35 AM

@rozenmd That is beautiful! Goodbye 3rd party search engines, for small/medium sized data

Rob M.•11/27/23, 12:14 PM

I need to return a response from the worker asap and also run a write query to D1. Is it possible to not wait for the API call or maybe do it after the "return new Response()"? Using Queues is an option but not a desirable one.

RRob M.I need to return a response from the worker asap and also run a write query to D...

Sam•11/27/23, 12:19 PM

https://developers.cloudflare.com/workers/runtime-apis/handlers/fetch/#contextwaituntil

RRob M.250ms from the API or worker?

johnswarbrick•11/27/23, 1:29 PM

250ms is the time taken for the D1 database to respond with results back to my worker. Sometimes dips to 210ms, but it's quite consistent. Really hoping having read-replicas speeds this up.

Jjohnswarbrick 250ms is the time taken for the D1 database to respond with results back to my w...

Max (@rozenmd)•11/27/23, 1:39 PM

Are you using indexes?

Max (@rozenmd)•11/27/23, 1:41 PM

because fetching a specific record in the northwind db takes 0.2ms: https://northwind.d1sql.com/dash

johnswarbrick•11/27/23, 1:50 PM

Yes, I think it's networking latency.

The Cloudflare Worker is being spun up in the UK, because that's the closest CF endpoint.

But then the SQL query is being sent to wherever the D1 core is currently.

So even a trivial 1 row select from a 1 row table takes 200 - 250ms to return.

I've not tested from APAC yet, I guess it will end up 500-750ms from there.

I'm really hoping read-replicas fixes that problem, although I guess writes will still be slow.

Jjohnswarbrick Yes, I think it's networking latency. The Cloudflare Worker is being spun up in...

Matt Silverlock•11/27/23, 1:55 PM

D1 will locate the leader close to where you created it, or if you provide a hint, close to where you request: https://developers.cloudflare.com/d1/learning/data-location/

Jjohnswarbrick Yes, I think it's networking latency. The Cloudflare Worker is being spun up in...

Matt Silverlock•11/27/23, 1:56 PM

You can see where your leader is located with

wrangler d1 info DB_NAME

wrangler d1 info DB_NAME

e.g.

``
➜ npx wrangler@latest d1 info db-enam
┌───────────────────┬──────────────────────────────────────┐
│ ENAM │ c89db32e-83f4-4e62-8cd7-7c8f97659029 │
├───────────────────┼──────────────────────────────────────┤
│ name │ db-enam │
├───────────────────┼──────────────────────────────────────┤
│ created_at │ 2023-06-12T16:52:03.071Z │
├───────────────────┼──────────────────────────────────────┤
│ version │ beta │
├───────────────────┼──────────────────────────────────────┤
│ num_tables │ 8 │
├───────────────────┼──────────────────────────────────────┤
│ running_in_region │ ENAM │
├───────────────────┼──────────────────────────────────────┤
│ database_size │ 1.83 GB │
├───────────────────┼──────────────────────────────────────┤
│ read_queries_24h │ 1,437 │
├───────────────────┼──────────────────────────────────────┤
│ write_queries_24h │ 1,551,235 │
└───────────────────┴──────────────────────────────────────┘
```

johnswarbrick•11/27/23, 2:01 PM

Thanks Matt, I'll take a look

SSantosh can someone provide an example on this topic: How do I insert and then append a ...

SantoshOP•11/27/23, 2:46 PM

Can someone give me an example on how to insert and append Json using json_insert? I see the example from the documentation with scalar but not an object. And i receive the following error :

Error: D1_TYPE_ERROR: Type 'object' not supported for value '[object Object]'
    at cloudflare-internal:d1-api:146:19

Error: D1_TYPE_ERROR: Type 'object' not supported for value '[object Object]'
    at cloudflare-internal:d1-api:146:19

SSantosh Can someone give me an example on how to insert and append Json using json_inse...

Matt Silverlock•11/27/23, 4:16 PM

Can you show what you have tried first?

SantoshOP•11/27/23, 5:49 PM

      const h = {id: 1, attr1: "abcdef} const t = JSON.stringify(h)
      //   console.log("JSON being inserted =>>>>", h)
        const result = await env.SMS_DEV.prepare(
          "UPDATE TestTable SET history = json_insert(history, '$.history[#]', '?1') WHERE id = '?2' ",
        )
          .bind(h, status.id)
          .run()

      const h = {id: 1, attr1: "abcdef} const t = JSON.stringify(h)
      //   console.log("JSON being inserted =>>>>", h)
        const result = await env.SMS_DEV.prepare(
          "UPDATE TestTable SET history = json_insert(history, '$.history[#]', '?1') WHERE id = '?2' ",
        )
          .bind(h, status.id)
          .run()

SantoshOP•11/27/23, 7:21 PM

Can someone point to Full Test search capabilities of D1? I couldn't locate any articles or discussions related to this

WoozyAnuki•11/27/23, 7:56 PM

what's the format for running SQL queries on D1 on PHP deployed as a project? I'm an absolute noob with web dev but using cloudflare to host a student project as well as my portfolio but for the project I'm requried to use SQL and saw D1 as an option so was wondering if it was possible to use d1 as I would use a normal SQL database (as in like XAMPP/MAMP)

WWoozyAnuki what's the format for running SQL queries on D1 on PHP deployed as a project? I'...

Hello, I’m Allie!•11/27/23, 8:04 PM

You can’t deploy PHP on Cloudflare

Hello, I’m Allie!•11/27/23, 8:04 PM

Or well, I think there is an adapter for it somewhere, but not sure it supports anything relatively modern

Hello, I’m Allie!•11/27/23, 8:05 PM

Or the PHP runtime someone wrote.

WoozyAnuki•11/27/23, 8:07 PM

Woof, thanks lol. Will still use it for my portfolio and just use R2 most likely since there's no extra mess lol. Tysm :)

SSantosh Can someone point to Full Test search capabilities of D1? I couldn't locate any ...

kian•11/27/23, 8:32 PM

It's SQLite's FTS5

SSantosh Can someone point to Full Test search capabilities of D1? I couldn't locate any ...

Rob M.•11/27/23, 10:16 PM

Look at this one:
https://www.sqlite.org/fts5.html

scotto•11/27/23, 10:43 PM

hi - any github example for a worker project with d1? need to build for mid-size project and a good example on how to structure the code would be great to start

RRob M.Look at this one: https://www.sqlite.org/fts5.html

SantoshOP•11/28/23, 2:03 AM

How do we use this on D1? An example would be perfect, not many of use SQLLite at work.

Village (Josef)•11/28/23, 4:59 AM

Two questions

Are all the limits except the DB size limit available to be potentially raised?
Is it feasible to make a DB per customer with D1? I.e. is there some way to dynamically set up bindings and such without having to manually put them all inside wrangler.toml and without incurring insane startup overhead or having to route through the API? (That was a lot of or’s sorry)

kian•11/28/23, 5:31 AM

Are all the limits except the DB size limit available to be potentially raised?

You'd need to be specific about the individual limit in question, but generally they aren't raised on a per-user basis and instead are global limits.

is there some way to dynamically set up bindings

Not currently.

Ravi kiran•11/28/23, 5:36 AM

guys can anyone tell me how i can use two d1 databases in a single worker?
i tried this in the toml file
name = "weathered-queen-4070"
main = "src/index.ts"
compatibility_date = "2023-10-30"

[[d1_databases]]
binding = "DB" # available in your Worker on env.DB
database_name = "dogpark"
database_id = "xxxxxxxxxxxx"

binding = "DB2" # i.e. available in your Worker on env.DB
database_name = "adminportalDB"
database_id = "xxxxxxxxxxx"

but then i get the error that Can't redefine existing key

/Users/apple/Documents/weathered-queen-4070/wrangler.toml:10:14:
10 │ binding = "DB2"

kian•11/28/23, 5:37 AM

name = "weathered-queen-4070"
main = "src/index.ts"
compatibility_date = "2023-10-30"

[[d1_databases]]
binding = "DB" # available in your Worker on env.DB
database_name = "dogpark"
database_id = "xxxxxxxxxxxx"

[[d1_databases]]
binding = "DB2" # i.e. available in your Worker on env.DB
database_name = "adminportalDB"
database_id = "xxxxxxxxxxx"

name = "weathered-queen-4070"
main = "src/index.ts"
compatibility_date = "2023-10-30"

[[d1_databases]]
binding = "DB" # available in your Worker on env.DB
database_name = "dogpark"
database_id = "xxxxxxxxxxxx"

[[d1_databases]]
binding = "DB2" # i.e. available in your Worker on env.DB
database_name = "adminportalDB"
database_id = "xxxxxxxxxxx"

Ravi kiran•11/28/23, 5:38 AM

oh thank you.

rodrigo•11/28/23, 8:55 AM

I understand when read replicas are implemented there would be one master where all writes take place, right?
Is there any plans to have multiple write masters, based on table partition (for instance)?
Imagine a User table, where users write their own data. Would it be possible to partition this table based on location?
I mean, to create a partition of this table for each location (america, europe, asia) that would become master for writes, and this way a user would write as master to the partition in its location (and then read replica would replicate it to other locations)

VI•11/28/23, 9:48 AM

Since d1 uses SQLite internally, does it run into the same limitations as SQLite, namely handling large number of independent writes/sec. From what I read SQLite is not capable of handling more than 1000 write reqs/sec.
https://www.sqlite.org/whentouse.html

Thanks for any help!

Rrodrigo I understand when read replicas are implemented there would be one master where ...

Max (@rozenmd)•11/28/23, 8:12 PM

So far the plan is just for a single writer

VVI Since d1 uses SQLite internally, does it run into the same limitations as SQLite...

Vy•11/28/23, 8:32 PM

Currently for open beta, ~1K QPS depending on result set size. Our eventual goal is to get 3-10K QPS for reads, 1-2K QPS for writes. Let us know what requirements your workload would need?

Kkian > Are all the limits except the DB size limit available to be potentially raised...

Village (Josef)•11/29/23, 3:06 AM

Sounds good, thank you! And I meant storage per account and number of DBs specifically

janat08•11/29/23, 8:51 AM

Can you promise at this point that the D1 size limitations will begone

const h = {id: 1, attr1: "abcdef} const t = JSON.stringify(h) // console.log("JSON being inserted =>>>>", h) const result = await env.SMS_DEV.prepare( "UPDATE TestTable SET history = json_insert(history, '$.history[#]', '?1') WHERE id = '?2' ", ) .bind(h, status.id) .run()

name = "weathered-queen-4070" main = "src/index.ts" compatibility_date = "2023-10-30" [[d1_databases]] binding = "DB" # available in your Worker on env.DB database_name = "dogpark" database_id = "xxxxxxxxxxxx" [[d1_databases]] binding = "DB2" # i.e. available in your Worker on env.DB database_name = "adminportalDB" database_id = "xxxxxxxxxxx"

can someone provide an example on this topic: How do I insert and then append a json object (not a s

Similar Threads

Similar Threads

Similar Threads