I would recommend d1 but its still in alpha and there are current caps/limitations, to me the best t

D

DahoomOP•8/8/23, 12:03 PM

Does anyone have an example worker/function using the query JSON api? I'm having a hard time getting a grasp of it from the docs

D

DahoomOP•8/8/23, 12:23 PM

anything really

D

DahoomOP•8/8/23, 12:24 PM

but prepared would be preferrable

DDahoom I would recommend d1 but its still in alpha and there are current caps/limitatio...

C

chientrm.com•8/8/23, 12:24 PM

D1 is based on Sqlite so that migration is not really an issue.

Original message was deleted

C

Cyb3r-Jak3•8/8/23, 12:28 PM

There is not. D1 is primarily for workers.

Original message was deleted

C

chientrm.com•8/8/23, 12:29 PM

You can use this package to run a D1 proxy and using D1 from your PHP https://www.npmjs.com/package/cf-workers-proxy

npm

cf-workers-proxy

Enable Cloudflare Workers runtime for local development. Latest version: 0.3.0, last published: 17 days ago. Start using cf-workers-proxy in your project by running

npm i cf-workers-proxy

npm i cf-workers-proxy

npm i cf-workers-proxy

npm i cf-workers-proxy. There are no other projects in the npm registry using cf-workers-proxy.

C

chientrm.com•8/8/23, 12:30 PM

nvm the package is js-only

D

DahoomOP•8/8/23, 12:31 PM

you can do a makeshift API with cloudflare workers and call it within php

D

DahoomOP•8/8/23, 12:32 PM

since the worker will be connected to d1 you could set a secret env variable with an API key, authenticate through the bearer header and pass in sql strings to be executed

D

DahoomOP•8/8/23, 12:35 PM

I've done similar but it was in php for cloudflare pages, as long as you write the token validation correctly everything should be smooth

D

DahoomOP•8/8/23, 12:36 PM

I've never tried firebase but probably?

Original message was deleted

C

Cyb3r-Jak3•8/8/23, 12:44 PM

The D1 API is not public and if it does become public it will fall under the API rate limit. Making a worker to proxy is your best bet

Ccyan Is provisioning a db (primary, non replicated) for each user a stupid idea? Is D...

G

GuillaumeLakano•8/8/23, 12:46 PM

It's not a stupid idea, and it's add a strong security layer, it's currently what we doing with DO ( a mini DB for each user ) and we wait about the same possibility in D1
D1 is based on DO, DO have unlimited Objects, and D1 will increase from 10 databases per account to « thousands to tens of thousands of tenants » , but it's still not enough if you have (expect) more users than this limit.

EDIT: I've said « D1 is based on DO » because I've read that here before, but Matt add this precisions: d1-database

GGuillaumeLakano It's not a stupid idea, and it's add a strong security layer, it's currently wha...

M

Matt Silverlock•8/8/23, 1:40 PM

We haven’t communicated any upper bound. We just have limits during alpha.

MMatt Silverlock We haven’t communicated any upper bound. We just have limits during alpha.

G

GuillaumeLakano•8/8/23, 2:00 PM

So, may be I've misunderstood you ( « We have some ideas here for the “thousands to tens of thousands of tenants” - that will isolate each DB you create. Can’t share more yet and won’t be “next month”. » in d1-database )

G

GuillaumeLakano•8/8/23, 2:02 PM

could be really useful if CF can officially claims this will be unlimited, so this means we can start new project, using D1 directly ( while it's in development period, until the stable release of D1 one day )

GGuillaumeLakano could be really useful if CF can officially claims this will be unlimited, so th...

M

Matt Silverlock•8/8/23, 2:40 PM

We at least anticipate “tens of thousands” by the end of the year. I’m not in the habit of making claims that we haven’t tested but there are no architectural reasons why millions will not work.

GGuillaumeLakano could be really useful if CF can officially claims this will be unlimited, so th...

M

Matt Silverlock•8/8/23, 2:41 PM

If it’s easier I can just say “eventually unlimited”. If you have millions of users we will make it work. One of the reasons we built Workers for Platforms.

G

GuillaumeLakano•8/8/23, 2:59 PM

Awesome, thank you @Matt for theses precisions!

R

rotator•8/8/23, 3:41 PM

Any comment regarding this? d1-database
To rephrase: if D1 is built on top of DO, then why is it so much cheaper?

Interestingly enough, I can wrap my D1 calls into DO transactions like that and it seems to work fine

    const value = 0;
    let d1Response: any;
    await this.state.storage.transaction(async (txn) => {
      try {
        txn.put('value', this.value);
        const stmt = this.db
          .prepare(`INSERT INTO items (itemData) VALUES (?) RETURNING *`)
          .bind(JSON.stringify({ value }));
        d1Response = await stmt.all();
      } catch (e) {
        console.error(e);
        txn.rollback();
        throw e;
      }
    });

    const value = 0;
    let d1Response: any;
    await this.state.storage.transaction(async (txn) => {
      try {
        txn.put('value', this.value);
        const stmt = this.db
          .prepare(`INSERT INTO items (itemData) VALUES (?) RETURNING *`)
          .bind(JSON.stringify({ value }));
        d1Response = await stmt.all();
      } catch (e) {
        console.error(e);
        txn.rollback();
        throw e;
      }
    });

    const value = 0;
    let d1Response: any;
    await this.state.storage.transaction(async (txn) => {
      try {
        txn.put('value', this.value);
        const stmt = this.db
          .prepare(`INSERT INTO items (itemData) VALUES (?) RETURNING *`)
          .bind(JSON.stringify({ value }));
        d1Response = await stmt.all();
      } catch (e) {
        console.error(e);
        txn.rollback();
        throw e;
      }
    });

    const value = 0;
    let d1Response: any;
    await this.state.storage.transaction(async (txn) => {
      try {
        txn.put('value', this.value);
        const stmt = this.db
          .prepare(`INSERT INTO items (itemData) VALUES (?) RETURNING *`)
          .bind(JSON.stringify({ value }));
        d1Response = await stmt.all();
      } catch (e) {
        console.error(e);
        txn.rollback();
        throw e;
      }
    });

but I wouldn't want to do this just for cost management purposes

Rrotator Any comment regarding this? https://discord.com/channels/595317990191398933/9920...

M

Matt Silverlock•8/8/23, 3:56 PM

I've never said it's built on top of DO. How they store data is entirely distinct.

M

marko•8/8/23, 4:17 PM

I'm testing D1 and I followed the comments API example but I'm getting

no such table

no such table

error.

wrangler d1 info

wrangler d1 info

shows table count correctly. Any ideas what I'm doing wrong?

MMatt Silverlock I've never said it's built on top of DO. How they store data is entirely distinc...

R

rotator•8/8/23, 4:22 PM

I see, thanks for clarifying this.

I can infer that for overlapping use cases D1 is heavily encouraged, unless prices change before D1 is generally available.

Mmarko I'm testing D1 and I followed the comments API example but I'm getting `no such ...

M

marko•8/8/23, 4:32 PM

Just noticed that

wrangler dev --remote

wrangler dev --remote

works, so I'm guessing that this is related to local env.

M

marko•8/8/23, 4:36 PM

But now with --remote the worker doesn't connect via websocket, uhm.

GGuillaumeLakano It's not a stupid idea, and it's add a strong security layer, it's currently wha...

C

cyan•8/8/23, 6:19 PM

That’s great thanks!

Rrotator Any comment regarding this? https://discord.com/channels/595317990191398933/9920...

D

DahoomOP•8/8/23, 6:26 PM

How did you get syntax highlighting on that example code?

DDahoom How did you get syntax highlighting on that example code?

C

cyan•8/8/23, 6:27 PM

https://gist.github.com/matthewzring/9f7bbfd102003963f9be7dbcf7d40e51#syntax-highlighting

Gist

A guide to Markdown on Discord.

A guide to Markdown on Discord. GitHub Gist: instantly share code, notes, and snippets.

C

cyan•8/8/23, 6:27 PM

its a discord syntax highlighting block

D

DahoomOP•8/8/23, 6:28 PM

test test

test test

D

DahoomOP•8/8/23, 6:28 PM

interesting, I didnt know highlightjs comes with discord

MMatt Silverlock I've never said it's built on top of DO. How they store data is entirely distinc...

A

Adi•8/8/23, 9:43 PM

Follow up question on this. I remember reading somewhere that DO will soon have an sqlite store connected as part of it's storage layer. Would this be using the same backend as D1?

S

sty•8/8/23, 10:06 PM

is kv still faster than d1 for query? or is d1 catching up/on par now?

Ggeelen Of course, you _can_ use the Cache API yourself (https://developers.cloudflare.c...

C

Chaika•8/8/23, 11:33 PM

unless I'm missing something or something changed recently, you couldn't use Cache API to match KV's new architecture, since KV 2.0 is something more close to Tiered Caching with generic + regional tier (which is Enterprise), and the Cache API doesn't interact with tiered caching at all.
KV would be faster over D1 if eventual consistency is fine/if you can have data cached for a while (and your data fits it), especially with its recent improvements.
(It would be cool if Cache API got Tiered caching one day

)

CChaika unless I'm missing something or something changed recently, you couldn't use Cac...

C

Codename_404•8/9/23, 2:11 AM

It would be great if they allowed caching with r2 custom domain. Then we could use fetch with custom domain on r2 with worker. Which would enable tired caching by default.

The r2 domain could be as part of a zone, just like worker route.

Would make things much easier.

G

Greg•8/9/23, 6:20 AM

Has anybody used the kysely adapter for d1 and run into this error?

Do not know how to serialize a BigInt

Do not know how to serialize a BigInt

more info here https://github.com/aidenwallis/kysely-d1/issues/21

G

Greg•8/9/23, 6:45 AM

Also unrelated, I'm created a table with this query:

CREATE TABLE Casts (
    id int8 NOT NULL PRIMARY KEY,
    created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
    updated_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
    deleted_at timestamp,
    timestamp timestamp NOT NULL,
    fid int8 NOT NULL,
    hash bytea NOT NULL,
    parent_hash bytea,
    parent_fid int8,
    parent_url text,
    text text NOT NULL,
    embeds text NOT NULL DEFAULT '{}',
    mentions int8 NOT NULL DEFAULT '{}',
    mentions_positions int2 NOT NULL DEFAULT '{}'
);

CREATE TABLE Casts (
    id int8 NOT NULL PRIMARY KEY,
    created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
    updated_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
    deleted_at timestamp,
    timestamp timestamp NOT NULL,
    fid int8 NOT NULL,
    hash bytea NOT NULL,
    parent_hash bytea,
    parent_fid int8,
    parent_url text,
    text text NOT NULL,
    embeds text NOT NULL DEFAULT '{}',
    mentions int8 NOT NULL DEFAULT '{}',
    mentions_positions int2 NOT NULL DEFAULT '{}'
);

CREATE TABLE Casts (
    id int8 NOT NULL PRIMARY KEY,
    created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
    updated_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
    deleted_at timestamp,
    timestamp timestamp NOT NULL,
    fid int8 NOT NULL,
    hash bytea NOT NULL,
    parent_hash bytea,
    parent_fid int8,
    parent_url text,
    text text NOT NULL,
    embeds text NOT NULL DEFAULT '{}',
    mentions int8 NOT NULL DEFAULT '{}',
    mentions_positions int2 NOT NULL DEFAULT '{}'
);

CREATE TABLE Casts (
    id int8 NOT NULL PRIMARY KEY,
    created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
    updated_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
    deleted_at timestamp,
    timestamp timestamp NOT NULL,
    fid int8 NOT NULL,
    hash bytea NOT NULL,
    parent_hash bytea,
    parent_fid int8,
    parent_url text,
    text text NOT NULL,
    embeds text NOT NULL DEFAULT '{}',
    mentions int8 NOT NULL DEFAULT '{}',
    mentions_positions int2 NOT NULL DEFAULT '{}'
);

I know it has some less common data types but the table seems to create just fine and I can add data from the Cloudflare dashboard.

Only problem is when I try to read it from a worker, I always gets

D1_ERROR: SqliteError: no such table: Casts

D1_ERROR: SqliteError: no such table: Casts

A

akazwz•8/9/23, 6:57 AM

look like than cloudflare pages with d1 binding has some problems, my account id is a275d70ec37a6046e6778c77d65dd38b, i am using remix with pages, the DB binding will be D1_BETADB and do not work.

Aakazwz look like than cloudflare pages with d1 binding has some problems, my account id...

F

forslunds•8/9/23, 10:20 AM

im using Remix/Pages/D1 without problems, atleast on Dev/Local. Were are you struggling? Get any errors?

H

Hasi•8/9/23, 10:41 AM

Hi

H

Hasi•8/9/23, 10:41 AM

I'm building a remix app

H

Hasi•8/9/23, 10:41 AM

And

H

Hasi•8/9/23, 10:41 AM

I've set up d1

H

Hasi•8/9/23, 10:41 AM

I'm using the local db for development

H

Hasi•8/9/23, 10:43 AM

but running a code like this

const result = req.context.db.prepare("SELECT * from accounts");
  const rows = await result.all();
  console.log(rows);

const result = req.context.db.prepare("SELECT * from accounts");
  const rows = await result.all();
  console.log(rows);

const result = req.context.db.prepare("SELECT * from accounts");
  const rows = await result.all();
  console.log(rows);

const result = req.context.db.prepare("SELECT * from accounts");
  const rows = await result.all();
  console.log(rows);

Results in:

Error: D1_ERROR
    at D1Database._send (C:\Users\hasx\OneDrive\Dokumentumok\Rókales\foxadopt\node_modules\@miniflare\d1\src\d1js.ts:148:13)
    at D1PreparedStatement.all (C:\Users\hasx\OneDrive\Dokumentumok\Rókales\foxadopt\node_modules\@miniflare\d1\src\d1js.ts:242:7)
    ... 7 lines matching cause stack trace ...
    at C:\Users\hasx\AppData\OneDrive\Dokumentumok\Rókales\foxadopt\functions\[[path]].js:3446:92 {
  [cause]: Error: Error: SqliteError: no such table: accounts

Error: D1_ERROR
    at D1Database._send (C:\Users\hasx\OneDrive\Dokumentumok\Rókales\foxadopt\node_modules\@miniflare\d1\src\d1js.ts:148:13)
    at D1PreparedStatement.all (C:\Users\hasx\OneDrive\Dokumentumok\Rókales\foxadopt\node_modules\@miniflare\d1\src\d1js.ts:242:7)
    ... 7 lines matching cause stack trace ...
    at C:\Users\hasx\AppData\OneDrive\Dokumentumok\Rókales\foxadopt\functions\[[path]].js:3446:92 {
  [cause]: Error: Error: SqliteError: no such table: accounts

Error: D1_ERROR
    at D1Database._send (C:\Users\hasx\OneDrive\Dokumentumok\Rókales\foxadopt\node_modules\@miniflare\d1\src\d1js.ts:148:13)
    at D1PreparedStatement.all (C:\Users\hasx\OneDrive\Dokumentumok\Rókales\foxadopt\node_modules\@miniflare\d1\src\d1js.ts:242:7)
    ... 7 lines matching cause stack trace ...
    at C:\Users\hasx\AppData\OneDrive\Dokumentumok\Rókales\foxadopt\functions\[[path]].js:3446:92 {
  [cause]: Error: Error: SqliteError: no such table: accounts

Error: D1_ERROR
    at D1Database._send (C:\Users\hasx\OneDrive\Dokumentumok\Rókales\foxadopt\node_modules\@miniflare\d1\src\d1js.ts:148:13)
    at D1PreparedStatement.all (C:\Users\hasx\OneDrive\Dokumentumok\Rókales\foxadopt\node_modules\@miniflare\d1\src\d1js.ts:242:7)
    ... 7 lines matching cause stack trace ...
    at C:\Users\hasx\AppData\OneDrive\Dokumentumok\Rókales\foxadopt\functions\[[path]].js:3446:92 {
  [cause]: Error: Error: SqliteError: no such table: accounts

When I have the tables set up

Aakazwz look like than cloudflare pages with d1 binding has some problems, my account id...

M

Matt Silverlock•8/9/23, 10:45 AM

Are you using wrangler 3.5.0? https://github.com/cloudflare/workers-sdk/releases/tag/wrangler%403.5.0

if not, please upgrade.

GitHub

Release wrangler@3.5.0 · cloudflare/workers-sdk

Minor Changes

#3703 e600f029 Thanks @jspspike! - Added --local option for r2 commands to interact with local persisted r2 objects

#3704 8e231afd Thanks @JacobMGEvans! - secret:bulk exit 1 on fa...

I would recommend d1 but its still in alpha and there are current caps/limitations, to me the best t

Similar Threads

Similar Threads

Similar Threads