I'd recommend just using https://github.com/porsager/postgres with Neon + Hyperdrive. There's no rea

I'd recommend just using https://github.com/porsager/postgres with Neon + Hyperdrive. There's no real benefit using the serverless driver in this case.

JamesOP•8/28/24, 5:34 PM

^ That's my exact setup and it works well. I don't always use Hyperdrive, but I do use Neon + postgres.js very successfully.

tester123•8/28/24, 5:34 PM

may I please ask what is the latency you get approximately for a simple query?

JJames I'd recommend just using https://github.com/porsager/postgres with Neon + Hyperd...

tester123•8/28/24, 5:35 PM

in fact yes, postgres.js works, but was giving me a latency of 200-300ms for a simple query

JamesOP•8/28/24, 5:36 PM

With hyperdrive? Moving to #hyperdrive might make more sense here

tester123•8/28/24, 5:36 PM

without hyperdrive was like 600+ms

JJames With hyperdrive? Moving to #hyperdrive might make more sense here

tester123•8/28/24, 5:38 PM

may I please ask which web framework you are using? I'm thinking to migrate to hono.js btw

JamesOP•8/28/24, 5:39 PM

we use hono for all of our workers projects these days

JJames we use hono for all of our workers projects these days

tester123•8/28/24, 5:40 PM

that's interesting, thanks for sharing:)

Max (@rozenmd)•8/28/24, 8:49 PM

we also use Hono for all our D1 projects fwiw

jose luis•8/29/24, 12:35 AM

When using the workers' vitest integration, should i be able to access my local d1 database for testing?

import { env } from "cloudflare:test";
import { describe, it, expect } from "vitest";
import { Env } from "../src/index.js";

//This is in the docs for env
declare module "cloudflare:test" {
    interface ProvidedEnv extends Env {}
}

it("Prepared statements", async () => {
//test some function that uses env, but doesn't query d1, and it works
...
//test some function which queries my local d1 db
// doesn't found the table/s used in my query, sql error.
const table_list = await env.MY_DB.exec("pragma table_list;");
console.log("Table list:", table_list);
// which prints: Table list: { count: 1, duration: 0 }
})

import { env } from "cloudflare:test";
import { describe, it, expect } from "vitest";
import { Env } from "../src/index.js";

//This is in the docs for env
declare module "cloudflare:test" {
    interface ProvidedEnv extends Env {}
}

it("Prepared statements", async () => {
//test some function that uses env, but doesn't query d1, and it works
...
//test some function which queries my local d1 db
// doesn't found the table/s used in my query, sql error.
const table_list = await env.MY_DB.exec("pragma table_list;");
console.log("Table list:", table_list);
// which prints: Table list: { count: 1, duration: 0 }
})

There is only one recipe for

d1

d1

in the github repo linked in the docs, and it creates the db for testing using migrations. I expected to test my local d1 directly and this will know my local db from my bindings (toml is passed to

vitest.config.mts

vitest.config.mts

HHurby you can try using raw sql but idk

veeque•8/29/24, 8:24 AM

I don't know, as it seems that doing a batch is more for performance reasons rather than acid-ity

diffthanevery01•8/29/24, 9:07 AM

Hi, I have a question, is there an option to have the Worker created on one cloudflare account and the D1 database on the other and everything works together?

zorous•8/29/24, 5:33 PM

rn i have some csv data converted to json...

how would i automatically upload them to d1?

Zzorous rn i have some csv data converted to json... how would i automatically upload t...

Hurby•8/29/24, 6:10 PM

I think you can just write basic script on cf workers to upload them

itsmatteomanf•8/29/24, 7:02 PM

Text...?

itsmatteomanf•8/29/24, 7:06 PM

There is none in SQLite, they differ in other database types.

itsmatteomanf•8/29/24, 7:06 PM

https://www.sqlite.org/datatype3.html

itsmatteomanf•8/29/24, 7:10 PM

Not as far as I know, the limits are the ones state on the dev docs.

itsmatteomanf•8/29/24, 7:10 PM

And those of SQLite, of course

IEatBeans•8/30/24, 2:09 AM

Is CF aware of the random intermittent D1 Errors that cause a 500? Such as:

Internal error in D1 storage caused object to be reset

IIEatBeans Is CF aware of the random intermittent D1 Errors that cause a 500? Such as: In...

Max (@rozenmd)•8/30/24, 5:47 AM

we are, and have rolled out a bug fix for it

on the off chance this is something new, what's your database id?

MMax (@rozenmd)we are, and have rolled out a bug fix for it on the off chance this is somethin...

IEatBeans•8/30/24, 5:49 AM

That’s great to hear. Here’s the ID if needed:

e2e1eb93-f9ba-4615-aea7-bb44f89b61c3

o7•8/30/24, 6:02 AM

Getting D1_ERROR: Internal error in D1 storage caused object to be reset.

Oo7 Getting D1_ERROR: Internal error in D1 storage caused object to be reset.

Max (@rozenmd)•8/30/24, 6:10 AM

what's your database id?

MMax (@rozenmd)what's your database id?

o7•8/30/24, 6:10 AM

b2ea5e97-258f-4752-a68b-4dea4b10bf6c

o7•8/30/24, 6:11 AM

its calmed down now but it was intermittent. alongside 520s on a different site which didnt belong to me

o7•8/30/24, 6:18 AM

it might've been 85951405-f007-4d17-b1d4-094ed462f582 as well, which is the auth layer, but it would let me authenticate before returning that error when trying to access the 2nd DB pasted above

RexNormous•8/31/24, 1:42 AM

Also seeing intermittent

Internal error in D1 storage caused object to be reset.

Internal error in D1 storage caused object to be reset.

Database ID: 82459739-eaf4-4ad4-ac50-24ff60338cb1

Original message was deleted

Vikash•8/31/24, 2:07 AM

Vikash•8/31/24, 2:10 AM

I had to convert my data files to JSON > Upload to R2 and run a scheduled task to import all the files one by one using db.batch command

cosbgn•8/31/24, 8:09 AM

D1_ERROR: Internal error in D1 storage caused object to be reset.

Any ideas what could cause this?

Hiky•8/31/24, 2:02 PM

what is the problem with this the log shows client is undefined pls help

Alex Taccard•8/31/24, 4:55 PM

When one runs

npx wrangler d1 create

npx wrangler d1 create

to create a D1 database, he has to manually insert the new D1 binding configuration to the file

wrangler.toml

wrangler.toml

. However, if we execute this command in a folder that already contains

wrangler.toml

wrangler.toml

, can the binding insertion be achieved automatically, e.g. by a command option? If not, wouldn't this be a nice feature to implement?

Original message was deleted

Max (@rozenmd)•8/31/24, 7:14 PM

will take a look on Monday

Original message was deleted

Matt Silverlock•8/31/24, 9:55 PM

You can use (e.g.) https://grafana.com/grafana/plugins/fifemon-graphql-datasource/ with our GraphQL API

Grafana Labs

GraphQL Data Source plugin for Grafana | Grafana Labs

GraphQL Data Source

lifeiscontent•8/31/24, 10:27 PM

If I have a worker and a pages app operating on the same d1 database will it result in data corruption?

Llifeiscontent If I have a worker and a pages app operating on the same d1 database will it res...

Chaika•8/31/24, 10:35 PM

no, can have as many bound to it as you want/use api/dash, etc. Your D1 DB runs in a single location on a single machine

R in Tokyo•9/1/24, 12:08 AM

To the D1 user may I have some clarification on whether D1 is globally distributed? I’ve read that there are read only clones replicated close to your workers?

IIsaac McFadyen See this message for some past details from an employee - in short, maybe a bit ...

R in Tokyo•9/1/24, 12:11 AM

Why is there cloudflare marketing material mentioned “read replication” if they don’t have it?

Mozzy•9/1/24, 12:11 AM

+1 on the increase of

Internal error in D1 storage caused object to be reset.

Internal error in D1 storage caused object to be reset.

errors

R in Tokyo•9/1/24, 12:15 AM

How to make data global?

D1 GA has focused on reliability and developer experience. Now, we plan on extending D1 to better support globally-distributed applications.

In the Workers model, an incoming request invokes serverless execution in the closest data center. A Worker application can scale globally with user requests. Application data, however, remains stored in centralized databases, and global user traffic must account for access round trips to data locations. For example, a D1 database today resides in a single location

So it’s obviously not global?

R in Tokyo•9/1/24, 12:17 AM

Can somone here decipher the marketing material here ?

https://blog.cloudflare.com/building-d1-a-global-database/

The Cloudflare Blog

Building D1: a Global Database

D1, Cloudflare’s SQL database, is now generally available. With new support for 10GB databases, data export, and enhanced query debugging, we empower developers to build production-ready applications with D1 to meet all their relational SQL needs

R in Tokyo•9/1/24, 12:20 AM

This is literally a screenshot of replicated D1

https://cf-assets.www.cloudflare.com/slt3lc6tev37/2R1p29j20c7szuRmY2Sjlp/d3da1494232913b556806d624a0c8f14/image3-34.png

R in Tokyo•9/1/24, 12:30 AM

Ok so D1 is actually not global. Dear Cloudflare transparency goes a long way, and improves customer loyalty.

This stunts will have your customers jump to the nearest honest competitor.

It’s 2024, that article should be title “D1 is not global yet”

RR in Tokyo Ok so D1 is actually not global. Dear Cloudflare transparency goes a long way, a...

JamesOP•9/1/24, 12:35 PM

Agreed. This is a question I've had to answer time and time again because of the confusing marketing. Sorry. Example: https://www.reddit.com/r/CloudFlare/comments/1c2a7im/is_d1_rest_api_performant_and_production_ready/kz8u3i5/

Kiwi•9/1/24, 10:35 PM

Is Smart Placement a likely cause of the D1_ERROR increase I'm seeing?

Original message was deleted

Cyb3r-Jak3•9/1/24, 10:37 PM

I'm confused. You want to query a D1 database from Grafana?

veeque•9/2/24, 4:03 PM

without deleting the db and making another one, how can i make the db empty as if i just made it? (basically a drop)

import { env } from "cloudflare:test"; import { describe, it, expect } from "vitest"; import { Env } from "../src/index.js"; //This is in the docs for env declare module "cloudflare:test" { interface ProvidedEnv extends Env {} } it("Prepared statements", async () => { //test some function that uses env, but doesn't query d1, and it works ... //test some function which queries my local d1 db // doesn't found the table/s used in my query, sql error. const table_list = await env.MY_DB.exec("pragma table_list;"); console.log("Table list:", table_list); // which prints: Table list: { count: 1, duration: 0 } })

I'd recommend just using https://github.com/porsager/postgres with Neon + Hyperdrive. There's no rea

Similar Threads