How large is your sql file? Can you reproduce it consistently? Can you share a reproduction if so?

JJames How large is your sql file? Can you reproduce it consistently? Can you share a r...

Z

Zack•2/18/25, 10:23 PM

It was working consistently earlier, I have uploaded the same sized files repeatedly and then wiped the database. The files are around 30MB each

Z

Zack•2/18/25, 10:24 PM

I'm uploading 50 one after another to chunk the ingestion

Z

Zack•2/18/25, 10:24 PM

Its around 6 million total rows

Z

Zack•2/18/25, 10:25 PM

The weird part was it wasn't even failing to upload, it was failing on the check to see if it needed to upload. It never actually got that far

Z

Zack•2/18/25, 10:57 PM

Just tested again now and still experiencing this problem, I effectively can't import any .sql files right now

Z

Zack•2/18/25, 10:58 PM

The entire D1 (non-worker) API seems to be significantly slower for me compared to earlier today

Z

Zack•2/18/25, 11:16 PM

It appears that wiping the table and starting again resolves the problem, so it may be something that gets worse as the table gets bigger?

J

Johnny Silverhand•2/19/25, 7:39 AM

idk if i am right here; but the german translation for table in the webui is absolutely wrong:

MbsPMuLneUAQUFBvqdnuxXRNQUHRN1PQkIAEJSCATUFBkGr7vKwGnPPX1zpgvCUhAAhKQgAQkIAEJDICAgmIAN8ksSkACEpCABCQgAQlIoK8EFBR9vTPmSwISkIAEJCABCUhAAgMgoKAYwE0yixKQgAQkIAEJSEACEugrAQVFXMZKABCQgAQlIQAISkMAACCgoBnCTzKIEJCABCUhAAhKQgAT6SkBB0dc7Y74kIAEJSEACEpCABCQwAAIKigHcJLMoAQlIQAISkIAEJCCBvhJQUPT1zpgvCUhAAhKQgAQkIAEJDICAgmIAN8ksSkACEpCABCQgAQlIoK8EFBR9vTPmSwISkIAEJCABCUhAAgMgoKAYwE0yixKQgAQkIAEJSEACEugrgf8DCbNRuY4FECAAAAAASUVORK5CYII.png

J

Johnny Silverhand•2/19/25, 7:39 AM

is there any way for cloudflare to change it to the correct translation? it would be "Tabellen"

JJohnny Silverhand is there any way for cloudflare to change it to the correct translation? it woul...

�

𝖆𝖚𝖙𝖎𝖘𝖙•2/19/25, 7:45 AM

Ki übersetzungen sind doch was tolles

Original message was deleted

Z

Zack•2/19/25, 6:35 PM

Z

Zack•2/19/25, 6:35 PM

Is this in preparation for read replicas?

ZZack Is this in preparation for read replicas? 👀

A

ajgeiss0702•2/19/25, 7:26 PM

i hope so

:crossed_fingers_intensifies:

A

ajgeiss0702•2/19/25, 7:33 PM

at this point I wonder if theyre waiting until developer week (which is april iirc?) to launch it

�𝖆𝖚𝖙𝖎𝖘𝖙 Ki übersetzungen sind doch was tolles 😂

J

Johnny Silverhand•2/19/25, 8:21 PM

ohja, richtig klasse :D

A

ac•2/19/25, 8:59 PM

Is D1 down? Getting lots of

Error: D1_ERROR: internal error

Error: D1_ERROR: internal error

A

ac•2/19/25, 9:03 PM

Looks to be working now, but there was about 4 mins of downtime (no changes made on our end)

A

Althaf•2/20/25, 1:52 AM

Can i ask a doubt like can we use the D1 on the nextjs hosted on verel and i am unable to use it on local development i have separrte backend in hono cf workers for admin pannel but for seo i am trying to access the data within the nextjs but still i can't can anybody help

AAlthaf Can i ask a doubt like can we use the D1 on the nextjs hosted on verel and i am ...

U

Unsmart•2/20/25, 1:58 AM

You cant use D1 on vercel in that way. Vercel doesnt have cloudflare bindings. If you wanted to query D1 from Vercel you would need to either use the Rest API or create a worker to proxy requests to the D1 binding.

UUnsmart You cant use D1 on vercel in that way. Vercel doesnt have cloudflare bindings. I...

A

Althaf•2/20/25, 2:07 AM

so are you mentioning that i should use my hono backend in cf workers to power the users front end also like fetch data in server are try to cache it to increase perfomance ? and also i don't get this point "create a worker to proxy requests to the D1 binding." like can you explain or do you have any article on it

A

Althaf•2/20/25, 2:09 AM

i also tried to host nextjs in pages but the styles are breaking in pages preview

AAlthaf so are you mentioning that i should use my hono backend in cf workers to power t...

U

Unsmart•2/20/25, 2:13 AM

Not referencing cache or performance at all. Just pointing out that you cannot access a cloudflare binding from vercel because vercel is not cloudflare.

The only way to query data from D1 from the vercel servers would be to call a cloudflare worker or the cloudflare api.

UUnsmart Not referencing cache or performance at all. Just pointing out that you cannot a...

A

Althaf•2/20/25, 3:53 AM

I am confused like on production we can use the db url in the drizzle and it will handle the db interactions so i thought that we can use d1 in the vercel but on development stage we need local db as per the d1 docs

AAlthaf I am confused like on production we can use the db url in the drizzle and it wil...

L

lambrospetrou•2/20/25, 8:44 AM

D1 doesn't have a connection URL as traditional databases. Either you query D1 through a Worker using the D1 binding (https://developers.cloudflare.com/d1/worker-api/) which is the fastest and most performant, or you should use the D1 REST API (https://developers.cloudflare.com/api/resources/d1/subresources/database/methods/query/).

I do not recommend the REST API if you will do several database queries (round trips), since it's not meant to be used yet in your hot paths. It's slow at the moment, more rate limited than the Workers binding, and you won't have a nice experience. We are improving this in the following months but it's a long ongoing process.

Llambrospetrou D1 doesn't have a connection URL as traditional databases. Either you query D1 t...

A

Althaf•2/20/25, 11:36 AM

Thankyou for the detailed info. so like i am planing to go with the acessing data from my workers using hono

Llambrospetrou Not using one 😃

L

Lars-Erik•2/21/25, 2:48 PM

Orange ORM supports D1 now.
https://github.com/alfateam/orange-orm

GitHub

GitHub - alfateam/orange-orm: The ultimate ORM for Node and Typescript

The ultimate ORM for Node and Typescript. Contribute to alfateam/orange-orm development by creating an account on GitHub.

Aajgeiss0702 at this point I wonder if theyre waiting until developer week (which is april ii...

M

murzin•2/21/25, 9:12 PM

wait, do they have multiple developer weeks a year? I recall one being just recently

W

Walshy•2/21/25, 9:14 PM

we have multiple innovation weeks (Birthday week, developer week, security week, privacy week, etc.)

W

Walshy•2/21/25, 9:14 PM

but only 1 developer week a year

M

murzin•2/21/25, 9:15 PM

haha got you. looking forward to it! I'm sure you are too :D

W

Walshy•2/21/25, 9:16 PM

it'll be a good one

S

SQQ•2/21/25, 11:30 PM

Is it possible to create a D1 database with the API and access it through a worker binding? What is the workflow for dynamically creating a database per user and accessing it in a worker?

SSQQ Is it possible to create a D1 database with the API and access it through a work...

D

Dean•2/22/25, 7:12 AM

Yes you can, this is how I do it: I create the database via the API, i use the API to also create the table / schema and then use the API to modify the worker settings to add the D1 binding. The last bit is the tricky bit, even from when I've done it the documentation / endpoints have changed.

DDean Yes you can, this is how I do it: I create the database via the API, i use the A...

S

SQQ•2/22/25, 6:35 PM

I had a look at the Workers API and you are right is looks tricky to say the least. Provisioning a database per client for a SaaS app automatically and tying it to a worker via various API calls seems fragile. I’d love to use D1 for my needs but Turso seems much easier for this scenario at the moment. I had a think about how I could do this without API calls and the only way I could think of was to create all the databases up front (100’s 1000’s?), bind them to a worker, and then control assignment via my app. Not the most elegant solution.

SSQQ I had a look at the Workers API and you are right is looks tricky to say the lea...

D

Dean•2/22/25, 6:38 PM

Dynamic bindings is suppose to fix this which is suppose to release soon but that was nearly 6 months ago and haven’t heard anything more on it. You’re also capped to 5000 databases at the moment per worker.

DDean Dynamic bindings is suppose to fix this which is suppose to release soon but tha...

S

SQQ•2/22/25, 9:00 PM

Sounds promising - hopefully they include some sort of smooth solution to the per worker limit for bindings and storage limit per database - thanks for the info!

L

Larry•2/23/25, 12:15 AM

In the meantime, you might want to take a look at Durable Objects (DOs) with the SQLite storage engine. The max database size there is only 1GB, but it will increase to match D1s (10GB) once it's out of beta. You get the equivalent of dynamic binding with DOs because each DO instance gets its own SQLite database.

H

Hamir Mahal•2/23/25, 3:51 AM

    try {
      // Check if the DB is reachable
      env.DB.prepare("SELECT 1").first();
      const result = env.DB.prepare("INSERT INTO ... (...) VALUES (?, ?, ?, ?)")
        .bind()
        .run()
        .then((result: unknown) => {
          console.debug("Insertion result:", result);
        })
        .catch((error: unknown) => {
          console.error("Error inserting data:", error);
          throw error; // Rethrow to handle in the catch block
        });

      console.debug({ ... }, "logged to D1 database", { result });
    } catch (err) {
      console.error("Insertion into", { env }, "failed", err);
    }

    try {
      // Check if the DB is reachable
      env.DB.prepare("SELECT 1").first();
      const result = env.DB.prepare("INSERT INTO ... (...) VALUES (?, ?, ?, ?)")
        .bind()
        .run()
        .then((result: unknown) => {
          console.debug("Insertion result:", result);
        })
        .catch((error: unknown) => {
          console.error("Error inserting data:", error);
          throw error; // Rethrow to handle in the catch block
        });

      console.debug({ ... }, "logged to D1 database", { result });
    } catch (err) {
      console.error("Insertion into", { env }, "failed", err);
    }

    try {
      // Check if the DB is reachable
      env.DB.prepare("SELECT 1").first();
      const result = env.DB.prepare("INSERT INTO ... (...) VALUES (?, ?, ?, ?)")
        .bind()
        .run()
        .then((result: unknown) => {
          console.debug("Insertion result:", result);
        })
        .catch((error: unknown) => {
          console.error("Error inserting data:", error);
          throw error; // Rethrow to handle in the catch block
        });

      console.debug({ ... }, "logged to D1 database", { result });
    } catch (err) {
      console.error("Insertion into", { env }, "failed", err);
    }

    try {
      // Check if the DB is reachable
      env.DB.prepare("SELECT 1").first();
      const result = env.DB.prepare("INSERT INTO ... (...) VALUES (?, ?, ?, ?)")
        .bind()
        .run()
        .then((result: unknown) => {
          console.debug("Insertion result:", result);
        })
        .catch((error: unknown) => {
          console.error("Error inserting data:", error);
          throw error; // Rethrow to handle in the catch block
        });

      console.debug({ ... }, "logged to D1 database", { result });
    } catch (err) {
      console.error("Insertion into", { env }, "failed", err);
    }

logs

... logged to D1 database { result: Promise { <pending> } }

... logged to D1 database { result: Promise { <pending> } }

... logged to D1 database { result: Promise { <pending> } }

... logged to D1 database { result: Promise { <pending> } }
but it should be logging an error, because I know there's an error in one of the values I

INSERT

INSERT

INSERT

INSERT.

H

Hamir Mahal•2/23/25, 3:51 AM

      const result = env.DB.prepare("INSERT INTO ... (...) VALUES (?, ?, ?, ?)")
        .bind()
        .run();

      const result = env.DB.prepare("INSERT INTO ... (...) VALUES (?, ?, ?, ?)")
        .bind()
        .run();

      const result = env.DB.prepare("INSERT INTO ... (...) VALUES (?, ?, ?, ?)")
        .bind()
        .run();

      const result = env.DB.prepare("INSERT INTO ... (...) VALUES (?, ?, ?, ?)")
        .bind()
        .run();

does seem to be a

Promise

Promise

Promise

Promise.

H

Hamir Mahal•2/23/25, 3:51 AM

Everything works fine if I just use

await

await

instead of

then

then

then

then and

catch

catch

, but I'd like to do this database insertion in a non-blocking way, preferably.

H

Hamir Mahal•2/23/25, 3:51 AM

Anyone know how to do this in a non-blocking way, while still being able to see database insertion errors?

H

Hamir Mahal•2/23/25, 3:51 AM

All of this is inside a

  async fetch(request: Request, env: any): Promise<Response>

  async fetch(request: Request, env: any): Promise<Response>

  async fetch(request: Request, env: any): Promise<Response>

  async fetch(request: Request, env: any): Promise<Response>

Cloudflare handler.

HHamir Mahal ```ts try { // Check if the DB is reachable env.DB.prepare("SELE...

H

Hamir Mahal•2/23/25, 3:52 AM

And if I switch to using

await

await

, I see the error, although I believe that makes the database insertion a blocking operation.

HHamir Mahal ```ts try { // Check if the DB is reachable env.DB.prepare("SELE...

I

Isaac McFadyen•2/23/25, 5:09 AM

This won't work unless you do the

await

await

. You can't see the value of a Promise before it's completed, and the console.debug({}) is run before the result of the Promise.

I

Isaac McFadyen•2/23/25, 5:09 AM

If it's for logging purposes and you want to avoid holding up the response, you can use ctx.waitUntil

I

Isaac McFadyen•2/23/25, 5:09 AM

https://developers.cloudflare.com/workers/runtime-apis/context/#waituntil

T

tt•2/23/25, 5:37 AM

Since yesterday, I have been encountering the error "D1_ERROR: Network connection lost." in a Worker that is performing a SELECT query on D1.
When I try to open the corresponding D1 dashboard in a web browser, it keeps loading indefinitely and does not display anything.
Is there any solution to this issue?

Sorry if it sounds weird, I'm using a translation tool.

LLarry In the meantime, you might want to take a look at Durable Objects (DOs) with the...

D

Dean•2/23/25, 11:45 AM

This is good advice, but it’s not exactly the same as D1, for example time travel backups doesn’t exist for durable objects (afaik) and no API support

Ttt Since yesterday, I have been encountering the error "D1_ERROR: Network connectio...

J

JamesOP•2/23/25, 2:46 PM

Is this still happening? If so, let us know your Database ID and we can escalate this to be looked at.

J

JamesOP•2/23/25, 3:24 PM

Your account ID would also be helpful @tt for the team, thanks

How large is your sql file? Can you reproduce it consistently? Can you share a reproduction if so?

Similar Threads

Similar Threads

Similar Threads