`D1_ERROR: internal error; reference = ...` - what is this?

D1_ERROR: internal error; reference = ...

D1_ERROR: internal error; reference = ...

D1_ERROR: internal error; reference = ...

D1_ERROR: internal error; reference = ... - what is this?

Wwelz `D1_ERROR: internal error; reference = ...` - what is this?

Hard@Work•7/21/25, 9:30 PM

An error originating within D1 itself

AAlpa Looks like this wasn't an issue with D1 itself. Apparently my provider (DTAG in ...

Silvan•7/22/25, 12:12 PM

the telekom special

AAlpa Looks like this wasn't an issue with D1 itself. Apparently my provider (DTAG in ...

Chaika•7/22/25, 1:13 PM

oh interesting, yea that is normal for dtag on free plan but should still be routed to a replica for select only requests (meta.served_by_primary) even in primary region, although it won't help response times of course

KK•7/23/25, 10:15 AM

I have a requirement to stream approximately 6,000 ticker price updates per minute. I then need to support querying by ticker for all prices received today. What storage solution would you recommend?

Key-Value (KV) stores could become very expensive with 6,000 writes per minute.

I'm unsure whether D1 would be a better fit in this scenario.

I considered using the Analytics Engine, but I don't see a way to delete data older than one day. By default, it appears to retain data for 3 months, potentially growing to hundreds of millions or even billions of rows. I'm concerned about performance implications at that scale.

KKK I have a requirement to stream approximately 6,000 ticker price updates per minu...

Silvan•7/23/25, 10:45 AM

Maybe a Durable Object per ticker?

SSilvan Maybe a Durable Object per ticker?

1984 Ford Laser•7/23/25, 11:09 AM

DO per ticket per day, to keep performance consistent while allowing for any variety of data retention policy

Silvan•7/23/25, 11:10 AM

with the Actor class you even get the tracking without any additional code

KKK I have a requirement to stream approximately 6,000 ticker price updates per minu...

Larry•7/23/25, 12:07 PM

Have you looked at Cloudflare Pipelines? It's designed for high-speed data ingestion. The data goes into an R2 bucket in time slices that you can batch process.

Sylvain•7/23/25, 1:18 PM

Any plans to allow using read replicas via the D1 REST API?

LLarry Have you looked at [Cloudflare Pipelines](https://developers.cloudflare.com/pipe...

KK•7/23/25, 1:20 PM

I'm also planning to query this data once it's ingested—do you know if R2 provides low enough latency and throughput for efficient querying, or would that require an additional system layered on top?

KKK I'm also planning to query this data once it's ingested—do you know if R2 provid...

Larry•7/23/25, 1:32 PM

Yeah, you would need to process the R2 into something you could query. Unless a bunch of raw entries is boiled down into a small number of rows, I don't think D1, DOs, or KV are ideal for your situation. D1 and DOs have a 10GB limit. You could shard with DOs as Silvan suggests but that's complexity. KV is not queriable. All three are probably costly for your use. I don't know anything about it but Cloudflare Analytics might be an option. There's also hyperdrive and 3rd party storage integrations

Cloudflare Docs

Hyperdrive

Hyperdrive is a service that accelerates queries you make to existing databases, making it faster to access your data from across the globe from Cloudflare Workers, irrespective of your users' location.

Cloudflare Docs

3rd Party Integrations

Connect to third-party databases such as Supabase, Turso and PlanetScale)

KK•7/23/25, 2:05 PM

thank you very much @Larry

JJames single thread on D1 and network latency is going to kill you, you're better off ...

TeamDave•7/23/25, 2:57 PM

So my sql is not great, but I got it down from 12 queries to 3. I'm sending 2 of them in a batch using drizzle, and using some results to build the last query.

Locally, this reduced my time by ~500ms. But when I go to staging & use actual D1, I'm getting a cloudflare error, "D1_ERROR: out of memory: SQLITE_NOMEM". I'm not finding a lot about that error, but it seems like I have gone too aggressive with my query combining?

Is it usually a balancing act trying to get the right number of requests with the right complexity of queries?

KKK thank you very much @Larry

Silvan•7/23/25, 3:05 PM

but yea i agree with insane numbers like that i would recommend planetscale

TTeamDave So my sql is not great, but I got it down from 12 queries to 3. I'm sending 2 of...

James•7/23/25, 3:55 PM

Probably a lot of table scanning w your queries, you should analyze the sql plan for your queries and optimize your indexing.

gruntlord6•7/23/25, 8:29 PM

Does anyone know an accurate way to see your D1 Writes? the billing page doesn't line up at all with the actual data in the d1 dashboard

gruntlord6•7/23/25, 8:30 PM

its saying I have millions more writes then any of my databases account for

gruntlord6•7/23/25, 8:31 PM

Edit: ignore this part, it was just my billing period and I I was reading it wrong

gruntlord6•7/23/25, 9:21 PM

that explains the date but they still don't line up

gruntlord6•7/23/25, 9:24 PM

theres a 50 million write difference between what the dashboard shows and what my bill shows

gruntlord6•7/23/25, 9:26 PM

I also never get my billing notification when I go over but apprently thats a beta so I just assume it never will work

gruntlord6•7/23/25, 9:26 PM

just a bit frustrating to get billed with no way to know whats happening

gruntlord6•7/23/25, 9:36 PM

well I managed to track it down but I can't say I am happy that the notification didn't trigger or that I had to dig so much just to find it

gruntlord6•7/24/25, 5:40 AM

no, it didn't line up with my 2 week history in the actual databases

gruntlord6•7/24/25, 5:40 AM

thanks for that though

Silvan•7/24/25, 12:15 PM

@Community Champion :bibicat:

mrcubi•7/24/25, 12:21 PM

hi i am using cloudflare d1 service and the quote of the service i have 25 billion row read free monthly but i still get bill of that 25 billion monthly can i know why

Mmrcubi hi i am using cloudflare d1 service and the quote of the service i have 25 billi...

Silvan•7/24/25, 12:22 PM

you are on the workers paid plan right

mrcubi•7/24/25, 12:23 PM

Yes

Silvan•7/24/25, 12:23 PM

were you billed 25 dollars?

mrcubi•7/24/25, 12:24 PM

I used 38 billion in July and I got a bill for 38 billion

Silvan•7/24/25, 12:24 PM

so 38 dollar?

mrcubi•7/24/25, 12:24 PM

Yes

mrcubi•7/24/25, 12:25 PM

Silvan•7/24/25, 12:25 PM

?billing

SuperHelpflare•7/24/25, 12:25 PM

Cloudflare's billing system is undergoing some changes/migration. During this period, you may notice extra bills, incorrect bills, or other issues.

To resolve these issues, create a Support ticket by using the following link (you must be logged into your Cloudflare account): https://dash.cloudflare.com/?to=/:account/support

Note that all accounts, even Free plan accounts, can create billing tickets. Tickets are the only way to resolve billing issues; we cannot help you in this Discord.

Silvan•7/24/25, 12:25 PM

best to ask them then

< Sun8ox />•7/24/25, 6:23 PM

Hi, I have a question, how do I use Astro with D1 using bindings? (how do I access database binding) Im searching anywhere for any good documentation but cant find any. Nothing really works.

<< Sun8ox />Hi, I have a question, how do I use Astro with D1 using bindings? (how do I acce...

Silvan•7/24/25, 7:24 PM

you need the cloudflare adapter. rest can be found here https://docs.astro.build/en/guides/integrations-guide/cloudflare/#cloudflare-runtime

Docs

@astrojs/cloudflare

Learn how to use the @astrojs/cloudflare adapter to deploy your Astro project.

SSilvan you need the cloudflare adapter. rest can be found here https://docs.astro.build...

< Sun8ox />•7/24/25, 7:31 PM

thanks

Mmrcubi Yes

1984 Ford Laser•7/25/25, 1:17 PM

Line items on invoices don't show the included quota with your plan. So you actually used 25+37billion rows

1984 Ford Laser•7/25/25, 1:17 PM

Best to check D1 metrics via GraphQL API to dig deeper

George•7/25/25, 6:42 PM

Hi, may I ask for some help?
So I'm using worker with D1 just fine and want to utilize the Read replication feature. I changed my worker code of SELECTSELECT statements to use env.DB.withSession() but get this error:

[UNKNOWN_ERROR] POST /login: Error: Cannot read properties of undefined (reading 'withSession')Internal cause: N/A
Stack: TypeError: Cannot read properties of undefined (reading 'withSession')
at index.js:3007:28
at async dispatch (index.js:929:17)
at async cors2 (index.js:3204:5)
at async dispatch (index.js:929:17)
at async logger2 (index.js:2495:5)
at async dispatch (index.js:929:17)
at async index.js:1773:26

I confirm I've enabled Read replicationRead replication in the D1 setting, any clue what's wrong?
btw I'm using hono.dev framework but I don't think it's related. Thanks

James•7/26/25, 1:39 PM

Can you share your full code and a way to reproduce? From that error, env.DBenv.DB is undefined.

Pepper•7/26/25, 3:18 PM

I was trying to add rows to my table and I somehow ended up with 6 rows that all have the same value for the primary key, which I didn't think was possible. The issue is that I am now unable to edit any of them. Whenever I make edits in the cloudflare d1 studio they all get reverted when I refresh the table (I am hitting save changes). Deleting the rows also doesn't work, they all reappear when I refresh the table. Does anyone know how I can get rid of these rows? Thanks.

Pepper•7/26/25, 3:28 PM

I think that part of the issue is that when I try to run an sql query selecting by the duplicate primary key it returns no results. But when I select * all of the duplicates appear. So when the d1 studio is trying to make delete requests it isn't able to delete anything because it isn't finding anything

Pepper•7/26/25, 3:34 PM

Since my primary keys were numbers, I was able to fix the problem and delete all of the duplicates by doing:
WHERE primary_key > 499
instead of:
WHERE primary_key = 500

Larry•7/26/25, 5:18 PM

I'm still more than simply curious to understand how an impossible thing happened. Can you run, PRAGMA table_info(table_name)PRAGMA table_info(table_name) and show us the output? I'm wondering if the table was originally created with a different primary key and changed your code to something like, CREATE TABLE IF NOT EXISTS...CREATE TABLE IF NOT EXISTS... with the primary key that you now think is the primary key but it actually isn't.

`D1_ERROR: internal error; reference = ...` - what is this?

Similar Threads

Similar Threads

Similar Threads