Cloudflare Developers•2y ago

hi, does cf d1 support/implement the `generate_series` function of sqlite?

hi, does cf d1 support/implement the

generate_series

generate_series

function of sqlite?

© 2026 Hedgehog Software, LLC

More

Communities Docs About Terms Privacy

Revan

RRevan hi, does cf d1 support/implement the `generate_series` function of sqlite?

Max (@rozenmd)

M

Max (@rozenmd)•3/25/24, 9:06 AM

doesn't look like we've enabled that extension yet, if you file a feature request for it here, that'd be awesome: https://github.com/cloudflare/workerd/issues

Revan

R

RevanOP•3/25/24, 9:11 AM

sure, will do

Revan

R

RevanOP•3/25/24, 9:21 AM

I made the issue here: https://github.com/cloudflare/workerd/issues/1888

wasnt really sure what to write

Revan

RRevan I made the issue here: https://github.com/cloudflare/workerd/issues/1888 wasnt ...

Max (@rozenmd)

M

Max (@rozenmd)•3/25/24, 9:53 AM

perfect, thank you!

sabin

S

sabin•3/25/24, 1:42 PM

how does d1 compare to turso?

sabin

S

sabin•3/25/24, 1:42 PM

couldnt find anything online abt it

AaronO

AAaronO Could you elaborate a little on the eager load proportional to DB size ? As an ...

Max (@rozenmd)

M

Max (@rozenmd)•3/25/24, 5:55 PM

Probably a better question for @kenton (when they're around)

Max (@rozenmd)

MMax (@rozenmd)D1 is built to work to the strengths of Cloudflare's network, so we can't offer ...

itsmatteomanf

I

itsmatteomanf•3/25/24, 5:58 PM

The biggest limit here (but I know you are working on it, so hopefully soon it gets fixed) is that you can't dynamically connect to a database.
If I have millions of users, how can I configure a database per user (if I can work around the limits that cross-user queries are impossible, at the moment) and reach it via Workers?

AaronO

AAaronO Could you elaborate a little on the eager load proportional to DB size ? As an ...

kenton

K

kenton•3/25/24, 6:10 PM

The way you are imagining it works is the way D1 v1 and v2 worked... which turned out to be way too slow. In v3 we pull the full database down to local disk upfront.

AaronO

AAaronO Did v1 and v2 use the local disk as cache too ? In the limit, eagerly pulling th...

kenton

K

kenton•3/25/24, 6:19 PM

I believe pages were actually cached in RAM (which is why the database size limit was 100MB). I wasn't closely involved with v1 or v2 though so can't comment in that much detail.

kenton

K

kenton•3/25/24, 6:20 PM

v3 is based on a new storage engine which I described a bit here: https://twitter.com/KentonVarda/status/1659551757796515846

Kenton Varda (@KentonVarda) on X

So I've spent the last ~three months writing a new distributed storage engine based on SQLite and it has been FUN!

https://t.co/NVorwS46x8

Twitter

•

5/19/23, 1:29 PM

kenton

K

kenton•3/25/24, 6:20 PM

(still haven't actually published the alluded blog post, sorry)

dave

D

dave•3/25/24, 7:21 PM

is there a way to skip to the end of a table in the Cloudflare Dashboard?

dave

D

dave•3/25/24, 7:22 PM

I have a ton of rows and it's a huge pain

AaronO

AAaronO Thanks, I'll look forward to the blog post

Isaac McFadyen

I

Isaac McFadyen•3/26/24, 2:27 AM

~~I believe the blog post was already published here: https://blog.cloudflare.com/d1-turning-it-up-to-11~~ Missed the fact that was referencing a yet-unpublished, longer blog post.

Kartoffel🟢🟡🔴

K

Kartoffel🟢🟡🔴•3/26/24, 3:54 PM

Kartoffel🟢🟡🔴

K

Kartoffel🟢🟡🔴•3/26/24, 3:54 PM

cannot load d1

Original message was deleted

Kartoffel🟢🟡🔴

K

Kartoffel🟢🟡🔴•3/26/24, 3:55 PM

thanks

Kartoffel🟢🟡🔴

KKartoffel🟢🟡🔴Click to see attachment

Max (@rozenmd)

M

Max (@rozenmd)•3/26/24, 4:13 PM

it's back

Jan Piotrowski (janpio)

J

Jan Piotrowski (janpio)•3/26/24, 10:10 PM

Is there a suggested way to "export" the schema of a remote D1 database to use it locally?

Jan Piotrowski (janpio)

JJan Piotrowski (janpio)Is there a suggested way to "export" the schema of a remote D1 database to use i...

Max (@rozenmd)

M

Max (@rozenmd)•3/27/24, 5:22 AM

natively, there will be soon - until then there's https://github.com/nora-soderlund/cloudflare-d1-backups

Max (@rozenmd)

MMax (@rozenmd)natively, there will be soon - until then there's https://github.com/nora-soderl...

Jan Piotrowski (janpio)

J

Jan Piotrowski (janpio)•3/27/24, 9:05 AM

Cool!
Can you share the approach the native solution will be going? CLI command? Library method?

Jan Piotrowski (janpio)

JJan Piotrowski (janpio)Cool! Can you share the approach the native solution will be going? CLI command?...

Matt Silverlock

M

Matt Silverlock•3/27/24, 1:19 PM

To confirm: do you mean the DB and contents, or just the schema?

Max (@rozenmd)

MMax (@rozenmd)D1 is built to work to the strengths of Cloudflare's network, so we can't offer ...

RexNormous

R

RexNormous•3/27/24, 3:18 PM

This DB-per-user vision is very exciting to me and very relevant to my application. I know D1 GA had been mentioned to be coming up soon (this or next month). Is this "sharding" of databases further out than that, and not something we should bank on anytime soon?

I'm actively working on migrating my application to D1 and wondering if sharding is something I can be planning for now as I migrate.

RexNormous

RRexNormous This DB-per-user vision is very exciting to me and very relevant to my applicati...

Max (@rozenmd)

M

Max (@rozenmd)•3/27/24, 3:30 PM

We have folks using it already via a rather clunky

user asks for a db -> create a new d1 db via api -> update worker script bindings via api

user asks for a db -> create a new d1 db via api -> update worker script bindings via api

workflow, but we'll be streamlining that as much as possible over the coming weeks/months

Max (@rozenmd)

MMax (@rozenmd)We have folks using it already via a rather clunky `user asks for a db -> create...

RexNormous

R

RexNormous•3/27/24, 3:35 PM

Interesting... that unfortunately seems like a lot of separate migrations to worry about

My unhelpful 2 cents is that it's be awesome to have selecting db instance be like choosing a Durable Object to interface with, e.g.:

env.DB.getByShardKey(id: string)

env.DB.getByShardKey(id: string)

. Then all DB's under that shard key would share a schema, get the same migrations, and effectively be "grouped' together, if that makes sense.

Original message was deleted

RexNormous

R

RexNormous•3/27/24, 3:40 PM

This is really awesome - didn't realize this existed.

Picturing my user's sign up flow though, I'd effectively need to pause a user's sign up and go into a "setup phase" for them (when a new deployment kicks off, db is created, etc.), then email them when we've "provisioned" their account. I think this would work if my app was invite-only & in beta, but unfortunately my app is on-demand and self-serve.

Rich

R

Rich•3/27/24, 4:40 PM

Does anybody know if it's possible to read environment variables in a wrangler.toml file?

For context I want to run

wrangler d1 migrations

wrangler d1 migrations

on our CI and would like to read the accountId and db Id from env vars rather than hard coding them in .toml files?

Rich

RRich Does anybody know if it's possible to read environment variables in a wrangler.t...

Hello, I’m Allie!

H

Hello, I’m Allie!•3/27/24, 5:05 PM

You cannot, unless you generate the file dynamically yourself

mare arcis

Mmare arcis In theory would d1 read replicas (when released) be faster than KV coming from a...

Max (@rozenmd)

M

Max (@rozenmd)•3/28/24, 7:29 AM

KV is architected to support millions of requests per second to a single namespace, D1 is not

Max (@rozenmd)

M

Max (@rozenmd)•3/28/24, 7:30 AM

(imo) KV would still be faster

Max (@rozenmd)

MMax (@rozenmd)KV is architected to support millions of requests per second to a single namespa...

Hello, I’m Allie!

H

Hello, I’m Allie!•3/28/24, 8:16 AM

Could D1 not horizontally replicate in a single location to handle those queries? It wouldn’t be immediate, but it should be possible to scale with demand, no?

Hello, I’m Allie!

H

Hello, I’m Allie!•3/28/24, 8:21 AM

Or well, actually, can multiple D1 instances connect to a single “stored” DB, assuming they are both in the same colo?

Hello, I’m Allie!

H

Hello, I’m Allie!•3/28/24, 8:21 AM

If so, spinning up a second read replica should only take a few milliseconds

Hello, I’m Allie!

H

Hello, I’m Allie!•3/28/24, 8:35 AM

That would be a killer feature against KV, since it would mean the ability to scale to millions/billions of RPS, while having replication time of 1-2 seconds or less(compared with KV's 1 minute+ cacheTtls).

Hello, I’m Allie!

H

Hello, I’m Allie!•3/28/24, 8:47 AM

And as for the limitations on the size of the DB, if you could shard along the primary key, then you should be able to get a lot more mileage out of it

Hello, I’m Allie!

HHello, I’m Allie!Could D1 not horizontally replicate in a single location to handle those queries...

Max (@rozenmd)

M

Max (@rozenmd)•3/28/24, 9:02 AM

oh of course - we're just getting started here, I just meant the current state

Max (@rozenmd)

MMax (@rozenmd)oh of course - we're just getting started here, I just meant the current state

Hello, I’m Allie!

H

Hello, I’m Allie!•3/28/24, 9:06 AM

Though I'm actually curious, if you are allowed to say, is D1 storage tied to a single instance, or could multiple instances connect to a single stored DB?

Hello, I’m Allie!

HHello, I’m Allie!Though I'm actually curious, if you are allowed to say, is D1 storage tied to a ...

Max (@rozenmd)

M

Max (@rozenmd)•3/28/24, 9:27 AM

depends what you mean by storage. replicas will work by basically spinning up new DOs with a copy of the db

Max (@rozenmd)

MMax (@rozenmd)depends what you mean by storage. replicas will work by basically spinning up ne...

Hello, I’m Allie!

H

Hello, I’m Allie!•3/28/24, 9:31 AM

As in, is the SQLite "file"(I know it is probably not actually a file in the background) tied to a single replica, or can multiple replicas in the same colo open that one "file" without needing to copy it?