Ok so I enabled read replication and that helped a lot, so the D1 must have been placed in far away
Ok so I enabled read replication and that helped a lot, so the D1 must have been placed in far away land somewhere
TThanks for pointing me in right direction
CIt creates a read-only replica in the same region as the primary, so it was likely just placed closer I'd guess. Interesting though, yea Asia Pacific is a bit of a messy region.
Afaik D1 doesn't actually expose the underlying placement of the instances anywhere, you can only tell by poking each location and looking at latency
Afaik D1 doesn't actually expose the underlying placement of the instances anywhere, you can only tell by poking each location and looking at latency
LWe are planning to add
So, you will have full transparency on where it is located for every query (and where it's moving when it moves). Replicas will also be reporting their own location if they serve queries themselves.
colo airport code in the D1 query responses as well, similar to served_by_region. It's annoying to not know where your DB is located (I know, we should do this sooner!).So, you will have full transparency on where it is located for every query (and where it's moving when it moves). Replicas will also be reporting their own location if they serve queries themselves.
CThat would be super helpful, espec for Asia Pacific region :MeowHeartCloudflare:
MAround 7am UTC, my D1 database started performing normally - with P99(durationMS) falling to ~400ms - but now (7pm UTC) it's getting worse again, P99(durationMS) is 4 seconds and rising. Nothing about my queries is changing, the SQLite query latency is <3ms. Anyone seen this before, and any ideas what to do? Noisy neighbours seems a possibility. There may be some fragmentation issues (I dropped a couple of large tables), but that feels like it would show up in query latency.
TCorrect me if I’m wrong but currently the D1 instances don’t move right? Once created in zone within a region they stay there?
I think it would be helpful to be able to just say run this worker next to my d1 because for APAC especially you end up with the traces above if run far apart. Then if it turns out traffic is primarily coming from somewhere else then they could move in tandem to keep latency down
I think it would be helpful to be able to just say run this worker next to my d1 because for APAC especially you end up with the traces above if run far apart. Then if it turns out traffic is primarily coming from somewhere else then they could move in tandem to keep latency down
Mout of memory: SQLITE_NOMEM
I'm facing this issue, I'm using D1 to keep track of requests for my api, and now I'm getting more and more requests more than 10m records reaching my 10gb limit.
I'm trying to delete them but getting that error out of memory is there any better way to delete them without issues?
I'm facing this issue, I'm using D1 to keep track of requests for my api, and now I'm getting more and more requests more than 10m records reaching my 10gb limit.
I'm trying to delete them but getting that error out of memory is there any better way to delete them without issues?
FYou can try to delete the rows in multiple batches and not all at once.
FThey should stay in the same region, but they can move between different colos
MI'm doing that already, but it's taking so much time trying to delete 10M records
also do you think Workers analytic engine will be good for my usecase, after tracking every day, I group the data and send to another d1, and schedule a cron to delete the rows in batches but again it takes so much time and doesn't delete all the records for that day.
with the analytics Engine if it's possible can we delete all at once (eg for a day 1-2M records) without the limits?
also do you think Workers analytic engine will be good for my usecase, after tracking every day, I group the data and send to another d1, and schedule a cron to delete the rows in batches but again it takes so much time and doesn't delete all the records for that day.
with the analytics Engine if it's possible can we delete all at once (eg for a day 1-2M records) without the limits?
T@Manuel Ogomigo you could try to create a table per day and then just drop it. The benefit of that is I also think you wouldn't pay for the deletes
MOh this is a good idea 
, I’ll work on that. Thank you so much
, I’ll work on that. Thank you so much
PHI, I'm looking for a good way to distinguish between dev mode and production mode.
I'm currently working on a project in Astro and have connected it to d1, but I don't understand how to use local sqlite.
I currently have a drizzle.config.local.ts file with the following :
Is this feasible?
Also because to run in dev I can't use the wrangler cli but
I'm currently working on a project in Astro and have connected it to d1, but I don't understand how to use local sqlite.
I currently have a drizzle.config.local.ts file with the following :
Is this feasible?
Also because to run in dev I can't use the wrangler cli but
astro dev.
Dcan someone explain to me the metrics here? Is it, that this single query caused 12.8k row reads for every execution, or that it caused 12.8k row reads total over 36k executions? If its the first case, why does it cause so many reads, when it should use the primary key here?
FHey, I'm not sure 100% sure, but I think it's the sum for all your queries based on the graphql documentation that is used to retrieve the number of rows read.
Having said that, the best way for you to check would be to
Having said that, the best way for you to check would be to
explain this specific query to find out what sqlite is doing under the hood https://sqlite.org/lang_explain.html.
JHere's how I do it - since the drizzle config is just a typescript file, I run some synchronous code to check for the existence of a
.dev.vars file and if I find one, I read it with an NPM package that can parse .env files so I can get the development vars. During a production build the .dev.vars file won't exist because it never goes into the repo. (With this tactic, I can have just the one standard drizzle.config.ts file and not have a separate local version of it.)JI do the same thing for my Vite-based SPA, in the vite config file I read the
.dev.vars file using a .env parser in order to get some details that I need related to Vite's devserver proxy config.
GThat’s what I do as well with durable objects that do 500k+ rows each day, drop the table for those days on a Regular basis and keep the recent ones
GYou can even have a worker cron handle that
CHello Cloudflare developers, I am facing issue with d1 export api. I've also posted details in https://community.cloudflare.com/t/d1-export-api-not-working/862800 Can some one please take a look at this?
MYeah really good idea. U've done that arleady and it's working fine! 
RWhat does https://developers.cloudflare.com/api/resources/d1/subresources/database/methods/get/ do if the database already exists?
Interact with Cloudflare's products and services via the Cloudflare API
HRyeah
Rapologies
FThe API returns a 400 if the database name already exist
Ralright thanks
Rthanks
HThanks for the assist, Florent!
FSo I am attempting to follow the "One D1 database per user" pattern since each user will uploading a sqlite database.
I have a pipeline where I process the sqlite DB, create a D1 db if one doesn't exist for the user, populate it and then bind it to my app worker.
The problem I'm running into is when I deploy my app worker the dynamic D1 bindings I added get removed since they aren't in the
Any ideas on how to handle that? I was thinking I could write a script to grab the current bindings before deploy but not exactly foolproof.
I have a pipeline where I process the sqlite DB, create a D1 db if one doesn't exist for the user, populate it and then bind it to my app worker.
The problem I'm running into is when I deploy my app worker the dynamic D1 bindings I added get removed since they aren't in the
wrangler.json. Any ideas on how to handle that? I was thinking I could write a script to grab the current bindings before deploy but not exactly foolproof.
LHow many DBs are you expecting? If it's more than a few tens, low hundreds, I strongly suggest using Durable Objects directly instead. You can create millions of SQLite databases using the same DO Namespace binding, instead of having to add a binding for every D1 DB.
Other customers doing dynamic wrangler bindings with D1 handle it entirely through the API, due to the issues you faced with wrangler not being in sync.
Other customers doing dynamic wrangler bindings with D1 handle it entirely through the API, due to the issues you faced with wrangler not being in sync.
FRealistically less than 100 users is probably right.
I'll read up on durable objects and what that solution would look like to move to.
I'll read up on durable objects and what that solution would look like to move to.
EI know you said you can't promise me a response from the team but on the faint chance that you only forgot to get back to me - were you able to get any information on this?
LYeah, we will expose the bookmark/PITR operations in the REST API next week hopefully (or soon after) to have parity with wrangler. Need some cleanup work to complete first.
EThanks, that's great news!
KI wonder how the rows_read is calculated. I have a dataset with an index that will match only one row INDEX(a, b) that would return 2 rows read. I will drop that index and create UNIQUE INDEX(a, b) and that would return only 1 rows read.
VI'd like to report that having 32+ (currently have 35) tables in the D1 studio, the scroll overflow stops and you can't see beyond that (but expanding dev tools shows the divs correctly there)
JI am seeing a really strange issue in D1 studio. When scrolling through my data, the scroll bar says that I am at the bottom when I am ~2 rows away from the actual end of the page. This makes it difficult to see that data because when I scroll to see it, as soon as I stop scrolling it will scroll back up and hide the data. The buttons to move between pages of data are also hidden. When I double click on one of those rows to edit a value, the page suddenly jumps down and I am able to see the bottom two rows normally, but all of the stuff right above the data in D1 studio disappear (tabs, query bar, add/remove row buttons, discard/commit buttons). This is happening to me on firefox. I have not tested any other browser
LThanks for the report, I will let someone on the team know and we will investigate.
VThanks for report. Fix is on the way. 
WHaving the same issue, also sometimes locks the scroll and can't fully scroll down - probably related.
CIs this actually faster? Curious since in other relational dbs there’s often a file per table so makes sense drop is quicker. However in SQLite since it’s one file wouldn’t it still need to find the rows?
GYes, its significantly faster. SQlite still has more overhead in doing tons of individual deletes vs doing it for the entire table.
GSqlite may be one file but it still has pages inside the file it needs to rearrange when you make writes or deletes
CThanks good to know, do you know if DO bills drops?
Gits the same net amount of changes regardless of how you do it, the difference is how long it takes
Gthat said durable objects do not currently bill for reads/writes like d1 does, but eventually they will
MHey guys, I'm trying to use some math lib with D1 but it's not possible.
D1_ERROR: not authorized to use function: POWER
D1_ERROR: not authorized to use function: POWER