Hi there, I'm in a scenario where I update/add data periodically to a local d1 db and need to sync t

Hi there, I'm in a scenario where I update/add data periodically to a local d1 db and need to sync that data progressively with the remote one. Is there some way to connect to the remote db from a node script so I can run the diffing logic (primarily based on

last_updated

last_updated

last_updated

last_updated fields) and push the updates from the local db to the remote one, or is

wrangler d1 execute

wrangler d1 execute

wrangler d1 execute

wrangler d1 execute the only way currently to run queries on a remote d1 db? Thanks in advance.

Original message was deleted

KaiM•10/30/24, 12:44 AM

Just following up here. With Smart Placement enabled the Worker was running in Brisbane - still very much the opposite side of the world to the D1 database. Any ideas?

Nnaourass Hi there, I'm in a scenario where I update/add data periodically to a local d1 d...

Isaac McFadyen•10/30/24, 12:45 AM

You can use the Cloudflare API, one sec.

Isaac McFadyen•10/30/24, 12:45 AM

https://developers.cloudflare.com/api/operations/cloudflare-d1-query-database

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

sz19•10/30/24, 12:51 AM

The d1 errors are back again in my end

TomK•10/30/24, 3:06 AM

On my end, there have been no recurrences of the D1 errors in the last hour.

TomK•10/30/24, 3:13 AM

Does anyone know how to find the location selected when a D1 database was created? I've looked in both the dash and the wrangler command, but I can't find any relevant information.

TTomK Does anyone know how to find the location selected when a D1 database was create...

Isaac McFadyen•10/30/24, 3:17 AM

Try

wrangler d1 info <dbname>

wrangler d1 info <dbname>

IIsaac McFadyen Try `wrangler d1 info <dbname>` 😄

TomK•10/30/24, 3:39 AM

There is no location information in the info command.

TomK•10/30/24, 3:40 AM

TomK•10/30/24, 3:40 AM

wrangler 3.83.0

Isaac McFadyen•10/30/24, 3:40 AM

If you do

wrangler d1 list

wrangler d1 list

do you see productionproduction as the version?

Isaac McFadyen•10/30/24, 3:41 AM

There should be a running_in_regionrunning_in_region field right between the size and read query count... strange that you don't see it

FFlikTeoh https://discord.com/channels/595317990191398933/799437470004412476/1298726326265...

Chaika•10/30/24, 3:55 AM

anything more interesting output if you try info with the --json flag after the name?

FFlikTeoh Nope. I had other db that wasn't using location hint though. Those shows its cur...

Chaika•10/30/24, 3:58 AM

What do you mean "Current regions"? D1 only runs in a single region/location, if you don't specify a location hint it just does it off the requestor's location, and it'll stay there forever

FFlikTeoh Yeah. What I mean is that, other db that was created without location hints, sho...

Chaika•10/30/24, 4:01 AM

hmm I can't reproduce that. If you have a second, try making another with the same location hint? Just trying to understand scope, I'd guess it's a weird one-off issue
The --json option is just returning what the database get endpoint responds with + graphql, so it missing is api being confused/not a specific wrangler thing

TTomK Click to see attachment

Chaika•10/30/24, 4:04 AM

what's the date of creation on that db?

Chaika•10/30/24, 4:05 AM

oop sorry I meant tomk, yea I saw yours, maybe just an issue for dbs created in a specific period

FFlikTeoh Should I create a new profile db? This one also takes 1.5s to 4s on initial conn...

Chaika•10/30/24, 4:07 AM

how much of that is query time?

FFlikTeoh Under 100ms. It's just the connection time is long for every initial request (wh...

Chaika•10/30/24, 4:09 AM

there's going to be some cold start time but shouldn't be that much

Chaika•10/30/24, 4:09 AM

where are you getting the response times from? The D1 Console wouldn't be a good indictator of response time for example, all requests still have to go you -> core -> d1 instance afaik

FFlikTeoh Yeah it was in the 1.5s to 2s range last week. Yesterday it was at 4.4s. Which w...

Chaika•10/30/24, 4:12 AM

it wouldn't go to sleep if it was constantly being accessed, if you're getting 800ms on some subsequent requests that's still pretty high though

FFlikTeoh This was from insights: ``` { "query": "SELECT * FROM profiles WHERE user_id...

Chaika•10/30/24, 4:13 AM

hmm that one's 0.3ms on avg

Chaika•10/30/24, 4:13 AM

800ms is really high if you have smart placement on, possible that it being borked is connected to the api not showing the location though

FFlikTeoh I can't remember the exact location hint I gave when I created that db. But I th...

Chaika•10/30/24, 4:14 AM

with smart placement on, what was the cf-placement response header?

Chaika•10/30/24, 4:20 AM

Smart Placement is currently ignored when making RPC calls. If Smart Placement is enabled for Worker A, and Worker B declares a Service Binding to it, when Worker B calls Worker A via RPC, Worker A will run locally, on the same machine.

Chaika•10/30/24, 4:20 AM

even if it did work if the underlying d1-calling worker was called via rpc the header wouldn't be passed up anyway

Chaika•10/30/24, 4:23 AM

means it's running wherever the original worker is running, which would be a cf location close to the requestor

Chaika•10/30/24, 4:25 AM

essentially yea. Assuming APAC is your location/the location you're calling the worker from. Not 100% sure about normal, still seems a bit high to me, don't understand how you're measuring the ~400-800ms response time though

CChaika what's the date of creation on that db?

TomK•10/30/24, 4:44 AM

2024-06-07T14:23:14.579Z

TomK•10/30/24, 4:49 AM

I noticed that some databases display "running_in_region," while others do not. I suspect that the ones not displaying this might be due to not specifying a location during creation, and the system automatically determined one.

JJames It's absolutely a valid concern and I don't think anyone is arguing that - if yo...

justaname•10/30/24, 5:01 AM

Hey this is kinda off topic for d1 but was just reading through and a question came to my head is their any way of protecting againts a DNS going down. Like i think cloudflare DNS went down a few times before is it just pray it doesnt happen or is their any way around that?

Hi there, I'm in a scenario where I update/add data periodically to a local d1 db and need to sync t

Similar Threads

Similar Threads

Similar Threads