ahh, thanks for that, it worked! I actually removed the default so that I can manually set it via ep

ahh, thanks for that, it worked! I actually removed the default so that I can manually set it via epoch time stamp generated via my worker. I am reading the worker time could slightly differ than the D1 time, so thinking not setting it via the D1 is a better idea, your thoughts?

Ccaptain ahh, thanks for that, it worked! I actually removed the default so that I can ma...

Isaac McFadyen•9/24/23, 4:52 PM

Both D1 time and Worker time aren't exact, because of security concerns. But they're close.

Isaac McFadyen•9/24/23, 4:53 PM

Usually milliseconds difference between the observed time and the real time, so unless you need that precision you should be fine to set it wherever.

IIsaac McFadyen Usually milliseconds difference between the observed time and the real time, so ...

captainOP•9/24/23, 4:55 PM

my use case is tracking webhooks from stripe and then showing the non paid messages to the user and telling them to get their act together! So wanting the timestamp webhook sending to act as the time stamp that i can then sort by

Ccaptain my use case is tracking webhooks from stripe and then showing the non paid messa...

Isaac McFadyen•9/24/23, 4:55 PM

Ah, OK. Does Stripe provide a timestamp within the webhook message?

Isaac McFadyen•9/24/23, 4:55 PM

If so you could use that.

IIsaac McFadyen Ah, OK. Does Stripe provide a timestamp within the webhook message?

captainOP•9/24/23, 4:57 PM

true, but either way best not to have D1 set it i think

captainOP•9/24/23, 4:58 PM

anyways thanks for pointing out that error, D1, very exciting stuff rather than using all this crazy logic i have written around KV storage

Ccaptain true, but either way best not to have D1 set it i think

Isaac McFadyen•9/24/23, 4:58 PM

Yeah - D1 setting it would occur on insert, not on message received.

Ccaptain anyways thanks for pointing out that error, D1, very exciting stuff rather than ...

Isaac McFadyen•9/24/23, 4:58 PM

No worries, happy to help

captainOP•9/24/23, 5:14 PM

more D1 questions! I created a D1 database many many months ago, but now reading this: https://blog.cloudflare.com/d1-turning-it-up-to-11/ it looks like things have changed and running the command wrangler d1 info <DB_NAME> I can see my db is using version alpha. The question, is there a single command to move this db to the newer version? Or is the proper move to simply manually recreate the DB and update the ID's in the TOML file

The Cloudflare Blog

D1: We turned it up to 11

We've been heads down iterating on D1, and we've just shipped a major new version that's substantially faster, more reliable, and introduces Time Travel: the ability to restore a D1 database to any point in time.

captainOP•9/24/23, 5:24 PM

@Matt is "INSERTZ INFO" found on https://developers.cloudflare.com/d1/learning/debug-d1/ a typo? Or is this something specific for D1?

Debug D1 · Cloudflare D1 docs

D1 allows you to capture exceptions and log errors returned when querying a database. To debug D1, you will use the same tools available when …

Ccaptain @Matt is "INSERTZ INFO" found on https://developers.cloudflare.com/d1/learning/d...

Isaac McFadyen•9/24/23, 5:28 PM

That's an intentional typo, yup.

captainOP•9/24/23, 6:25 PM

question, is it not possible to backup an alpha database and then restore it into a beta database. When I attempt the basic method of restoring in that situation i get:
Internal error: failed to restore backup: Experimental DBs do not support backups yet:
74244d9f-afa8-4ca2-ab7f-8067f4505c97 (v3) [code: 7500]

Ccaptain question, is it not possible to backup an alpha database and then restore it int...

Isaac McFadyen•9/24/23, 6:43 PM

I believe you need to dump it to a SQL file and reupload.

IIsaac McFadyen I believe you need to dump it to a SQL file and reupload.

captainOP•9/24/23, 7:12 PM

success!

captainOP•9/24/23, 7:44 PM

another question...did a dump from an alpha database, created a new database, importanted the dump, success. NOW to the that new database I'm running the sql:

DROP TABLE IF EXISTS messages;
CREATE TABLE IF NOT EXISTS messages (
  title TEXT NOT NULL,
  body TEXT NOT NULL,
  amount INTEGER NOT NULL,
  color TEXT NOT NULL,
  event_type TEXT NOT NULL,
  e_id TEXT NOT NULL, 
  i_id TEXT NOT NULL, -- invoice id
  o_id TEXT NOT NULL, --
  s_id TEXT NOT NULL, -- subscription id
  p_id TEXT NOT NULL, -- payment id
  c_id TEXT NOT NULL, -- customer id
  service TEXT NOT NULL, 
  email TEXT NOT NULL,
  name TEXT NOT NULL,
  link TEXT NOT NULL,
  related_locations TEXT NOT NULL,
  time_stamp INTEGER NOT NULL, -- time_stamp INTEGER NOT NULL DEFAULT CURRENT_TIMESTAMP,
  customer_dismiss INTEGER, -- 0 or 1
  location_id TEXT NOT NULL,
  status TEXT NOT NULL, -- new or dismissed
  status_type TEXT NOT NULL, -- notification / 
  other TEXT NOT NULL
);

DROP TABLE IF EXISTS messages;
CREATE TABLE IF NOT EXISTS messages (
  title TEXT NOT NULL,
  body TEXT NOT NULL,
  amount INTEGER NOT NULL,
  color TEXT NOT NULL,
  event_type TEXT NOT NULL,
  e_id TEXT NOT NULL, 
  i_id TEXT NOT NULL, -- invoice id
  o_id TEXT NOT NULL, --
  s_id TEXT NOT NULL, -- subscription id
  p_id TEXT NOT NULL, -- payment id
  c_id TEXT NOT NULL, -- customer id
  service TEXT NOT NULL, 
  email TEXT NOT NULL,
  name TEXT NOT NULL,
  link TEXT NOT NULL,
  related_locations TEXT NOT NULL,
  time_stamp INTEGER NOT NULL, -- time_stamp INTEGER NOT NULL DEFAULT CURRENT_TIMESTAMP,
  customer_dismiss INTEGER, -- 0 or 1
  location_id TEXT NOT NULL,
  status TEXT NOT NULL, -- new or dismissed
  status_type TEXT NOT NULL, -- notification / 
  other TEXT NOT NULL
);

and getting the following error:

SQL code did not contain a statement. [code: 7500]

This same statement seemed to work on the alpha database, thoughts?

Resolved, I had a comment with a semi colon which is an bug documented here: https://github.com/cloudflare/workers-sdk/issues/3892 whats interesting is this bug only applies to the beta D1 and not the alpha

GitHub

🐛 BUG: d1 execution with comment including semicolon causes SQL cod...

Which Cloudflare product(s) does this pertain to? D1 What version(s) of the tool(s) are you using? 3.6.0 What version of Node are you using? 20.5.1 What operating system are you using? Mac Describe...

MMatt Silverlock Thanks: looking into why it's sticking.

benthecarman•9/24/23, 9:42 PM

it's still down, any updates here?

Bbenthecarman it's still down, any updates here?

Matt Silverlock•9/24/23, 9:43 PM

We’re still chasing down the cause of the bug. I’ll provide an update when we have one.

benthecarman•9/24/23, 9:44 PM

Is it possible we can just get the sqlite file? We are just trying to move off of this because it is unusable

Bbenthecarman Is it possible we can just get the sqlite file? We are just trying to move off o...

Matt Silverlock•9/24/23, 11:17 PM

There isn't the concept of a 'single SQLite file' in D1 (as that wouldn't allow us to be distributed).

Bbenthecarman Is it possible we can just get the sqlite file? We are just trying to move off o...

Matt Silverlock•9/24/23, 11:18 PM

And I understand the frustration: we don't want these kind of bugs, but this is an alpha/beta.

benthecarman•9/24/23, 11:20 PM

okay, if guys can get it back up for a couple hours we can transfer off of it

Bbenthecarman okay, if guys can get it back up for a couple hours we can transfer off of it

Matt Silverlock•9/24/23, 11:41 PM

Can you confirm it's running now?

MMatt Silverlock Can you confirm it's running now?

benthecarman•9/25/23, 12:04 AM

Back up, thanks

Mrxbox98•9/25/23, 12:30 AM

Is there a way to get a database size limit increase? I have a multi-gb sqlite database that I wish to use with d1.

MMrxbox98 Is there a way to get a database size limit increase? I have a multi-gb sqlite d...

benthecarman•9/25/23, 12:40 AM

I wouldn't use d1, it only has like 20% uptime

Cyb3r-Jak3•9/25/23, 12:41 AM

I definitely doesn't have that low. I've been using it since the very start and had one issue

benthecarman•9/25/23, 1:01 AM

we just had like a 20 hour downtime

Bbenthecarman Back up, thanks

captainOP•9/25/23, 1:07 AM

Just curious, have you been doing anything complicated with your DB, is it a big DB? I haven't been having any issues but would like to look ahead to understand posisble concerns. And you are seeing only 20% uptime?

Ccaptain Just curious, have you been doing anything complicated with your DB, is it a big...

benthecarman•9/25/23, 1:09 AM

db is around 100mb, we basically just used it like a kv store for encrypted cloud storage

Bbenthecarman db is around 100mb, we basically just used it like a kv store for encrypted clou...

captainOP•9/25/23, 1:10 AM

was it on alpha or beta and did you find out why it went down?

captainOP•9/25/23, 1:10 AM

what errors did u get?

benthecarman•9/25/23, 1:10 AM

20% is an exageration but it would go down for about a day every week

Ccaptain was it on alpha or beta and did you find out why it went down?

benthecarman•9/25/23, 1:10 AM

CF doesn't tell us why, one said to stop using the product, we just can't connect to the db

Bbenthecarman CF doesn't tell us why, one said to stop using the product, we just can't connec...

captainOP•9/25/23, 1:11 AM

what region are you hinting with? I'm still on alpha but haven't had any downtime...ever. Was going to move it to beta DB today.

benthecarman•9/25/23, 1:12 AM

we have people globally

Bbenthecarman we have people globally

captainOP•9/25/23, 1:12 AM

got it

captainOP•9/25/23, 1:13 AM

well hopefully they are seeing your bugs

benthecarman•9/25/23, 1:13 AM

half the time they were unaware their system was down lol

captainOP•9/25/23, 1:16 AM

D1 is a game changer and so is workers, god speed!

benthecarman•9/25/23, 1:16 AM

it would have been perfect for our usecase if it actually worked

captainOP•9/25/23, 1:18 AM

it wil be perfect for many, but what they are attempting to accomplish is super difficult, but workers is pretty amazing, kv is pretty solid, so just gotta be patient with the bleeding edge tech. I'm attempting to rewrite things from KV to D1 to shadow what I am doing with KV, but i'm still leaving KV as the baseline till D1 is out of ALpha/beta

captainOP•9/25/23, 1:19 AM

but perhaps if you need it now, you should be exploring durable objects more with nosql

Bbenthecarman I wouldn't use d1, it only has like 20% uptime

Matt Silverlock•9/25/23, 1:27 AM

This has been one of the very rare cases where we’ve seen this — it’s certainly not 20% (or even 5%). I get that you’re frustrated from your experience with D1, but 20% isn’t true.

MMatt Silverlock This has been one of the very rare cases where we’ve seen this — it’s certainly ...

captainOP•9/25/23, 1:43 AM

are you finding the version alpha or beta is currently more stable?

Ccaptain are you finding the version alpha or beta is currently more stable?

Matt Silverlock•9/25/23, 1:46 AM

The new storage subsystem is substantially more reliable and has been for months.

Matt Silverlock•9/25/23, 1:46 AM

(That’s why we made it the default)

MMatt Silverlock (That’s why we made it the default)

captainOP•9/25/23, 1:47 AM

copy that...in the midst of transfering over now

captainOP•9/25/23, 1:47 AM

enjoying being around for the cloudflare ride!

Adi•9/25/23, 3:18 AM

Genuinely can't wait to see what's announced for birthday week!
Hopefully some news around D1 as well

DROP TABLE IF EXISTS messages; CREATE TABLE IF NOT EXISTS messages ( title TEXT NOT NULL, body TEXT NOT NULL, amount INTEGER NOT NULL, color TEXT NOT NULL, event_type TEXT NOT NULL, e_id TEXT NOT NULL, i_id TEXT NOT NULL, -- invoice id o_id TEXT NOT NULL, -- s_id TEXT NOT NULL, -- subscription id p_id TEXT NOT NULL, -- payment id c_id TEXT NOT NULL, -- customer id service TEXT NOT NULL, email TEXT NOT NULL, name TEXT NOT NULL, link TEXT NOT NULL, related_locations TEXT NOT NULL, time_stamp INTEGER NOT NULL, -- time_stamp INTEGER NOT NULL DEFAULT CURRENT_TIMESTAMP, customer_dismiss INTEGER, -- 0 or 1 location_id TEXT NOT NULL, status TEXT NOT NULL, -- new or dismissed status_type TEXT NOT NULL, -- notification / other TEXT NOT NULL );

ahh, thanks for that, it worked! I actually removed the default so that I can manually set it via ep

Similar Threads