hey guys I have almost per monthly KV write operations around 5 million, and it is costing me almost

J

Jadugar_JagguOP•9/7/23, 4:19 PM

I want to lower the cost of my services, and I was checking our D1 database it has very low cost compare to KV

JJadugar_Jaggu hey guys I have almost per monthly KV write operations around 5 million, and it ...

U

Unsmart•9/7/23, 4:25 PM

Its not recommended to use for production critical applications, but #durable-objects are a good alternative to KV that's cheaper.

JJadugar_Jaggu hey guys I have almost per monthly KV write operations around 5 million, and it ...

I

itsmatteomanf•9/7/23, 5:34 PM

What @Unsmart | Tech debt said is valid, but do note that D1 it can handle millions of write per minute easily. I have done it, consistently for hours and days on end.

I

itsmatteomanf•9/7/23, 5:34 PM

(While still reading fast, and read replicas aren’t even here yet)

JJamie If I redirect the output to a file then I see the results locally, but don't see...

I

itsmatteomanf•9/8/23, 6:43 AM

No, I don’t think I understood here… which output should be turned off? If you write it doesn’t return anything, apart from meta and success.

JJamie If I redirect the output to a file then I see the results locally, but don't see...

M

Max (@rozenmd)•9/8/23, 8:56 AM

run wrangler with the

WRANGLER_LOG

WRANGLER_LOG

WRANGLER_LOG

WRANGLER_LOG env variable set to

error

error

or

none

none

none

none

VViper What my `.env` file should be like? I have it like this. ```.env binding="DB"...

M

Matt Silverlock•9/8/23, 11:20 AM

Nothing to do with your

.env

.env

file. You need to create the Pages binding to D1 in the dashboard first: https://developers.cloudflare.com/pages/platform/functions/bindings/#d1-databases

Bindings · Cloudflare Pages docs

A binding enables your Pages Functions to interact with resources on the Cloudflare developer platform. Use bindings to integrate your Pages Functions …

VViper What my `.env` file should be like? I have it like this. ```.env binding="DB"...

M

Matt Silverlock•9/8/23, 11:20 AM

process.env

process.env

is how they are injected into Next.js within your code.

VViper It works fine in production now but, I was wondering if there's a way to test it...

J

James•9/8/23, 11:43 AM

If you want to use your bindings in

next dev

next dev

, I would suggest using a library like

cf-bindings-proxy

cf-bindings-proxy

cf-bindings-proxy

cf-bindings-proxy

JJames If you want to use your bindings in `next dev`, I would suggest using a library ...

U

Unsmart•9/8/23, 2:55 PM

Speaking of this was just bouta ask... you ever run into an issue where deployed works perfect as expected but locally it will say the command worked but it did nothing?

Not sure if this is the proxy somehow like caching a response or if its a miniflare issue

S

slowpoke•9/8/23, 8:05 PM

Baby goo-goo-ga-ga question if I want to create a table with a key that is an array or strings, is that possible / what is the syntax?

CREATE TABLE IF NOT EXISTS 'profiles' (
    'full_name' TEXT,
    'fields_of_study' Array of Text?
);

CREATE TABLE IF NOT EXISTS 'profiles' (
    'full_name' TEXT,
    'fields_of_study' Array of Text?
);

CREATE TABLE IF NOT EXISTS 'profiles' (
    'full_name' TEXT,
    'fields_of_study' Array of Text?
);

CREATE TABLE IF NOT EXISTS 'profiles' (
    'full_name' TEXT,
    'fields_of_study' Array of Text?
);

K

kian•9/8/23, 8:07 PM

I don't think there's any native support for arrays, but you could store it as a JSON array & query it as such - https://developers.cloudflare.com/d1/learning/querying-json/

Query JSON · Cloudflare D1 docs

D1 has built-in support for querying and parsing JSON data stored within a database. This enables you to:

S

slowpoke•9/8/23, 8:17 PM

Using

    'fields_of_study' TEXT ARRAY[4],

'fields_of_study' TEXT ARRAY[4],

    'fields_of_study' TEXT ARRAY[4],

'fields_of_study' TEXT ARRAY[4], seems to do something, although when running a

.prepare

.prepare

.prepare

.prepare query to insert / retrieve data I'm not sure if it's working how I think it should.

Thanks though I'll look into that. :)

S

slowpoke•9/8/23, 8:41 PM

Ok so I'm unsure if I'm understanding this documentation correctly, but it seems there are

json

json

json

json helper functions for use with your SQL queries i.e.

SELECT * ...

SELECT * ...

SELECT * ...

SELECT * ... or

db.prepare("SELECT * ...

db.prepare("SELECT * ...

db.prepare("SELECT * ...

db.prepare("SELECT * ...

I'm just trying to understand how to put it into practice -
I have

const data = db.prepare("SELECT * FROM profiles WHERE user_id = ?").bind(id).first()`

const data = db.prepare("SELECT * FROM profiles WHERE user_id = ?").bind(id).first()`

const data = db.prepare("SELECT * FROM profiles WHERE user_id = ?").bind(id).first()`

const data = db.prepare("SELECT * FROM profiles WHERE user_id = ?").bind(id).first()`

This data is already just about what I want, except my

fields_of_study

fields_of_study

fields_of_study

fields_of_study key is the string

"['Major 1', 'Major 2']"

"['Major 1', 'Major 2']"

"['Major 1', 'Major 2']"

"['Major 1', 'Major 2']".

Am I storing the data incorrectly in the column or do I need to restructure my query so that D1 will parse the array into a javascript object?

Sslowpoke Ok so I'm unsure if I'm understanding this documentation correctly, but it seems...

I

itsmatteomanf•9/8/23, 8:43 PM

I'm sure you already read this, but there are examples here

https://developers.cloudflare.com/d1/learning/querying-json/

I

itsmatteomanf•9/8/23, 8:43 PM

What's the question, though? The return value?

I

itsmatteomanf•9/8/23, 8:43 PM

The JSON helper functions help for traversing or fetching specific parts of a JSON object stored in a column.

I

itsmatteomanf•9/8/23, 8:43 PM

They run at the SQL query level, not in the Worker.

S

slowpoke•9/8/23, 8:44 PM

I just thought though, I think I could always just do JSON.parse() on the key before returning it to the client

I

itsmatteomanf•9/8/23, 8:44 PM

Yeah, you can lul

S

slowpoke•9/8/23, 8:44 PM

I think I thought that this was used to also parse JSON strings without needing the use of parse. Lmao

S

slowpoke•9/8/23, 8:45 PM

I see what you mean. This will help you traverse your JSON strings in a column without needing to like… do everything that entails to update one object key.

I

itsmatteomanf•9/8/23, 8:45 PM

Exactly, but also allows you to create virtual columns from a JSON object, avoiding the work of iterating through data or returning more than necessary.

I

itsmatteomanf•9/8/23, 8:46 PM

Or do sort of multi-query with a single query using the

json_each()

json_each()

json_each()

json_each()

S

slowpoke•9/8/23, 8:47 PM

Yep, tysm! Sorry about me overthinking it. thonk

I

itsmatteomanf•9/8/23, 8:52 PM

No prob

Original message was deleted

M

Matt Silverlock•9/9/23, 2:01 AM

Yes, they're required: https://developers.cloudflare.com/d1/get-started/#3-bind-your-worker-to-your-d1-database

Get started · Cloudflare D1 docs

This guide will instruct you through:

S

shxgreen•9/9/23, 8:57 AM

help
When I execute the command "npx wrangler d1 backup create my-database-name" to backup the D1 database, an error occurs.

# npx wrangler d1 backup create my-database-name 


✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxxxxxxx/d1/database/xxxx-xxxx-xxxx-xxxx-xxxx/backup) failed.

  The request is malformed: Experimental DBs do not support backups yet:
  xxxx-xxxx-xxxx-xxxx-xxxx (v3) [code: 7400]
  
  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

# npx wrangler d1 backup create my-database-name 


✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxxxxxxx/d1/database/xxxx-xxxx-xxxx-xxxx-xxxx/backup) failed.

  The request is malformed: Experimental DBs do not support backups yet:
  xxxx-xxxx-xxxx-xxxx-xxxx (v3) [code: 7400]
  
  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

# npx wrangler d1 backup create my-database-name 


✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxxxxxxx/d1/database/xxxx-xxxx-xxxx-xxxx-xxxx/backup) failed.

  The request is malformed: Experimental DBs do not support backups yet:
  xxxx-xxxx-xxxx-xxxx-xxxx (v3) [code: 7400]
  
  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

# npx wrangler d1 backup create my-database-name 


✘ [ERROR] A request to the Cloudflare API (/accounts/xxxxxxxxxx/d1/database/xxxx-xxxx-xxxx-xxxx-xxxx/backup) failed.

  The request is malformed: Experimental DBs do not support backups yet:
  xxxx-xxxx-xxxx-xxxx-xxxx (v3) [code: 7400]
  
  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

Sshxgreen 🙏 help When I execute the command "npx wrangler d1 backup create my-database-na...

I

itsmatteomanf•9/9/23, 9:26 AM

Backups aren’t supported in the new experimental backend (yet, but no ETA). There is the Time Travel feature which accomplishes a similar goal, with some pros and cons.

I

itsmatteomanf•9/9/23, 9:27 AM

https://developers.cloudflare.com/d1/learning/time-travel/

Time Travel and backups · Cloudflare D1 docs

Time Travel is D1’s approach to backups and point-in-time-recovery, and allows you to restore a database to any minute within the last 30 days.

S

Southpaw•9/9/23, 7:27 PM

As a Ghost user...why would you want to do that?

Are you wanting to store your mailing lists in D1 or...?

S

Southpaw•9/9/23, 7:27 PM

Or are you wanting to use D1 as the Ghost database?

M

mrwind•9/10/23, 2:46 AM

https://community.cloudflare.com/t/cant-access-bound-d1-database-from-cloudflare-pages/552176/5 Hi is wrangler version 3.7.0 fix that bug?

Cloudflare Community

Can't access bound D1 database from Cloudflare Pages

Yes I have: and

Mmrwind https://community.cloudflare.com/t/cant-access-bound-d1-database-from-cloudflare...

C

Cyb3r-Jak3•9/10/23, 3:09 AM

I don’t believe so. The public issue I can find is https://github.com/cloudflare/workers-sdk/issues/3756

GitHub

🐛 BUG: cannot find bound D1 DB using wrangler 3.5.0 in pages app · ...

Which Cloudflare product(s) does this pertain to? D1, Pages What version(s) of the tool(s) are you using? 3.5.0 Wrangler What version of Node are you using? 18.14.2 on windows, 18.17.0 on github ac...

SSrikanth This. I believe I can run a full fledged blog with their current data storage ca...

S

Southpaw•9/10/23, 6:39 AM

Given that they only support MySQL and D1 is based off SQLite, I think you’re going to have a very difficult time.

I don’t see the point in decentralising Ghost, but if you really wanted to do that the best way to do it would be to use it as a headless CMS for a site you can host on Cloudflare Pages, not trying to run the entire site in a serverless environment.

M

Mage•9/10/23, 9:39 AM

I would like to use this in production, any ETA on beta?

D

digeratidave•9/11/23, 2:22 PM

I am trying to perform a quick POC/Prototype of D1 being called from a worker.
Can it be implemented from the cloudflare dashboard (https://dash.cloudflare.com/) or only locally on a desktop with Wrangler? is there any documentation for implementing in the dashboard, it seems it is all Wrangler docs.

hey guys I have almost per monthly KV write operations around 5 million, and it is costing me almost

Similar Threads