Does D1 optimize duplicate queries that are in the same `.batch(...)`?
Does D1 optimize duplicate queries that are in the same
.batch(...)?.batch(...)wrangler d1 migrations which is essentially what I was trying to do so I no longer need that query.{"US":30,"FR":10,"DE":5} with each value being incremented by 1 once someone from that country clicks the url within that same 5 minute interval. This is the script I'm using in a worker D1_ERROR: no such column: country despite it being there
i_clicks while your query is inserting into interval_clicks?PRAGMA table_xinfo('interval_clicks') prior to your insert, and log the result?D1_ERROR: FOREIGN KEY constraint failed, fixed it and now the hardcoded values have been writtenwrangler pages dev --d1=X works with a local database that would seem to have a relationship to the "live" one but is really a completely independent DB (correct me if I'm misunderstanding this). This seems unusual to me as a default (and only?) option. I would think that many people would often want to develop against some kind of "live" db without always having to manage a local copy. Am I missing something, is this seen as a bad practice? What is the recommended workflow?wrangler pages devpages dev doesn't support remote anything atm--remote functionality that hasn't been added yet?PRAGMA defer_foreign_keys = on to disable temporarily, but you must ensure FK constraints are met. Draft docs: https://silverlock-d1-foreign-keys.cloudflare-docs-7ou.pages.dev/d1/learning/foreign-keys/
you cannot scale Facebook on D1If you manage to reach facebook-level success on almost any vendor, that vendor tends to work with you to find a way to scale
--local
CREATE TABLE IF NOT EXISTS interval_clicks (
id INTEGER PRIMARY KEY,
url_id INTEGER,
interval_start TIMESTAMP, -- Rounded to the nearest 5-minute mark
country TEXT,
device_type TEXT,
click_count INTEGER DEFAULT 0,
FOREIGN KEY(url_id) REFERENCES url(id)
);{"US":30,"FR":10,"DE":5} const ans = await env.DATABASE.prepare(`INSERT INTO interval_clicks (url_id, interval_start, country, device_type, click_count) VALUES (?1, ?2, json_insert(IFNULL(json_extract(country, '$'), '{}'), '$.?3', IFNULL(json_extract(country, '$.?3'), 0) + 1), json_insert(IFNULL(json_extract(device_type, '$'), '{}'), '$.?4', IFNULL(json_extract(device_type, '$.?4'), 0) + 1), 1) ON CONFLICT(url_id, interval_start) DO UPDATE SET country = json_insert(country, '$.?3', json_extract(country, '$.?3') + 1), device_type = json_insert(device_type, '$.?4', json_extract(device_type, '$.?4') + 1), click_count = click_count + 1; `)
.bind(url_id, interval_start, country, deviceType).run(); console.log(ans); const ans = await env.DATABASE.prepare(`INSERT INTO interval_clicks (url_id, interval_start, country, device_type, click_count) VALUES (?1, ?2, json_insert(IFNULL(json_extract(country, '$'), '{}'), '$.?3', IFNULL(json_extract(country, '$.?3'), 0) + 1), json_insert(IFNULL(json_extract(device_type, '$'), '{}'), '$.?4', IFNULL(json_extract(device_type, '$.?4'), 0) + 1), 1) ON CONFLICT(url_id, interval_start) DO UPDATE SET country = json_insert(country, '$.?3', json_extract(country, '$.?3') + 1), device_type = json_insert(device_type, '$.?4', json_extract(device_type, '$.?4') + 1), click_count = click_count + 1; `)
.bind(url_id, interval_start, country, deviceType).run(); console.log(ans);D1_ERROR: no such column: countryi_clicksinterval_clicksPRAGMA table_xinfo('interval_clicks') "logs": [
{
"message": [
{
"success": true,
"meta": {
"served_by": "v3-prod",
"duration": 0.1650210000007064,
"changes": 0,
"last_row_id": 0,
"changed_db": false,
"size_after": 65536,
"rows_read": 0,
"rows_written": 0
},
"results": [
{
"cid": 0,
"name": "id",
"type": "INTEGER",
"notnull": 0,
"dflt_value": null,
"pk": 1,
"hidden": 0
},
{
"cid": 1,
"name": "url_id",
"type": "INTEGER",
"notnull": 0,
"dflt_value": null,
"pk": 0,
"hidden": 0
},
{
"cid": 2,
"name": "interval_start",
"type": "TIMESTAMP",
"notnull": 0,
"dflt_value": null,
"pk": 0,
"hidden": 0
},
{
"cid": 3,
"name": "country",
"type": "TEXT",
"notnull": 0,
"dflt_value": null,
"pk": 0,
"hidden": 0
},
{
"cid": 4,
"name": "device_type",
"type": "TEXT",
"notnull": 0,
"dflt_value": null,
"pk": 0,
"hidden": 0
},
{
"cid": 5,
"name": "click_count",
"type": "INTEGER",
"notnull": 0,
"dflt_value": "0",
"pk": 0,
"hidden": 0
}
]
}
],
"level": "log",
"timestamp": 1694857686407
}
],D1_ERROR: FOREIGN KEY constraint failed {
"DE": 77,
"US": 52,
"CA": 11,
"AU": 13,
}wrangler pages dev --d1=Xwrangler pages devpages devPRAGMA defer_foreign_keys = on A prepared SQL statement must contain only one statement. [code: 7500]--local┌───────────────────┬──────────────────────────────────────┐
│ │ removed-removed-removed-removed- │
├───────────────────┼──────────────────────────────────────┤
│ name │ removed-worker-d1 │
├───────────────────┼──────────────────────────────────────┤
│ created_at │ 2023-09-09T03:49:34.773Z │
├───────────────────┼──────────────────────────────────────┤
│ version │ beta │
├───────────────────┼──────────────────────────────────────┤
│ num_tables │ 6 │
├───────────────────┼──────────────────────────────────────┤
│ running_in_region │ ENAM │
├───────────────────┼──────────────────────────────────────┤
│ database_size │ 2.59 MB │
├───────────────────┼──────────────────────────────────────┤
│ read_queries_24h │ 533 │
├───────────────────┼──────────────────────────────────────┤
│ write_queries_24h │ 755 │
└───────────────────┴──────────────────────────────────────┘--batch-size 1