I'm just worried that someone might have something funky going on with their internet connection (eg
I'm just worried that someone might have something funky going on with their internet connection (eg. vpn stuff) and then I get strange heisenbugs
JI'm making assumptions about where their requests are going to come from
HEven then, unless you have multiple clients with the same session making changes, it should still generally be consistent
JIt's also a little frustrating because I don't really need all the edge stuff.. people using my site are all going to be from one country
DThe only time I would actively be concerned about missing the presence of data is if I access some data that's undefined, it's cached for at least a minute, create the data, and then I have to wait a minute for the changes to appear
JBut I guess there's no way to say, always store the data here
Dfor example someone tries to sign in, I check if their account exists in KV, return nope, they sign up, and they have to wait a minute for the account to "appear"
JI'm thinking the user does something after logging in, it tries to fetch their session to make sure they're authenticated, and it can't find it so logs them out. Is that not how it behaves?
JOr does it only returned cached data, not cached lack of data
JSo it would always return the correct data when a key is newly created
JJust there might be a delay logging someone out
JSo if I try to read a newly created key from a different colo to the one it was created in, it will return as not existing?
JI was optimistic that this is only an issue if there was a previous cached version of that key
JHmm
JSo this problem would only happen if I check if the user is already logged in from Colo A, log them in on Colo B and then check again on Colo A
JWhich shouldn't happen as the session key will be randomly generated, so won't have been checked before being written
Jpossibly more of an issue if I'm using the session to store information key to the functioning of the app that changes whenever the user performs an action
Jthen any changing of colo will cause weird stuff to happen
Jin that case probably better to use a durable object
Jit would only take them doing something after switching colo, and then the connection switching back for bad things to happen
Jstill a little unnerving to have that possibility though
LI have a CSV file with 3 million rows. I want to store these one time and then later read the results from this. Is KV well suited for this and how can I get my data in KV?
J@louisdck how are you expecting to query the data? just simple key lookups?
Jis the data static?
LYes, it is static data (chess puzzles). I want to query the data through a key lookup. I see there are only 1000 free writes a day. How can I write multiple lines in 1 write opertaion?
LHowever, now that I think of it, each chess puzzle is also put in a category. It would be nice if I could also query using these categories. For example, return me 5 random chess puzzles from the endgame category.
LThis is an example row of my csv:
000rO,3R4/8/K7/pB2b3/1p6/1P2k3/3p4/8 w - - 4 58,a6a5 e5c7 a5b4 c7d8,1039,80,85,61,crushing endgame fork short,https://lichess.org/tzeeBEc2#115,
000rO,3R4/8/K7/pB2b3/1p6/1P2k3/3p4/8 w - - 4 58,a6a5 e5c7 a5b4 c7d8,1039,80,85,61,crushing endgame fork short,https://lichess.org/tzeeBEc2#115,
LThe '000rO' is a unique id for the chess puzzle.
J@louisdck you can do some quite nice stuff using a sqlite database and static hosting https://github.com/phiresky/sql.js-httpvfs
JI'm currently hosting 4mil-ish rows with 3 or 4 indexes on R2, and the whole thing fits in cloudflare's cache
Jdefinitely easy to get uploaded as it's just a single file
LMostly ID, but as mentioned it would be nice only to return records within a specific category.
LOk, what else would you recommend?
Jsqlite is perfect for large read-only datasets
HDoes each record only have one category?
JI feel like 3mil rows containing chess puzzles is probably going to be more than a GB
Jit's like a game of guess the number of sweets in the jar 
LNo, see the example earlier, this has 4 categories: crushing endgame fork short
000rO,3R4/8/K7/pB2b3/1p6/1P2k3/3p4/8 w - - 4 58,a6a5 e5c7 a5b4 c7d8,1039,80,85,61,crushing endgame fork short,https://lichess.org/tzeeBEc2#115,
000rO,3R4/8/K7/pB2b3/1p6/1P2k3/3p4/8 w - - 4 58,a6a5 e5c7 a5b4 c7d8,1039,80,85,61,crushing endgame fork short,https://lichess.org/tzeeBEc2#115,
LI don't mind using an alpha product, but the CSV is 550MB.
LThanks for the suggestion, but I don't really see how I use that GitHub project together with R2 (I'm quite a newbie).
HYou might be able to do this with a series of extra keys? Like crushing:000r0 maps to 000r0, which maps to the row?
LFeels like reinventing a relational database 
So would the value each time be one ID or an array of all ID's in that category?
So would the value each time be one ID or an array of all ID's in that category?
LI can look into it, but I'm quite happy with some other CF products so was looking if they had something suiting my needs.
HThe Value would either be another Key, or a row of your CSV
LSo this second KV would have more than 12 million rows? 3 million games with on average 4 categories.
Jbasically, you need to write a schema with your indexes, and import your data into a sqlite database (something like this: https://gist.github.com/johtso/c4de17de513c36f91c948498c8fc7908)
LWhere can I find more about D1's restrictions?
J@louisdck and then you specify the location and shape of the database in
createDbWorker