Cloudflare Developers•2mo ago

Hi all, this product seems interesting and I'd like to try it out for a new project. My main issue i

Hi all, this product seems interesting and I'd like to try it out for a new project. My main issue is the 3 month limit - are there plans to increase this? My use case would be user behavior data influencing recommendations. Around 100 million writes per month, and 15 million reads per month. Is read latency viable for sub 0.5s responses? If it's really fast I'll put it as part of the SSR (Cloudflare Pages with svelte adapter, durable object with sql for data) If latency is an issue I'd alternatively still write to analytics, then read from analytics + recalculate recommendations on a schedule and push those into DO/SQL, then client facing page render would read from DO/SQL for recommendations. It's hard to keep up with all the new cloudflare features though, so perhaps there's something designed for this? We're an enterprise client, is it tricky to get this product added to our plan?

6 Replies

LabbedRat•4w ago

Hello! Do we know if there's any plan to support something like last_value or first_value in Analytics Engine. Or if there's a query that can return something equivalent? I've considered ORDER BY, however when providing a value for a field is optional, this would fail. Additionally, ORDER BY would not allow getting both a first and last value in the same query. I'm not working for Cloudflare so I don't really have many answers, but if it's any help, there was a poll back in february about this. I'm not aware of any follow up though. https://discord.com/channels/595317990191398933/981314061268578304/1338903066837254154 According to the documentation, count() should be replaced by sum(_sample_interval) to account for sampling. What about count(DISTINCT <column_name>)? Will the result provided be accurate?

John Spurlock•4w ago

short answer: no, count(distinct <column>) will not be accurate long answer: you might be tempted to add more indexes to help with this, but that is also usually not going to work: https://developers.cloudflare.com/analytics/faq/wae-faqs/#could-i-just-use-many-unique-index-values-to-get-better-unique-counts

LabbedRat•3w ago

Indeed, I had already read the article about the side effect of having too many effects, but simply wanted to make sure as count(DISTINCT <column>) 's accuracy was not documented. Thank you very much for the clarification!

eckoln•3w ago

Hello, I need to ingest metrics in a Durable Object WebSocket server, like total daily/monthly connections, messages, etc. I also need to see these metrics on my dashboard every 5 seconds or realtime. What are the best practices for this? I'm using Analytics Engine right now, but is it a good idea to query this data every time?

Sergi A.•2w ago

Two things left that I think are really important to add to Analytics Engine: - Longer data retention > 3 months: minimum 24 months, ideally data stored forever and charged based on GB stored, like D1 - Select DB location: it is important due to Europe's GDPR to have our data hosted in Europe. D1 also solves this with location hints

Philipp•19h ago

Hey! My logs show occasional errors (around 3-5% of all requests) when querying Workers Analytics Engine (from my Worker). Most of the time I get 200s but I randomly experience 500s and 502s. The requests are not changing. Example request:

COUNT() as count,
blob1 as eventName
FROM ${context.cloudflare.env.ANALYTICS_DATASET}
WHERE
timestamp >= toDateTime(...)
AND timestamp < toDateTime(...)
AND blob1 IN ('measurement_start', 'measurement_complete', 'button_display')
AND blob2 = '...'
AND blob3 = '...'
GROUP BY eventName

COUNT() as count,
blob1 as eventName
FROM ${context.cloudflare.env.ANALYTICS_DATASET}
WHERE
timestamp >= toDateTime(...)
AND timestamp < toDateTime(...)
AND blob1 IN ('measurement_start', 'measurement_complete', 'button_display')
AND blob2 = '...'
AND blob3 = '...'
GROUP BY eventName

https://api.cloudflare.com/client/v4/accounts/${context.cloudflare.env.CLOUDFLARE_ACCOUNT_ID}/analytics_engine/sql Is anyone else experiencing this flakiness of the API?

Gaming

Programming

Hi all, this product seems interesting and I'd like to try it out for a new project. My main issue i

Did you find this page helpful?