Cloudflare Developers•3y ago

What are you writing in your Worker? Looks like you are trying to store an object that can’t be sere

What are you writing in your Worker? Looks like you are trying to store an object that can’t be serealized

HHello, I’m Allie!What are you writing in your Worker? Looks like you are trying to store an objec...

sdan•6/23/23, 9:17 PM

[response.status, response.headers]

Ssdan [response.status, response.headers]

Hello, I’m Allie!OP•6/23/23, 9:18 PM

Try

[response.status, Object.fromEntries(response.headers.entries())]

[response.status, Object.fromEntries(response.headers.entries())]

LLiudmila Hey everyone, silly question: I'm trying to understand what is Cloudflare Analyt...

Hello, I’m Allie!OP•6/26/23, 11:47 AM

1) Yes, analytics events
2) https://clickhouse.com/
3) No
4) No
5) Basically like

console.log

console.log

a place to quickly and easily dump data to, but querying said data generally shouldn't be in the hot path of your app.

ClickHouse

Fast Open-Source OLAP DBMS - ClickHouse

ClickHouse is a fast open-source column-oriented database management system that allows generating analytical data reports in real-time using SQL queries

DDeleted User Is JDBC/ODBC on the roadmap?

John Spurlock•6/26/23, 4:52 PM

Have you tried https://clickhouse.com/docs/en/integrations/java ? the ClickHouse driver appears to support custom headers

Java Language Client Options for ClickHouse | ClickHouse Docs

Options for connecting to ClickHouse from Java

DDeleted User Does this work with Cloudflare AE?

John Spurlock•6/26/23, 9:23 PM

doesn't look like it supports custom endpoint urls - probably not usable without some kind of local proxy

BGwin•6/26/23, 9:59 PM

Beta 16 OS

DDeleted User I'll give it a shot later

John Spurlock•6/26/23, 11:05 PM

I poked around with it to see if I could get it to work given AE's constraints. Turns out there is a secret option called

web_context

web_context

to set a custom endpoint path, so it kind of works. Basic sample here: https://tsplay.dev/m3Eyqw

TS Playground - An online editor for exploring TypeScript and JavaS...

The Playground lets you write TypeScript or JavaScript online in a safe and sharable way.

John Spurlock•6/26/23, 11:06 PM

The results are always returned in text lines though, not sure why - you get one string column called "results" for the text line, so best to use line-based formats like TSV or JSONEachRow

Advany•6/30/23, 5:44 PM

Hi there, I am doing the following:

SELECT 
    formatDateTime(timestamp, '%Y-%m-%d') as day,
    sum(_sample_interval) AS hits,
    _sample_interval
FROM USAGE
WHERE
    timestamp >= now() - INTERVAL '30' DAY
GROUP BY day, _sample_interval
ORDER BY day

SELECT 
    formatDateTime(timestamp, '%Y-%m-%d') as day,
    sum(_sample_interval) AS hits,
    _sample_interval
FROM USAGE
WHERE
    timestamp >= now() - INTERVAL '30' DAY
GROUP BY day, _sample_interval
ORDER BY day

The sample interval is always 10. Only doing 85 million events in the last 20 days. Highest was around 5 million. Would that cause sampling for one index?

Mmiki / 46bit 85 million rows is the kind of scale where we start to expect sampling

Advany•6/30/23, 7:45 PM

still 10. Ill try to add the data using the customer index. See if that helps not sample for smaller customers

Advany•6/30/23, 10:01 PM

I want to ingest events in a browser to analytics engine. Is there a better way than calling a worker each time a event happens for ingest? Using bound workers for it.

AAdvany I want to ingest events in a browser to analytics engine. Is there a better way ...

John Spurlock•6/30/23, 10:04 PM

depending on how long the browser session is, you many want to open a websocket and reuse it

JJohn Spurlock depending on how long the browser session is, you many want to open a websocket ...

Advany•6/30/23, 10:35 PM

That would based on unbound and wall clock time right?

Would that be cheaper then using just a unbound worker for each event?

AAdvany That would based on unbound and wall clock time right? Would that be cheaper th...

John Spurlock•6/30/23, 10:42 PM

dunno, that just for efficiently sending them asap, if you are concerned about costs, batching em up would probably be the best option

Advany•7/5/23, 1:00 PM

I can't seem to access the analytics engine inside hono. Does anyone know how to do that?

I tried c.env.DOMAIN_ANALYTICS.writeDataPoint(dataPoint);

I have this in wrangler

[[analytics_engine_datasets]]
binding = "DOMAIN_ANALYTICS"
dataset = "DOMAIN_ANALYTICS"

Advany•7/20/23, 11:46 AM

Loving analytics engine

When can we expect pricing? Really want to scale it up more to billions...

AAdvany Loving analytics engine 🙂 When can we expect pricing? Really want to scale it u...

Matt Silverlock•7/20/23, 3:06 PM

We intend to announce public pricing by October. We want to give folks confidence to build on Analytics Engine, and recognize that pricing — or sticker shock — is a big point of friction.

Advany•7/20/23, 3:38 PM

2-3 months. Will there be a private annoucement before october lol

Unsmart•7/20/23, 3:39 PM

I mean cloudflare needs to know it first thats the private announcement

asp•7/21/23, 9:56 AM

Are there any performance implications of storing and filtering by a custom timestamp stored in a doubledouble column compared to the

timestamp

timestamp

column? I'm writing ingesting Logpush batches into Analytics Engine, but they are delayed by up to a couple of minutes, so I need to use the timestamp specified in each log entry.

asp•7/21/23, 11:13 AM

Alternatively, if you need to be very accurate, maybe you can filter using both columns? You could increase the range on the timestamp one slightly, if needed.

That sounds like the best solution for now. I know for sure the real timestamp won't be larger than the datapoint's, and, unless there's an issue with Logpush, the datapoint will be written within a couple of minutes of the event.

Jjimh It will be quite inefficient to filter on that double colum, whereas the indexin...

asp•7/21/23, 11:14 AM

Are there any plans to allow specifying a custom timestamp?

SELECT formatDateTime(timestamp, '%Y-%m-%d') as day, sum(_sample_interval) AS hits, _sample_interval FROM USAGE WHERE timestamp >= now() - INTERVAL '30' DAY GROUP BY day, _sample_interval ORDER BY day

What are you writing in your Worker? Looks like you are trying to store an object that can’t be sere

Similar Threads