Is local development support *planned* for AE?

JJustinNoel Is local development support *planned* for AE?

D

DaniFoldi•6/21/23, 9:31 PM

definitely planned, there was even a PR for miniflare v2 - I haven't seen anything for v3 but I'd assume eventually they want all bindings to function locally

K

kian•6/21/23, 9:56 PM

there's an open PR for workerd

K

kian•6/21/23, 9:57 PM

so

W

Walshy•6/21/23, 10:31 PM

workerd part is only half the work though

W

Walshy•6/21/23, 10:32 PM

it can invoke a HTTP service but there still needs to be the storage part done

K

kian•6/21/23, 10:32 PM

always applies

J

JustinNoelOP•6/22/23, 9:23 PM

There's a problem with Pages -> <project_name> -> Settings -> Functions -> Analytics Engine bindings

You can't delete any existing bindings. Console shows this error.

U

Unsmart•6/22/23, 9:24 PM

pretty sure sparrow is just analytics I would look at the actual request

A

Advany•6/22/23, 9:40 PM

Will analytics engine be available in snippets?

A

Advany•6/22/23, 9:41 PM

If not, is there a other way to ingest data from clients (browsers)? That doenst involve calling a worker for each ingestion?

S

sdan•6/23/23, 9:09 PM

i parsed in json data and am getting a response via SQL as such:

"response_data": "200,[object Object]"

"response_data": "200,[object Object]"

"response_data": "200,[object Object]"

"response_data": "200,[object Object]"

with the sql query:

SELECT
    blob3 AS response_data

SELECT
    blob3 AS response_data

SELECT
    blob3 AS response_data

SELECT
    blob3 AS response_data

is there anyway to parse json in sql?

Ssdan i parsed in json data and am getting a response via SQL as such: ``` "response_d...

H

Hello, I’m Allie!•6/23/23, 9:14 PM

What are you writing in your Worker? Looks like you are trying to store an object that can’t be serealized

HHello, I’m Allie!What are you writing in your Worker? Looks like you are trying to store an objec...

S

sdan•6/23/23, 9:17 PM

[response.status, response.headers]

Ssdan [response.status, response.headers]

H

Hello, I’m Allie!•6/23/23, 9:18 PM

Try

[response.status, Object.fromEntries(response.headers.entries())]

[response.status, Object.fromEntries(response.headers.entries())]

[response.status, Object.fromEntries(response.headers.entries())]

[response.status, Object.fromEntries(response.headers.entries())]

LLiudmila Hey everyone, silly question: I'm trying to understand what is Cloudflare Analyt...

H

Hello, I’m Allie!•6/26/23, 11:47 AM

1) Yes, analytics events
2) https://clickhouse.com/
3) No
4) No
5) Basically like

console.log

console.log

console.log

console.log a place to quickly and easily dump data to, but querying said data generally shouldn't be in the hot path of your app.

ClickHouse

Fast Open-Source OLAP DBMS - ClickHouse

ClickHouse is a fast open-source column-oriented database management system that allows generating analytical data reports in real-time using SQL queries

DDeleted User Is JDBC/ODBC on the roadmap?

J

John Spurlock•6/26/23, 4:52 PM

Have you tried https://clickhouse.com/docs/en/integrations/java ? the ClickHouse driver appears to support custom headers

Java Language Client Options for ClickHouse | ClickHouse Docs

Options for connecting to ClickHouse from Java

DDeleted User Does this work with Cloudflare AE?

J

John Spurlock•6/26/23, 9:23 PM

doesn't look like it supports custom endpoint urls - probably not usable without some kind of local proxy

B

BGwin•6/26/23, 9:59 PM

Beta 16 OS

DDeleted User I'll give it a shot later

J

John Spurlock•6/26/23, 11:05 PM

I poked around with it to see if I could get it to work given AE's constraints. Turns out there is a secret option called

web_context

web_context

web_context

web_context to set a custom endpoint path, so it kind of works. Basic sample here: https://tsplay.dev/m3Eyqw

TS Playground - An online editor for exploring TypeScript and JavaS...

The Playground lets you write TypeScript or JavaScript online in a safe and sharable way.

J

John Spurlock•6/26/23, 11:06 PM

The results are always returned in text lines though, not sure why - you get one string column called "results" for the text line, so best to use line-based formats like TSV or JSONEachRow

A

Advany•6/30/23, 5:44 PM

Hi there, I am doing the following:

SELECT 
    formatDateTime(timestamp, '%Y-%m-%d') as day,
    sum(_sample_interval) AS hits,
    _sample_interval
FROM USAGE
WHERE
    timestamp >= now() - INTERVAL '30' DAY
GROUP BY day, _sample_interval
ORDER BY day

SELECT 
    formatDateTime(timestamp, '%Y-%m-%d') as day,
    sum(_sample_interval) AS hits,
    _sample_interval
FROM USAGE
WHERE
    timestamp >= now() - INTERVAL '30' DAY
GROUP BY day, _sample_interval
ORDER BY day

SELECT 
    formatDateTime(timestamp, '%Y-%m-%d') as day,
    sum(_sample_interval) AS hits,
    _sample_interval
FROM USAGE
WHERE
    timestamp >= now() - INTERVAL '30' DAY
GROUP BY day, _sample_interval
ORDER BY day

SELECT 
    formatDateTime(timestamp, '%Y-%m-%d') as day,
    sum(_sample_interval) AS hits,
    _sample_interval
FROM USAGE
WHERE
    timestamp >= now() - INTERVAL '30' DAY
GROUP BY day, _sample_interval
ORDER BY day

The sample interval is always 10. Only doing 85 million events in the last 20 days. Highest was around 5 million. Would that cause sampling for one index?

Mmiki / 46bit 85 million rows is the kind of scale where we start to expect sampling

A

Advany•6/30/23, 7:45 PM

still 10. Ill try to add the data using the customer index. See if that helps not sample for smaller customers

A

Advany•6/30/23, 10:01 PM

I want to ingest events in a browser to analytics engine. Is there a better way than calling a worker each time a event happens for ingest? Using bound workers for it.

AAdvany I want to ingest events in a browser to analytics engine. Is there a better way ...

J

John Spurlock•6/30/23, 10:04 PM

depending on how long the browser session is, you many want to open a websocket and reuse it

JJohn Spurlock depending on how long the browser session is, you many want to open a websocket ...

A

Advany•6/30/23, 10:35 PM

That would based on unbound and wall clock time right?

Would that be cheaper then using just a unbound worker for each event?

AAdvany That would based on unbound and wall clock time right? Would that be cheaper th...

J

John Spurlock•6/30/23, 10:42 PM

dunno, that just for efficiently sending them asap, if you are concerned about costs, batching em up would probably be the best option

A

Advany•7/5/23, 1:00 PM

I can't seem to access the analytics engine inside hono. Does anyone know how to do that?

I tried c.env.DOMAIN_ANALYTICS.writeDataPoint(dataPoint);

I have this in wrangler

[[analytics_engine_datasets]]
binding = "DOMAIN_ANALYTICS"
dataset = "DOMAIN_ANALYTICS"

Is local development support planned for AE?

Similar Threads

Similar Threads

Similar Threads