You use them the same way you use a single dataset. Just keep each one to its own binding

A

Aries•8/22/23, 2:04 AM

https://files.aries.fyi/2023/08/21/Code_83154a37af784293.png

VVictor You use them the same way you use a single dataset. Just keep each one to its ow...

A

Aries•8/22/23, 2:04 AM

when accessing dataImpressions, it is undefined.

V

VictorOP•8/22/23, 2:04 AM

are you running it locally in wrangler?

A

Aries•8/22/23, 2:05 AM

Yes, I deployed it and was getting undefined, so I reverted the deployment.

AAries Yes, I deployed it and was getting undefined, so I reverted the deployment.

V

VictorOP•8/22/23, 2:08 AM

analytics_engine_datasets = [
    { binding = "TRACKING_CLOUDFLARE_REQUEST", dataset = "TRACKING_CLOUDFLARE_REQUEST" },
    { binding = "TRACKING_CLOUDFLARE_KV", dataset = "TRACKING_CLOUDFLARE_KV" },
    { binding = "TRACKING_CLOUDFLARE_D1", dataset = "TRACKING_CLOUDFLARE_D1" },
    { binding = "TRACKING_OPENAI_LM", dataset = "TRACKING_OPENAI_LM_DEV" },
    { binding = "TRACKING_USER_AI_GENERAL", dataset = "TRACKING_USER_AI_GENERAL_DEV" },
]

analytics_engine_datasets = [
    { binding = "TRACKING_CLOUDFLARE_REQUEST", dataset = "TRACKING_CLOUDFLARE_REQUEST" },
    { binding = "TRACKING_CLOUDFLARE_KV", dataset = "TRACKING_CLOUDFLARE_KV" },
    { binding = "TRACKING_CLOUDFLARE_D1", dataset = "TRACKING_CLOUDFLARE_D1" },
    { binding = "TRACKING_OPENAI_LM", dataset = "TRACKING_OPENAI_LM_DEV" },
    { binding = "TRACKING_USER_AI_GENERAL", dataset = "TRACKING_USER_AI_GENERAL_DEV" },
]

analytics_engine_datasets = [
    { binding = "TRACKING_CLOUDFLARE_REQUEST", dataset = "TRACKING_CLOUDFLARE_REQUEST" },
    { binding = "TRACKING_CLOUDFLARE_KV", dataset = "TRACKING_CLOUDFLARE_KV" },
    { binding = "TRACKING_CLOUDFLARE_D1", dataset = "TRACKING_CLOUDFLARE_D1" },
    { binding = "TRACKING_OPENAI_LM", dataset = "TRACKING_OPENAI_LM_DEV" },
    { binding = "TRACKING_USER_AI_GENERAL", dataset = "TRACKING_USER_AI_GENERAL_DEV" },
]

analytics_engine_datasets = [
    { binding = "TRACKING_CLOUDFLARE_REQUEST", dataset = "TRACKING_CLOUDFLARE_REQUEST" },
    { binding = "TRACKING_CLOUDFLARE_KV", dataset = "TRACKING_CLOUDFLARE_KV" },
    { binding = "TRACKING_CLOUDFLARE_D1", dataset = "TRACKING_CLOUDFLARE_D1" },
    { binding = "TRACKING_OPENAI_LM", dataset = "TRACKING_OPENAI_LM_DEV" },
    { binding = "TRACKING_USER_AI_GENERAL", dataset = "TRACKING_USER_AI_GENERAL_DEV" },
]

This is what I have live right now and it works

export type Bindings = Record<string, any> & {
    ...
    TRACKING_CLOUDFLARE_REQUEST?: AnalyticsEngineDataset;
    TRACKING_CLOUDFLARE_KV?: AnalyticsEngineDataset;
    TRACKING_CLOUDFLARE_D1?: AnalyticsEngineDataset;
    TRACKING_OPENAI_LM?: AnalyticsEngineDataset;
    TRACKING_USER_AI_GENERAL?: AnalyticsEngineDataset;
    ...
};

export type Bindings = Record<string, any> & {
    ...
    TRACKING_CLOUDFLARE_REQUEST?: AnalyticsEngineDataset;
    TRACKING_CLOUDFLARE_KV?: AnalyticsEngineDataset;
    TRACKING_CLOUDFLARE_D1?: AnalyticsEngineDataset;
    TRACKING_OPENAI_LM?: AnalyticsEngineDataset;
    TRACKING_USER_AI_GENERAL?: AnalyticsEngineDataset;
    ...
};

export type Bindings = Record<string, any> & {
    ...
    TRACKING_CLOUDFLARE_REQUEST?: AnalyticsEngineDataset;
    TRACKING_CLOUDFLARE_KV?: AnalyticsEngineDataset;
    TRACKING_CLOUDFLARE_D1?: AnalyticsEngineDataset;
    TRACKING_OPENAI_LM?: AnalyticsEngineDataset;
    TRACKING_USER_AI_GENERAL?: AnalyticsEngineDataset;
    ...
};

export type Bindings = Record<string, any> & {
    ...
    TRACKING_CLOUDFLARE_REQUEST?: AnalyticsEngineDataset;
    TRACKING_CLOUDFLARE_KV?: AnalyticsEngineDataset;
    TRACKING_CLOUDFLARE_D1?: AnalyticsEngineDataset;
    TRACKING_OPENAI_LM?: AnalyticsEngineDataset;
    TRACKING_USER_AI_GENERAL?: AnalyticsEngineDataset;
    ...
};

And calling it

env.TRACKING_OPENAI_LM?.writeDataPoint({
    indexes: [...],
    blobs: [...],
    doubles: [...],
});

env.TRACKING_OPENAI_LM?.writeDataPoint({
    indexes: [...],
    blobs: [...],
    doubles: [...],
});

env.TRACKING_OPENAI_LM?.writeDataPoint({
    indexes: [...],
    blobs: [...],
    doubles: [...],
});

env.TRACKING_OPENAI_LM?.writeDataPoint({
    indexes: [...],
    blobs: [...],
    doubles: [...],
});

A

Aries•8/22/23, 2:09 AM

Optional access operator means it would fail silenty though?

A

Aries•8/22/23, 2:10 AM

https://files.aries.fyi/2023/08/21/Code_ec26a151f9c753c8.png

V

VictorOP•8/22/23, 2:11 AM

Yea, but that only happens if running locally in wrangler for me

A

Aries•8/22/23, 2:11 AM

analytics is exposed but not

dataImpressions

dataImpressions

dataImpressions

dataImpressions

A

Aries•8/22/23, 2:12 AM

Very weird.

AAries analytics is exposed but not ``dataImpressions``

V

VictorOP•8/22/23, 2:14 AM

Going to

https://dash.cloudflare.com/<account id>/workers/analytics-engine

https://dash.cloudflare.com/<account id>/workers/analytics-engine

https://dash.cloudflare.com/<account id>/workers/analytics-engine

https://dash.cloudflare.com/<account id>/workers/analytics-engine, do you see stats?

A

Aries•8/22/23, 2:14 AM

Only the old dataset.
https://files.aries.fyi/2023/08/21/firefox_1e1881fe7a266c61.png

A

Aries•8/22/23, 2:16 AM

When uploading, it does not acknowledge it at all.
https://files.aries.fyi/2023/08/21/Code_9957cb0c3fb79cea.png

VVictor Going to `https://dash.cloudflare.com/<account id>/workers/analytics-engine`, do...

A

Aries•8/22/23, 2:17 AM

Weird..

A

Aries•8/22/23, 2:17 AM

And when deployed:
https://files.aries.fyi/2023/08/21/firefox_e676d010ecf6d878.png

VVictor ```toml analytics_engine_datasets = [ { binding = "TRACKING_CLOUDFLARE_REQUE...

A

Aries•8/22/23, 2:34 AM

Copying your syntax exactly doesn't register them, but doing it this way does: https://files.aries.fyi/2023/08/21/Code_a16a743b078b69f7.png

A

Aries•8/22/23, 2:35 AM

dataset not dataset_name as well, my bad.

AAries Copying your syntax exactly doesn't register them, but doing it this way does: h...

V

VictorOP•8/22/23, 2:36 AM

My syntax is setup for multiple environments, that probably why it's not picking up

A

Aries•8/22/23, 2:49 AM

Is

25 writes (writeDataPoint invocations) per client HTTP request

25 writes (writeDataPoint invocations) per client HTTP request

25 writes (writeDataPoint invocations) per client HTTP request

25 writes (writeDataPoint invocations) per client HTTP request a hard limit?

C

chientrm.com•8/22/23, 3:49 AM

is analytics engine available on free plan?

AAries Is ``25 writes (writeDataPoint invocations) per client HTTP request`` a hard lim...

V

VictorOP•8/22/23, 5:16 AM

I don't know of any limits

E

Elliot Hesp•8/22/23, 3:20 PM

If I'm wrapping analytics engine in an ORM, what flavour of SQL is it using?

EElliot Hesp If I'm wrapping analytics engine in an ORM, what flavour of SQL is it using?

J

John Spurlock•8/22/23, 3:21 PM

it's a subset of clickhouse: https://clickhouse.com/docs/en/sql-reference

A

Aries•8/24/23, 6:24 AM

In the SQL, how would I say give me the top 100, then group the rest as "Other"?

A

Aries•8/24/23, 6:25 AM

https://files.aries.fyi/2023/08/24/firefox_469e36f8aac75ae2.png

AAries In the SQL, how would I say give me the top 100, then group the rest as "Other"?

D

DaniFoldi•8/24/23, 7:04 AM

In SQL, you can use a statement like below

SELECT id, ...,
CASE
  WHEN id <= 100 THEN id
  ELSE 'Other'
END AS groupedId
FROM myTable

SELECT id, ...,
CASE
  WHEN id <= 100 THEN id
  ELSE 'Other'
END AS groupedId
FROM myTable

SELECT id, ...,
CASE
  WHEN id <= 100 THEN id
  ELSE 'Other'
END AS groupedId
FROM myTable

SELECT id, ...,
CASE
  WHEN id <= 100 THEN id
  ELSE 'Other'
END AS groupedId
FROM myTable

However, I'm not sure if clickhouse/AE supports that as I haven't played around with it too much yet, so if it doesn't work you're likely best selecting the top 100, and summing the rest "manually" in subqueries

I

itsmatteomanf•8/31/23, 12:46 PM

Is there a way to get more than the default 25 ingestions per invocation? I have a queue that processes 100 items which need to push things a few things to AE per item.

I

itsmatteomanf•8/31/23, 1:58 PM

Oh, I read now... maybe I should ping @46bit

J

John Spurlock•8/31/23, 9:28 PM

having used AE from basically day 1, you definitely have to deal with pretty much every

5xx

5xx

5xx

5xx error in the book coming back, and retry-after-delay. Most of these errors are either one-offs or transient

P

pigri•9/1/23, 7:22 AM

Hmm... anybody else who also has experience with analytics_engine is not working? My grafana dashboards are down and I can see this error:

Sorry, we were unable to evaluate your query

Sorry, we were unable to evaluate your query

Sorry, we were unable to evaluate your query

Sorry, we were unable to evaluate your query

Jjimh Hi @.pigri - can you post an example of a query that returns this error?

P

pigri•9/1/23, 8:29 AM

Sure,
SELECT
timestamp AS t,
*
FROM
usageDataOutput
WHERE
t >= NOW() - INTERVAL '1' DAY
ORDER BY t

P

pigri•9/1/23, 8:29 AM

2 days ago this query is worked

P

pigri•9/1/23, 9:26 AM

Thank you! We try to figure out who can fix it.

Jjimh I searched through errors on our end based on a hunch and I'm pretty sure the is...

K

Kasumi•9/1/23, 9:53 AM

Sorry for disturbing but does it mean that cf cuts the responses off as soon as the response hits 100mb or is it just analytics-engine ?

Jjimh The quickest way to reduce response size for this query will be to remove the `*...

P

pigri•9/1/23, 10:52 AM

Thank you we fixed it!

U

Unsmart•9/1/23, 3:58 PM

Any new plugin we should be using since I guess the altinity plugin thats recommended is using legacy stuff and will eventually break unless they are working on updating it maybe

Jjimh It looks like they're aware: https://github.com/Altinity/clickhouse-grafana/issu...

I

itsmatteomanf•9/1/23, 5:22 PM

It’s been open since March 22… doesn’t feel like it’s really moving along

A

Aries•9/4/23, 8:02 AM

Does this support prepared statements?

You use them the same way you use a single dataset. Just keep each one to its own binding

Similar Threads