Cloudflare Developers•2y ago

Got it thanks. Do you have it in the backlog or something? Its not critical but nice feature to have

OOleksii Kilevoi Got it thanks. Do you have it in the backlog or something? Its not critical but ...

Hello, I’m Allie!•7/31/24, 11:49 AM

I’m not sure, tbh(I’m not on the AE team). I can try to check, but not sure if/when I can get a response back

Jjimh Quite a few people ask for the ability to insert old data, and we've noted that....

Oleksii KilevoiOP•8/1/24, 9:29 AM

Roger that

thank you for clarifications!

Original message was deleted

Hard@Work•8/7/24, 3:52 PM

It shouldn't require a waitlist?

Hard@Work•8/7/24, 4:22 PM

Have you clicked the setup button on the dashboard?

Unsmart•8/9/24, 12:31 AM

It is a bit hidden yeah but also open beta = no waitlist. Wouldnt be very open if it was waitlisted

Walshy•8/11/24, 3:25 PM

yes

Original message was deleted

Walshy•8/11/24, 6:37 PM

Write sampling is per index

Walshy•8/11/24, 6:39 PM

I'd recommend reading this though: https://developers.cloudflare.com/analytics/faq/wae-faqs/#could-i-just-use-many-unique-index-values-to-get-better-unique-counts

Walshy•8/11/24, 6:40 PM

Lots of info in the docs, highly recommend the read

iska (RETIRED)•8/15/24, 8:57 AM

how can you build event system or audit system production grade with 3month retention ... makes little sense

Rares•9/1/24, 11:51 AM

Hey, how long is the data stored?

Pavlos•9/3/24, 9:51 AM

Hi Cloudflare Devs

I'm a noob dev here that trying to query the Zero Trust Gateway on how many queries we blocked or redirected.
I have setup my API keys and I'm using the GraphiQL tool.

How can I figure out ehat GraphQL query I need to perform? Thank you!

Rob M.•9/8/24, 6:52 PM

Hi guys

Does WHERE clause support LIKE operator in AE?

Jjimh There is no `LIKE` but if you want a "string contains" type query then you can u...

Rob M.•9/9/24, 2:01 PM

Thanks! Is it a good idea to create a complex index like:

"platform_id+project_id+user_id+session_id+event_subtype"

"platform_id+project_id+user_id+session_id+event_subtype"

and then use your approach to, let's say, sum(_sample_interval) all events for the specific project_id?

Tyler Maxey•9/9/24, 3:23 PM

Noob here trying to get analytics-engine to show counts on how often each event type happens, but I keep getting errors. Can someone please help with having a website built in react track events so that analytics engine can display counts?

I have my

wrangler.toml

wrangler.toml

[env.production]
analytics_engine_datasets = [{ binding = "ANALYTICS", dataset = "event_logs" }]

[env.production]
analytics_engine_datasets = [{ binding = "ANALYTICS", dataset = "event_logs" }]

Here's my

index.js

index.js

export default {
    async fetch(request, env) {
      if (request.method === 'POST') {
        // Parse the incoming request body
        const requestBody = await request.json();
        const eventType = requestBody.eventType || 'unknown_event';
        const url = requestBody.url || 'unknown_url';
  
        // Log the event using the Workers Analytics Engine
        await env.ANALYTICS.writeDataPoint({
          blobs: [eventType, url],
          indexes: [eventType],
          tags: [url],
        });
  
        return new Response("Event logged successfully!", { status: 200 });
      } else {
        return new Response("Method Not Allowed", { status: 405 });
      }
    }
  };

export default {
    async fetch(request, env) {
      if (request.method === 'POST') {
        // Parse the incoming request body
        const requestBody = await request.json();
        const eventType = requestBody.eventType || 'unknown_event';
        const url = requestBody.url || 'unknown_url';
  
        // Log the event using the Workers Analytics Engine
        await env.ANALYTICS.writeDataPoint({
          blobs: [eventType, url],
          indexes: [eventType],
          tags: [url],
        });
  
        return new Response("Event logged successfully!", { status: 200 });
      } else {
        return new Response("Method Not Allowed", { status: 405 });
      }
    }
  };

Tyler Maxey•9/9/24, 3:23 PM

And then in my react app, I use this function where I call the event (the input is a string that's the event's name).

export const logEvent = async (eventType) => {
    try {
      // Replace with your actual worker URL
      const workerUrl = "https://event-tracking-worker.<account>.workers.dev/";
  
      // Send event data to the Cloudflare Worker
      const response = await fetch(workerUrl, {
        method: "POST",  // Use POST or GET as per your preference
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          eventType: eventType,
          url: window.location.href,  // Include the current page URL
        }),
      });
  
      if (!response.ok) {
        console.error("Failed to log event", response.statusText);
      }
    } catch (error) {
      console.error("Error logging event:", error);
    }
  };

export const logEvent = async (eventType) => {
    try {
      // Replace with your actual worker URL
      const workerUrl = "https://event-tracking-worker.<account>.workers.dev/";
  
      // Send event data to the Cloudflare Worker
      const response = await fetch(workerUrl, {
        method: "POST",  // Use POST or GET as per your preference
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          eventType: eventType,
          url: window.location.href,  // Include the current page URL
        }),
      });
  
      if (!response.ok) {
        console.error("Failed to log event", response.statusText);
      }
    } catch (error) {
      console.error("Error logging event:", error);
    }
  };

I've tried and tried, but I just keep getting errors for fetching the API and loading resource (note that I changed my account to

<account>

<account>

[Error] Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500
[Error] Fetch API cannot load https://event-tracking-worker.<account>.workers.dev/ due to access control checks.
[Error] Failed to load resource: Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500 (event-tracking-worker.<account>.workers.dev, line 0)

[Error] Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500
[Error] Fetch API cannot load https://event-tracking-worker.<account>.workers.dev/ due to access control checks.
[Error] Failed to load resource: Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500 (event-tracking-worker.<account>.workers.dev, line 0)

Tyler Maxey•9/9/24, 3:24 PM

And then in my react app, I use this function where I call the event (the input is a string that's the event's name).

export const logEvent = async (eventType) => {
    try {
      // Replace with your actual worker URL
      const workerUrl = "https://event-tracking-worker.<account>.workers.dev/";
  
      // Send event data to the Cloudflare Worker
      const response = await fetch(workerUrl, {
        method: "POST",  // Use POST or GET as per your preference
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          eventType: eventType,
          url: window.location.href,  // Include the current page URL
        }),
      });
  
      if (!response.ok) {
        console.error("Failed to log event", response.statusText);
      }
    } catch (error) {
      console.error("Error logging event:", error);
    }
  };

export const logEvent = async (eventType) => {
    try {
      // Replace with your actual worker URL
      const workerUrl = "https://event-tracking-worker.<account>.workers.dev/";
  
      // Send event data to the Cloudflare Worker
      const response = await fetch(workerUrl, {
        method: "POST",  // Use POST or GET as per your preference
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          eventType: eventType,
          url: window.location.href,  // Include the current page URL
        }),
      });
  
      if (!response.ok) {
        console.error("Failed to log event", response.statusText);
      }
    } catch (error) {
      console.error("Error logging event:", error);
    }
  };

I've tried and tried, but I just keep getting errors for fetching the API and loading resource (note that I changed my account to

<account>

<account>

[Error] Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500
[Error] Fetch API cannot load https://event-tracking-worker.<account>.workers.dev/ due to access control checks.
[Error] Failed to load resource: Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500 (event-tracking-worker.<account>.workers.dev, line 0)

[Error] Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500
[Error] Fetch API cannot load https://event-tracking-worker.<account>.workers.dev/ due to access control checks.
[Error] Failed to load resource: Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500 (event-tracking-worker.<account>.workers.dev, line 0)

Tyler Maxey•9/9/24, 5:03 PM

Hmm, I added that to my

index.js

index.js

but it's still giving errors (I replaced projectname and account). Giving errors for both localhost and for when I go on the deployed site

[Error] Origin https://projectname.pages.dev is not allowed by Access-Control-Allow-Origin. Status code: 500
[Error] Fetch API cannot load https://event-tracking-worker.<account>.workers.dev/ due to access control checks.
[Error] Failed to load resource: Origin https://projectname.pages.dev is not allowed by Access-Control-Allow-Origin. Status code: 500 (event-tracking-worker.<account>.workers.dev, line 0)

[Error] Origin https://projectname.pages.dev is not allowed by Access-Control-Allow-Origin. Status code: 500
[Error] Fetch API cannot load https://event-tracking-worker.<account>.workers.dev/ due to access control checks.
[Error] Failed to load resource: Origin https://projectname.pages.dev is not allowed by Access-Control-Allow-Origin. Status code: 500 (event-tracking-worker.<account>.workers.dev, line 0)

index.js

index.js

file

export default {
    async fetch(request, env) {

    const corsHeaders = {
        "Access-Control-Allow-Origin": "*",  // Allow all origins
        "Access-Control-Allow-Methods": "POST, OPTIONS",  // Allow POST and OPTIONS methods
        "Access-Control-Allow-Headers": "Content-Type",  // Allow specific headers
        };
    
        if (request.method === 'OPTIONS') {
        // Handle CORS preflight requests
        return new Response(null, {
            headers: corsHeaders,
            status: 204,  // No content
        });
        }
        
        if (request.method === 'POST') {
        // Parse the incoming request body
        const requestBody = await request.json();
        const eventType = requestBody.eventType || 'unknown_event';
        const url = requestBody.url || 'unknown_url';
  
        // Log the event using the Workers Analytics Engine
        await env.ANALYTICS.writeDataPoint({
            blobs: [eventType, url],
            indexes: [eventType],
            tags: [url],
        });
  
        return new Response("Event logged successfully!", { status: 200 });
        } else {
        return new Response("Method Not Allowed", { status: 405 });
        }
    }
  };

export default {
    async fetch(request, env) {

    const corsHeaders = {
        "Access-Control-Allow-Origin": "*",  // Allow all origins
        "Access-Control-Allow-Methods": "POST, OPTIONS",  // Allow POST and OPTIONS methods
        "Access-Control-Allow-Headers": "Content-Type",  // Allow specific headers
        };
    
        if (request.method === 'OPTIONS') {
        // Handle CORS preflight requests
        return new Response(null, {
            headers: corsHeaders,
            status: 204,  // No content
        });
        }
        
        if (request.method === 'POST') {
        // Parse the incoming request body
        const requestBody = await request.json();
        const eventType = requestBody.eventType || 'unknown_event';
        const url = requestBody.url || 'unknown_url';
  
        // Log the event using the Workers Analytics Engine
        await env.ANALYTICS.writeDataPoint({
            blobs: [eventType, url],
            indexes: [eventType],
            tags: [url],
        });
  
        return new Response("Event logged successfully!", { status: 200 });
        } else {
        return new Response("Method Not Allowed", { status: 405 });
        }
    }
  };

Tyler Maxey•9/9/24, 5:18 PM

Do I need to set it up to an API? I think I'm missing something. Does anyone have a step-by-step for setting up super basic event analytics like this for a react site?

Walshy•9/9/24, 10:08 PM

no need for await on writeDataPoint &

tags

tags

is not valid

we'd need the error to debug further though so if they don't solve, please provide the error

Tyler Maxey•9/10/24, 4:45 AM

Still getting errors:

[Error] Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500
[Error] Fetch API cannot load https://event-tracking-worker.<account>.workers.dev/ due to access control checks.
[Error] Failed to load resource: Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500 (event-tracking-worker.<account>.workers.dev, line 0)
[Error] Error logging event:
TypeError: Load failed
    (anonymous function) (logEventUtility.js:20)

[Error] Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500
[Error] Fetch API cannot load https://event-tracking-worker.<account>.workers.dev/ due to access control checks.
[Error] Failed to load resource: Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500 (event-tracking-worker.<account>.workers.dev, line 0)
[Error] Error logging event:
TypeError: Load failed
    (anonymous function) (logEventUtility.js:20)

Updated

index.js

index.js

export default {
    async fetch(request, env) {
      const corsHeaders = {
        "Access-Control-Allow-Origin": "*",  // Allow all origins
        "Access-Control-Allow-Methods": "POST, OPTIONS",  // Allow POST and OPTIONS methods
        "Access-Control-Allow-Headers": "Content-Type",  // Allow specific headers
      };
  
      if (request.method === 'OPTIONS') {
        // Handle CORS preflight requests
        return new Response(null, {
          headers: corsHeaders,
          status: 204,  // No content
        });
      }
      
      if (request.method === 'POST') {
        let requestBody;
  
        try {
          requestBody = await request.json();  // Try parsing the JSON
        } catch (error) {
          return new Response("Invalid JSON", { status: 400 });
        }
  
        const eventType = requestBody.eventType || 'unknown_event';
        const url = requestBody.url || 'unknown_url';
  
        // Log the event using the Workers Analytics Engine
        env.ANALYTICS.writeDataPoint({
          blobs: [eventType, url],
          indexes: [eventType]
        });
  
        return new Response("Event logged successfully!", { status: 200 });
      } else {
        return new Response("Method Not Allowed", { status: 405 });
      }
    }
  };

export default {
    async fetch(request, env) {
      const corsHeaders = {
        "Access-Control-Allow-Origin": "*",  // Allow all origins
        "Access-Control-Allow-Methods": "POST, OPTIONS",  // Allow POST and OPTIONS methods
        "Access-Control-Allow-Headers": "Content-Type",  // Allow specific headers
      };
  
      if (request.method === 'OPTIONS') {
        // Handle CORS preflight requests
        return new Response(null, {
          headers: corsHeaders,
          status: 204,  // No content
        });
      }
      
      if (request.method === 'POST') {
        let requestBody;
  
        try {
          requestBody = await request.json();  // Try parsing the JSON
        } catch (error) {
          return new Response("Invalid JSON", { status: 400 });
        }
  
        const eventType = requestBody.eventType || 'unknown_event';
        const url = requestBody.url || 'unknown_url';
  
        // Log the event using the Workers Analytics Engine
        env.ANALYTICS.writeDataPoint({
          blobs: [eventType, url],
          indexes: [eventType]
        });
  
        return new Response("Event logged successfully!", { status: 200 });
      } else {
        return new Response("Method Not Allowed", { status: 405 });
      }
    }
  };

Tyler Maxey•9/10/24, 4:45 AM

logEventUtility.js

logEventUtility.js

I'm using to trigger the events in react

export const logEvent = async (eventType) => {
    try {
      const workerUrl = "https://event-tracking-worker.<account>.workers.dev/";
  
      const response = await fetch(workerUrl, {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          eventType: eventType,
          url: window.location.href,
        }),
      });
  
      if (!response.ok) {
        console.error("Failed to log event", response.statusText);
      }
    } catch (error) {
      console.error("Error logging event:", error);
    }
  };

export const logEvent = async (eventType) => {
    try {
      const workerUrl = "https://event-tracking-worker.<account>.workers.dev/";
  
      const response = await fetch(workerUrl, {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          eventType: eventType,
          url: window.location.href,
        }),
      });
  
      if (!response.ok) {
        console.error("Failed to log event", response.statusText);
      }
    } catch (error) {
      console.error("Error logging event:", error);
    }
  };

Walshy•9/10/24, 9:41 AM

yeah ok so it is indeed the cors that's blocking it

TypeError: Load failed

you'll need to fix your cors

chrome doesn't really handle cors on localhost, i'd recommend just testing on a real domain

Tyler Maxey•9/11/24, 7:06 AM

Thank y'all so much—I got it to work!!!

Oleksii KilevoiOP•9/11/24, 10:01 AM

Has anyone encountered 502 Sorry, we were unable to evaluate your query?

Original message was deleted

Cyril (runs-on.com)•9/12/24, 8:24 AM

Hello, any plan to be able to up that 93 days limit? I'd be willing to pay, I really like the product. Makes building grafana dashboards really easy without the need for storing metrics myself.

alan•9/15/24, 1:08 PM

if we add a new blob or double

alan•9/15/24, 1:08 PM

what happens to previous records being fetched which didnt have that blob or double

Walshy•9/15/24, 1:18 PM

They stay in the current state (defaulted - double is 0 and blob is empty string)

Ward•9/19/24, 11:09 AM

Hi Everyone, I am wondering how this could be the case. I am writing a data response every single time my worker gets called then they just send back "Hello World". I did a test with sending 1000 requests but when running the query I only see 864. How can this difference happen? Can anyone help me figure out where this problem is coming from?

acusti•9/20/24, 8:59 PM

very excited to try this out, seems like an easy but scalable way to handle custom analytics log storage and querying. a couple of questions:

i'm currently building a POC to use this to provide a basic webpage analytics dashboard to my customers for my workers-powered website builder. does that make sense? i figured i can cache results in KV based on query params / timestamp if read limits are a problem.
like @Cyril (runs-on.com), i’d be happy to pay for longer retention. i think i can tell customers that their built-in analytics can be exported but will expire at a certain point, but 93 days definitely seems unreasonable. without having done any user research, my intuition says that a year could be pretty intuitive and feel reasonable, but it would also be cool to be able to offer longer data retention to my customers for a fee. is that something CF is considering?

anhhoang•9/23/24, 9:33 AM

Hi everyone, missing data when i write datapoint from worker. Anybody have solutions for this? Please help me!

anhhoang•9/23/24, 3:01 PM

I write data different day and sample_interval = 1 when I select data order by timestamp

anhhoang•9/23/24, 3:40 PM

Still miss row when i try select sum(_sample_iterval)

Ben•9/25/24, 6:56 PM

how reliable is the data in the analytics engine?

I want to track my active users with it.

However, I need to know exactly who was active during a certain period of time (lets say a period of 10 seconds)

is this a good choice, I cannot miss users that were active during a period of time.

Jjimh We're probably going to need more information in order to help you. But have you...

anhhoang•9/26/24, 1:13 AM

Please help me

BBen how reliable is the data in the analytics engine? I want to track my active use...

Isaac McFadyen•9/26/24, 3:24 AM

Probably not. Analytics engine performs sampling which means that some data will be discarded and aggregated over a certain period of time (i.e. over 5 minutes) and you don't control when that happens. https://developers.cloudflare.com/analytics/analytics-engine/sampling/

Isaac McFadyen•9/26/24, 3:24 AM

You should probably use a relational DB or another time-series DB that doesn't sample.

BBen how reliable is the data in the analytics engine? I want to track my active use...

rohin•9/27/24, 12:11 AM

It really depends on the index you choose. If you're tracking a user_id or some other unique identifier for a user, and you set that as the index (https://developers.cloudflare.com/analytics/analytics-engine/sampling/#how-to-select-an-index), then sampling should not affect you. if you write lots of data to a single index value, then you may see that single index value get sampled, but the_sample_interval should be fairly accurate. we still read and process every event, ABR just helps us optimize how we store and serve queries from Clickhouse

Cloudflare Docs

Sampling with WAE | Cloudflare Analytics docs

How data written to Workers Analytics Engine is automatically sampled at scale

export default { async fetch(request, env) { if (request.method === 'POST') { // Parse the incoming request body const requestBody = await request.json(); const eventType = requestBody.eventType || 'unknown_event'; const url = requestBody.url || 'unknown_url'; // Log the event using the Workers Analytics Engine await env.ANALYTICS.writeDataPoint({ blobs: [eventType, url], indexes: [eventType], tags: [url], }); return new Response("Event logged successfully!", { status: 200 }); } else { return new Response("Method Not Allowed", { status: 405 }); } } };

export const logEvent = async (eventType) => { try { // Replace with your actual worker URL const workerUrl = "https://event-tracking-worker.<account>.workers.dev/"; // Send event data to the Cloudflare Worker const response = await fetch(workerUrl, { method: "POST", // Use POST or GET as per your preference headers: { "Content-Type": "application/json", }, body: JSON.stringify({ eventType: eventType, url: window.location.href, // Include the current page URL }), }); if (!response.ok) { console.error("Failed to log event", response.statusText); } } catch (error) { console.error("Error logging event:", error); } };

[Error] Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500 [Error] Fetch API cannot load https://event-tracking-worker.<account>.workers.dev/ due to access control checks. [Error] Failed to load resource: Origin http://localhost:5173 is not allowed by Access-Control-Allow-Origin. Status code: 500 (event-tracking-worker.<account>.workers.dev, line 0)

[Error] Origin https://projectname.pages.dev is not allowed by Access-Control-Allow-Origin. Status code: 500 [Error] Fetch API cannot load https://event-tracking-worker.<account>.workers.dev/ due to access control checks. [Error] Failed to load resource: Origin https://projectname.pages.dev is not allowed by Access-Control-Allow-Origin. Status code: 500 (event-tracking-worker.<account>.workers.dev, line 0)

export default { async fetch(request, env) { const corsHeaders = { "Access-Control-Allow-Origin": "*", // Allow all origins "Access-Control-Allow-Methods": "POST, OPTIONS", // Allow POST and OPTIONS methods "Access-Control-Allow-Headers": "Content-Type", // Allow specific headers }; if (request.method === 'OPTIONS') { // Handle CORS preflight requests return new Response(null, { headers: corsHeaders, status: 204, // No content }); } if (request.method === 'POST') { // Parse the incoming request body const requestBody = await request.json(); const eventType = requestBody.eventType || 'unknown_event'; const url = requestBody.url || 'unknown_url'; // Log the event using the Workers Analytics Engine await env.ANALYTICS.writeDataPoint({ blobs: [eventType, url], indexes: [eventType], tags: [url], }); return new Response("Event logged successfully!", { status: 200 }); } else { return new Response("Method Not Allowed", { status: 405 }); } } };

export default { async fetch(request, env) { const corsHeaders = { "Access-Control-Allow-Origin": "*", // Allow all origins "Access-Control-Allow-Methods": "POST, OPTIONS", // Allow POST and OPTIONS methods "Access-Control-Allow-Headers": "Content-Type", // Allow specific headers }; if (request.method === 'OPTIONS') { // Handle CORS preflight requests return new Response(null, { headers: corsHeaders, status: 204, // No content }); } if (request.method === 'POST') { let requestBody; try { requestBody = await request.json(); // Try parsing the JSON } catch (error) { return new Response("Invalid JSON", { status: 400 }); } const eventType = requestBody.eventType || 'unknown_event'; const url = requestBody.url || 'unknown_url'; // Log the event using the Workers Analytics Engine env.ANALYTICS.writeDataPoint({ blobs: [eventType, url], indexes: [eventType] }); return new Response("Event logged successfully!", { status: 200 }); } else { return new Response("Method Not Allowed", { status: 405 }); } } };

export const logEvent = async (eventType) => { try { const workerUrl = "https://event-tracking-worker.<account>.workers.dev/"; const response = await fetch(workerUrl, { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ eventType: eventType, url: window.location.href, }), }); if (!response.ok) { console.error("Failed to log event", response.statusText); } } catch (error) { console.error("Error logging event:", error); } };