Hey guys, anyone know anything about timeframe on plans to introduce workers observability into clou

Xxiaoyu2er Is there a way that I can add `observability` in terraform? https://community.cl...

L

libor.vachal•10/23/24, 7:37 PM

Same here

I have endup to enable logs manually in UI dasboard after deploying workers through terraform..

Xxiaoyu2er Is there a way that I can add `observability` in terraform? https://community.cl...

C

Cyb3r-Jak3•10/23/24, 10:08 PM

I don't think so

LLeif Hey guys, anyone know anything about timeframe on plans to introduce workers obs...

T

Thomas Ankcorn•10/24/24, 6:31 PM

We currently have no plans to support cloudflare pages.

Do you have any blockers moving this project to workers assets?

Llibor.vachal Same here 👍 I have endup to enable logs manually in UI dasboard after deploying...

M

midnight•10/25/24, 12:57 PM

tagging @boristane since I am also waiting on this.

S

Sly•10/25/24, 10:02 PM

Is logs the beginning and you’re bringing over traces and so on as well?

M

Marius B.•11/5/24, 12:14 PM

Is there a way to programmatically log certein requests in Workers Logs? For example triggered by specific routes or url params. Thanks in advance.

J

JfrAziz•11/7/24, 1:59 AM

can we export the logs?, or do you have plan to create dashboard or something than we can customize based on logs data?

A

Abakir•11/7/24, 2:46 PM

I wanted to export logs too can their be a way to export it as simple a text file depeding on your search query

N

nclevenger•11/7/24, 10:25 PM

Yes - would love the opportunity to pipe the logs directly into R2 ... I know there's LogPush, but the Schema for Observability is much better ...

B

bennycondon•11/8/24, 12:12 AM

Same here. Would love the option to push the same schema from Observability directly into R2 for long-term storage.

M

maj•11/8/24, 6:35 AM

I'm no longer seeing any logs today. They didn't just stop, but phased out slowly. Yet the worker does get it. It's only the logging which doesn't work any more.

M

maj•11/8/24, 6:37 AM

despite being called my worker function calls aren't being logged any more. I'm also seeing that the "average sampling rate" top right would go from 100% to zero when zooming in.

Bildschirmfoto_2024-11-08_um_07.36.20.png

M

maj•11/8/24, 10:23 AM

Must have been a temporary hiccup: logging is back.

T

tornado•11/10/24, 1:50 AM

For the new worker logs, is there a way to explicity designate which console.log statements should be captured. I feel like the pricing model and how this is integrated to

console.log

console.log

console.log

console.log statements make it a bit weird to use.

I think a way to have balance is to have the logger be in

runtime.env

runtime.env

runtime.env

runtime.env instead of hijacking

console.log

console.log

console.log

console.log or at least be able to set the logging level at which logs should be captured (e.g only for

console.error

console.error

console.error

console.error). This way we can have logging for important things and if we need normal debugging then the live stream logs should include all

console.logs

console.logs

console.logs

console.logs.

S

Shados•11/12/24, 3:22 PM

Hi all, can someone confirm to me if logging (both the observability, as well as logpush) is expected to work in preview urls? I got thrown for a loop today because it does not seem to generate any logs, so I couldn't figure out an exception until I promoted the version to a deploy. Is that currently expected? In Pages we didn't have log push, but you could view real time logs even for previews.

Mmaj despite being called my worker function calls aren't being logged any more. I'm ...

�

😈 Donkey 💫•11/13/24, 3:11 PM

I see a lot of log missing too
and the filter on dashboard works incorrectly, sometimes

X

xochrisxo•11/17/24, 5:18 PM

Nicat die letuten 24 stunden

K

kchro3•11/18/24, 6:57 PM

Do we know what is the timeline for alerting support? I am considering Datadog as an alternative for now

Kkchro3 Do we know what is the timeline for alerting support? I am considering Datadog a...

R

rohin•11/18/24, 7:01 PM

early next year!

R

rohin•11/18/24, 7:04 PM

DX Improvement

We introduced invocation logs to enrich the context we give you for your Workers invocation with zero instrumentation. However, we have heard from many of you that you want the ability to opt-out of invocation logs because a) added noise for high-volume log producers and b) billing implications. Starting today, you can disable invocation logs by adding the following to your wrangler.toml

[observability.logs]
invocation_logs = false

[observability.logs]
invocation_logs = false

[observability.logs]
invocation_logs = false

[observability.logs]
invocation_logs = false

You can also disable Invocation Logs via the Dashboard.

Screenshot_2024-11-18_at_10.51.45_AM.png

Rrohin early next year!

K

kchro3•11/18/24, 7:32 PM

that's awesome! if there are any beta testing openings etc, please let me know!

E

ElddisOne•11/20/24, 12:13 PM

Hi all, we're on CF Enterprise using Workers to manipulate the streaming HTML. We're looking through the logs and noticed that there are multipe entries for the same request. We're trying to find out what the events are in relation to the workers, because the CF documentation is quite thin. We are assuming that the "ParentRayID":"00" is the ingress request which is assigned the RayID of "1234", which then gets ran through a CF worker which creates a new ID "5678" and then returns that response to the client? We are uncertain because the

EdgeResponseBytes

EdgeResponseBytes

EdgeResponseBytes

EdgeResponseBytes is high for both entries. Any ideas? I've chopped some attributes out of the JSON object for brevity

  {
    "BotScore":2,
    "CacheCacheStatus":"dynamic",
    "ClientIP":"66.249.77.39",
    "ClientIPClass":"searchEngine",
    "EdgeResponseBytes":308840,
    "EdgeResponseContentType":"text/html; charset=UTF-8",
    "EdgeResponseStatus":200,
    "EdgeStartTimestamp":"2024-11-07T14:54:53Z",
    "EdgeTimeToFirstByteMs":1037,
    "ClientRequestURI":"/123",
    "ParentRayID":"1234",
    "RayID":"5678"
  }{
    "BotScore":1,
    "CacheCacheStatus":"dynamic",
    "ClientIP":"66.249.77.39",
    "ClientIPClass":"searchEngine",
    "EdgeResponseBytes":113263,
    "EdgeResponseContentType":"text/html; charset=UTF-8",
    "EdgeResponseStatus":200,
    "EdgeStartTimestamp":"2024-11-07T14:54:53Z",
    "EdgeTimeToFirstByteMs":1083,
    "ClientRequestURI":"/123",
    "BotDetectionTags":[
      "seo_imitator",
      "googlebot"
    ],
    "ParentRayID":"00",
    "RayID":"1234"
  }

  {
    "BotScore":2,
    "CacheCacheStatus":"dynamic",
    "ClientIP":"66.249.77.39",
    "ClientIPClass":"searchEngine",
    "EdgeResponseBytes":308840,
    "EdgeResponseContentType":"text/html; charset=UTF-8",
    "EdgeResponseStatus":200,
    "EdgeStartTimestamp":"2024-11-07T14:54:53Z",
    "EdgeTimeToFirstByteMs":1037,
    "ClientRequestURI":"/123",
    "ParentRayID":"1234",
    "RayID":"5678"
  }{
    "BotScore":1,
    "CacheCacheStatus":"dynamic",
    "ClientIP":"66.249.77.39",
    "ClientIPClass":"searchEngine",
    "EdgeResponseBytes":113263,
    "EdgeResponseContentType":"text/html; charset=UTF-8",
    "EdgeResponseStatus":200,
    "EdgeStartTimestamp":"2024-11-07T14:54:53Z",
    "EdgeTimeToFirstByteMs":1083,
    "ClientRequestURI":"/123",
    "BotDetectionTags":[
      "seo_imitator",
      "googlebot"
    ],
    "ParentRayID":"00",
    "RayID":"1234"
  }

  {
    "BotScore":2,
    "CacheCacheStatus":"dynamic",
    "ClientIP":"66.249.77.39",
    "ClientIPClass":"searchEngine",
    "EdgeResponseBytes":308840,
    "EdgeResponseContentType":"text/html; charset=UTF-8",
    "EdgeResponseStatus":200,
    "EdgeStartTimestamp":"2024-11-07T14:54:53Z",
    "EdgeTimeToFirstByteMs":1037,
    "ClientRequestURI":"/123",
    "ParentRayID":"1234",
    "RayID":"5678"
  }{
    "BotScore":1,
    "CacheCacheStatus":"dynamic",
    "ClientIP":"66.249.77.39",
    "ClientIPClass":"searchEngine",
    "EdgeResponseBytes":113263,
    "EdgeResponseContentType":"text/html; charset=UTF-8",
    "EdgeResponseStatus":200,
    "EdgeStartTimestamp":"2024-11-07T14:54:53Z",
    "EdgeTimeToFirstByteMs":1083,
    "ClientRequestURI":"/123",
    "BotDetectionTags":[
      "seo_imitator",
      "googlebot"
    ],
    "ParentRayID":"00",
    "RayID":"1234"
  }

  {
    "BotScore":2,
    "CacheCacheStatus":"dynamic",
    "ClientIP":"66.249.77.39",
    "ClientIPClass":"searchEngine",
    "EdgeResponseBytes":308840,
    "EdgeResponseContentType":"text/html; charset=UTF-8",
    "EdgeResponseStatus":200,
    "EdgeStartTimestamp":"2024-11-07T14:54:53Z",
    "EdgeTimeToFirstByteMs":1037,
    "ClientRequestURI":"/123",
    "ParentRayID":"1234",
    "RayID":"5678"
  }{
    "BotScore":1,
    "CacheCacheStatus":"dynamic",
    "ClientIP":"66.249.77.39",
    "ClientIPClass":"searchEngine",
    "EdgeResponseBytes":113263,
    "EdgeResponseContentType":"text/html; charset=UTF-8",
    "EdgeResponseStatus":200,
    "EdgeStartTimestamp":"2024-11-07T14:54:53Z",
    "EdgeTimeToFirstByteMs":1083,
    "ClientRequestURI":"/123",
    "BotDetectionTags":[
      "seo_imitator",
      "googlebot"
    ],
    "ParentRayID":"00",
    "RayID":"1234"
  }

W

Walshy•11/20/24, 12:28 PM

Yeah, so in this case - that one on top is a subrequest made from the Worker and the second one is the parent request
So, request from Googlebot came in, it fired a Worker (log 2) and that Worker made a subrequest to xyz (log 1)

You can match up parent to child by the

ParentRayID

ParentRayID

ParentRayID

ParentRayID like you already mentioned

WWalshy Yeah, so in this case - that one on top is a subrequest made from the Worker and...

E

ElddisOne•11/20/24, 12:31 PM

Thanks Walshy, that makes sense. We want to get the RayID which is returned to the client because we're matching the log entries here against a web crawler. For example

Googlebot makes request -> Goes into CF ("RayID: 1234") -> Goes into Worker ("RayID: 5678") -> Response returned to Googlebot ("RayID: ?????")

W

Walshy•11/20/24, 12:33 PM

the response would be the highest parent, that's the request from the eyeball and the response to the eyeball

W

Walshy•11/20/24, 12:33 PM

So, 1234 in this case

WWalshy So, 1234 in this case

E

ElddisOne•11/20/24, 12:39 PM

Ok great. In my example above, it is just simple HTML manipulation. Another case is that we are using Workers as a way to check a redirection database too. In those instances, if the origin server is expected to return a 200 status code and we intercept it via the Worker and the Worker changes it to a 301 redirect, would it then use the parent request or the worker request? The end goal is that we are trying to figure out which entry log we need to keep for some analysis of Googlebot. At the moment, having both entries is causing havoc because it's showing a url path can be both a 200 and a 301 at the exact same timestamp

W

Walshy•11/20/24, 12:42 PM

Workers are still part of the initial request so the parent - the top level one

WWalshy Workers are still part of the initial request so the parent - the top level one

E

ElddisOne•11/20/24, 12:43 PM

Nice, so we can assume if there is no ParentRayID or the ParentRayID = "00" then keep it. All others entries can be disgarded.

L

Luke•11/20/24, 5:13 PM

I'm trying to start using worker logs on my project but need a little better control over the sampling. Is there any way that I can programmatically determine whether a requests logs should be sampled or not?

For example - I'm invoking 500MM worker requests a month and each request can log anywhere from 10-1000 messages which adds up very quickly. I don't want to persist

debug

debug

debug

debug logs unless there is an

error

error

error

error. Or i don't want to persist any logs if its for a well known static content endpoint.

I was thinking I could just log everything and then use a

tail

tail

tail

tail worker to check for the presence of an

error

error

error

error log to decide what to send over to worker logs?

Original message was deleted

L

Luke•11/20/24, 6:33 PM

I see, thank you! If I were to use a tail worker and just conditionally

console.log

console.log

console.log

console.log from there w/ a 100% sample rate on that tail worker do you think that would accomplish the same thing?

L

Luke•11/20/24, 6:39 PM

ok thanks I'll give it a shot. With tail workers do you know if there is any way to pass it additional events/data or is it just given the results of console.log/fetch/exceptions?

LLuke I'm trying to start using worker logs on my project but need a little better con...

T

Thomas Ankcorn•11/21/24, 12:39 PM

For this scenario its always much easier to fix it in your code. We do plan on supporting tail sampling but you should use a logger with log level support

X

xintercept•11/21/24, 1:43 PM

morning all, I'm trying to disable the

invocation_logs

invocation_logs

invocation_logs

invocation_logs through

wrangler.toml

wrangler.toml

but it's doesn't seem to be taking effect, and resets to being enabled on each deploy - anyone hit anything similar?

Xxintercept morning all, I'm trying to disable the `invocation_logs` through `wrangler.toml...

X

xintercept•11/21/24, 1:43 PM

here's the blocks in wrangler.toml:

[env.dev.observability]
enabled = true
head_sampling_rate = 1

[env.prod.observability]
enabled = true
head_sampling_rate = 0.2

[observability.logs]
enabled = true
invocation_logs = false

[env.dev.observability]
enabled = true
head_sampling_rate = 1

[env.prod.observability]
enabled = true
head_sampling_rate = 0.2

[observability.logs]
enabled = true
invocation_logs = false

[env.dev.observability]
enabled = true
head_sampling_rate = 1

[env.prod.observability]
enabled = true
head_sampling_rate = 0.2

[observability.logs]
enabled = true
invocation_logs = false

[env.dev.observability]
enabled = true
head_sampling_rate = 1

[env.prod.observability]
enabled = true
head_sampling_rate = 0.2

[observability.logs]
enabled = true
invocation_logs = false

Xxintercept morning all, I'm trying to disable the `invocation_logs` through `wrangler.toml...

X

xintercept•11/21/24, 1:45 PM

(it would be nice to have even more control over which invocation logs are disabled, ideally I would keep the external requests, but disable the ones for queues and DOs)

Xxintercept morning all, I'm trying to disable the `invocation_logs` through `wrangler.toml...

T

Thomas Ankcorn•11/21/24, 4:12 PM

Are you on wrangler 3.88?

TThomas Ankcorn Are you on wrangler 3.88?

X

xintercept•11/21/24, 4:13 PM

yeah, 3.88.0

S

Saleem Padinharkkara•11/21/24, 4:14 PM

hi folks, I am new to Cloudflare. Loving it so far. But observability seems to be an issue. I am trying to integrate Baselime using the Edge Logger, but i dont see the logs in my baselime namespace

Xxintercept yeah, 3.88.0

T

Thomas Ankcorn•11/21/24, 4:14 PM

Ooh it looks like you need to set env.prod.observability.logs to invocation_logs = false

TThomas Ankcorn Ooh it looks like you need to set env.prod.observability.logs to invocation_logs...

X

xintercept•11/21/24, 4:16 PM

I had tried it like this too, but had the same problem:

[env.dev.observability]
enabled = true
head_sampling_rate = 1

[env.dev.observability.logs]
enabled = true
invocation_logs = false

[env.prod.observability]
enabled = true
head_sampling_rate = 0.2

[env.prod.observability.logs]
enabled = true
invocation_logs = false

[env.dev.observability]
enabled = true
head_sampling_rate = 1

[env.dev.observability.logs]
enabled = true
invocation_logs = false

[env.prod.observability]
enabled = true
head_sampling_rate = 0.2

[env.prod.observability.logs]
enabled = true
invocation_logs = false

[env.dev.observability]
enabled = true
head_sampling_rate = 1

[env.dev.observability.logs]
enabled = true
invocation_logs = false

[env.prod.observability]
enabled = true
head_sampling_rate = 0.2

[env.prod.observability.logs]
enabled = true
invocation_logs = false

[env.dev.observability]
enabled = true
head_sampling_rate = 1

[env.dev.observability.logs]
enabled = true
invocation_logs = false

[env.prod.observability]
enabled = true
head_sampling_rate = 0.2

[env.prod.observability.logs]
enabled = true
invocation_logs = false

Xxintercept I had tried it like this too, but had the same problem: ``` [env.dev.observabili...

T

Thomas Ankcorn•11/21/24, 4:48 PM

Is this just in the dash drawer? That looks like a visual bug. It seems to be disabled correctly. Can you DM me your account id and worker name please so I can check for you

TThomas Ankcorn Is this just in the dash drawer? That looks like a visual bug. It seems to be di...

X

xintercept•11/21/24, 4:53 PM

hmm, I still see the logs, it's not until I manually disable it in the UI that comes into effect, account id sent

M

Magic Greg•11/21/24, 5:11 PM

hey guys - i'm not sure if this is the right channel, but i was wondering how to get the tail worker working in local dev?

MMagic Greg hey guys - i'm not sure if this is the right channel, but i was wondering how to...

T

Thomas Ankcorn•11/21/24, 5:14 PM

Tail doesn't work right now in local dev, best to test it in prod!

M