Logpush log format is can't parse in Datadog

I contacted with DataDog support. The answer is this:
Thanks for reaching out to the Datadog support team! My name is Victorien and I'll handle your request from now on.

After further investigation, this does appear to be a limitation from Cloudflare. Logs are sent as arrays of objects and sent as part of Workers Trace Events: https://developers.cloudflare.com/logs/reference/log-fields/account/workers_trace_events/.

On our end, there is no option to split logs after ingestion. This action must be performed upstream. Therefore, I would suggest reaching out to Cloudflare to inquire whether they could eventually provide flexibility in the way logs are output.

Alternatively, you could send your Cloudflare logs to a custom destination, split those logs on your end (with a script for example), and then forward these logs to Datadog (it could be achieved by using the Datadog Agent or our API: https://docs.datadoghq.com/logs/log_collection/?tab=host).
Thanks for reaching out to the Datadog support team! My name is Victorien and I'll handle your request from now on.

After further investigation, this does appear to be a limitation from Cloudflare. Logs are sent as arrays of objects and sent as part of Workers Trace Events: https://developers.cloudflare.com/logs/reference/log-fields/account/workers_trace_events/.

On our end, there is no option to split logs after ingestion. This action must be performed upstream. Therefore, I would suggest reaching out to Cloudflare to inquire whether they could eventually provide flexibility in the way logs are output.

Alternatively, you could send your Cloudflare logs to a custom destination, split those logs on your end (with a script for example), and then forward these logs to Datadog (it could be achieved by using the Datadog Agent or our API: https://docs.datadoghq.com/logs/log_collection/?tab=host).
Any ideas for this?
2 Replies
kian
kian6mo ago
You could use Tail Workers which change the format to be whatever you want and then they make a subrequest to DataDog I suppose. https://developers.cloudflare.com/workers/observability/tail-workers/
pigri
pigri6mo ago
But this extra cost datadog log push is an official integration. I hope Cloudflare team is fix it