Cloudflare Developers

CD

Cloudflare Developers

Welcome to the official Cloudflare Developers server. Here you can ask for help and stay updated with the latest news

Join

workers-help

pages-help

general-help

durable-objects

workers-and-pages-discussions

pages-discussions

wrangler

coding-help

kv

🦀rust-on-workers

miniflare

stream

general-discussions

functions

zaraz

⚡instant-logs

email-routing

r2

pubsub-beta

analytics-engine

d1-database

queues

workers-for-platforms

workerd-runtime

web3

🤖turnstile

radar

web-research

logs-engine

cloudflare-go

terraform-provider-cloudflare

workers-ai

browser-rendering-api

logs-and-analytics

next-on-pages

cloudflare-ai

build-caching-beta

hyperdrive

vectorize

ai-gateway

python-workers-beta

vitest-integration-beta

workers-observability

workflows

vite-plugin

pipelines-beta

containers-beta

Tony Leung
Tony Leung
View on Discord
8/19/2025

hi, I'm reading gzip file produced by

hi, I'm reading gzip file produced by pipeline saved into R2 using boto3. But using s3.get_object on that object cause FlexibleChecksumError
tminima
tminima
View on Discord
8/6/2025

Hello everyone,

Hello everyone, I am trying to follow along this tutorial: https://developers.cloudflare.com/pipelines/tutorials/query-data-with-motherduck/. I am stuck at the pipeline creation stage. When I run npx wrangler pipelines create clickstream-pipeline --r2-bucket clickstream-data --compression none --batch-max-seconds 5, I get the following error: ``` ⛅️ wrangler 4.27.0 (update available 4.28.0)...
naegelin
naegelin
View on Discord
7/30/2025

We are using pipelines in two different

We are using pipelines in two different customer cloudflare accounts. Both pipelines are standard default deployments with a worker sending json data to the pipeline. In both cases pipelines ends up leaving aborted multipart data files in r2 which leads to permanent data loss. Anyone have any ideas?
jsneedles
jsneedles
View on Discord
7/22/2025

Hi @kagitac I am trying to POC a switch

Hi @kagitac I am trying to POC a switch from kinesis firehose to CF pipelines, initially using the HTTP endpoint but also eventually moving from lambda as the entrypoint to a worker.... So far, I'm seeing pretty sluggish perf on ingestion, ranging from 400 to 1200ms, both when using the http api & a worker binding. With the worker binding im also seeing some Internal Operation Failed (which i've added retry for, but that makes the process even slower) Is this expected perf or could I be doing something funky? ...
Stockton
Stockton
View on Discord
7/8/2025

Can we assign a custom URL to the

Can we assign a custom URL to the pipeline endpoint?
Hugo
Hugo
View on Discord
7/4/2025

Hi Cole, yes sure:

Hi Cole, yes sure: bb5b7d2a19c4455faa46a808919b50aa thanks for the help!...
danamajid
danamajid
View on Discord
5/11/2025

hi @Matt Silverlock, re: dynamic

hi @Matt Silverlock, re: dynamic partitioning – the documentation states "By default, Pipelines partition data by event date and time. This will be customizable in the future.". can you share any rough timelines for when you expect to ship this? pretty critical for multi tenant workloads, so looking to assess if i can give Pipelines a shot or go down another route 🙂 thx!...
disturbing
disturbing
View on Discord
5/7/2025

Linking Cloudflare R2 as a source - Docs...

My goal is to store events for something like PostHog to injest via R2. Seems like csv and Parquet files format? https://posthog.com/docs/cdp/sources/r2 Would be great to collab on this!...
disturbing
disturbing
View on Discord
5/7/2025

Whats the difference between pipelines

Whats the difference between pipelines and logpush exactly? Assuming a custom api vs the generic one controlled by logpush?
dave
dave
View on Discord
4/28/2025

Is there any plans to go beyond 100MB

Is there any plans to go beyond 100MB for input files? Right now for some of my applications, I have to upload large files to AWS S3 first to calculate the MD5 and length, and then stream it to R2 after.