Thread

7 Replies

Hey! Can you provide any other details about how you got here? Is this a custom schema or created via the dashboard/wrangler?

switzOP•2mo ago

I shared in the other thread https://discord.com/channels/595317990191398933/1420923406869856297/1421029269164916869 fixed by using snake_case

Micah Wylde•2mo ago

ok, interesting — I wouldn't expect that error message for that issue, but I'll look into what happened there

switzOP•2mo ago

btw, my test data never showed up in the bucket

curl -X POST https://[redacted].ingest.cloudflare.com \
                                      -H "Content-Type: application/json" \
                                      -d '[
                                  {
                                    "match_id": 42,
                                    "time": 1758870176032,
                                    "svms": 3.14
                                  }
                                  ]'

curl -X POST https://[redacted].ingest.cloudflare.com \
                                      -H "Content-Type: application/json" \
                                      -d '[
                                  {
                                    "match_id": 42,
                                    "time": 1758870176032,
                                    "svms": 3.14
                                  }
                                  ]'

{"success":true,"result":{"committed":1}}⏎ oh nevermind, it did now maybe just was batching are there plans for a UI to query data pipelines?

Micah Wylde•2mo ago

Sorry for missing this—how often we write to the bucket is controlled by the rollover configs on the sink, which can be defied by time, size, or inactivity: https://developers.cloudflare.com/pipelines/sinks/available-sinks/r2/#batching-and-rolling-policy

Cloudflare Docs

Write data as JSON or Parquet files to R2 object storage

Micah Wylde•2mo ago

By default, we use a 5 minute rollover interval but it can be set as low as 10 seconds for lower latency

Micah Wylde•2mo ago

If you’re ingesting into r2 data catalog (iceberg) you can use r2 sql to query it https://developers.cloudflare.com/r2-sql/ We don’t yet have a UI, just wrangler and an API but we’ll be adding a UI in the future.

Cloudflare Docs

R2 SQL

A distributed SQL engine for R2 Data Catalog

Gaming

Programming

Thread

Did you find this page helpful?