Workers & R2 fall under the "Developer Platform" supplemental agreement in ToS. Cache does not. If

Workers & R2 fall under the "Developer Platform" supplemental agreement in ToS.
Cache does not.

If you're putting non-HTML content into cache (workers cache API or configuring Cloudflare Cache on your zone for that content), you're going to run afoul of the ToS.

Hope that clears it up.

beelzabubOP•7/12/22, 10:37 PM

There's a new release:

S3 XML documents sent to R2 that have an XML declaration aren't rejected with 400 Bad Request / MalformedXML.
Minor S3 XML compatability fix impacting Arq Backup on Windows only (not the Mac version). Response now contains XML declaration tag prefix and the xmlns attribute is present on all top-level tags in the response.
Beta ListMultipartUploads support

kian•7/12/22, 10:37 PM

cc @Erisa | Out of office

Kkian cc @Erisa | Out of office

Erisa•7/12/22, 10:38 PM

Erisa•7/12/22, 10:38 PM

Its alive!

Erisa•7/12/22, 10:40 PM

listing and restoring backup records seems to work perfectly as well

kian•7/12/22, 10:41 PM

Erisa•7/12/22, 10:42 PM

kian•7/12/22, 10:43 PM

kian•7/12/22, 10:43 PM

Awesomeeeeeeeee

kian•7/12/22, 10:51 PM

Before I move this into , are all of those query params now supported or are any still unimplemented?

beelzabubOP•7/12/22, 11:24 PM

I think we wanted to hold off on moving it to supported. Maybe it should have a

next to it since there may be bugs?

beelzabubOP•7/12/22, 11:25 PM

I know isn't correctly applied to all fields

kian•7/12/22, 11:30 PM

I'm not good at wording - , , blah - want to keep it concise like the other two.

kian•7/12/22, 11:30 PM

sounds like a branch or different endpoint - would probably be better or just the former

kian•7/12/22, 11:33 PM

The rest of the docs usage of just has at the end of it

beelzabubOP•7/12/22, 11:33 PM

I don't have strong opinions. I'll defer to Kate

beelzabubOP•7/12/22, 11:34 PM

Delimiter, prefix, and key-marker we can mark as supported

beelzabubOP•7/12/22, 11:34 PM

Same for max-uploads

beelzabubOP•7/12/22, 11:34 PM

Although technically the (hard to hit unless you're John) corner case bugs are when you do specify a delimiter so....

beelzabubOP•7/12/22, 11:35 PM

Actually I think even @john.spurlock didn't find the issues because they're bugs in

kian•7/12/22, 11:35 PM

If there's a bug I usually assume John or Albert found it

kian•7/12/22, 11:35 PM

beelzabubOP•7/12/22, 11:36 PM

This one was the developer who implemented it (but using John's test case)

John Spurlock•7/12/22, 11:38 PM

ListObjectsV1 ??? I don't know how to count that low

kian•7/12/22, 11:43 PM

When I first looked at Arq’s incompatibility like a month ago, I remember wondering why S3 and R2’s ListObjects responses were so different

kian•7/12/22, 11:43 PM

Eventually realised one was V1 and one was V2

kian•7/12/22, 11:43 PM

Not used to having to throw the query params in myself

Bbeelzabub Workers & R2 fall under the "Developer Platform" supplemental agreement in ToS. ...

Cat smoocher•7/12/22, 11:52 PM

That sorta clears it up, thanks! Still wrapping my head around the services. Regarding the Cloudflare Cache, doesn't it already cache default file types (including images) as mentioned here https://developers.cloudflare.com/cache/about/default-cache-behavior/, or am I mixing up the services

kian•7/12/22, 11:56 PM

Yes, no and maybe.

On a serious note, yeah. Cloudflare's 'cache' for any proxied records (or subrequest

fetch()

fetch()

calls in a Worker) caches based on file extension & media assets are included. The specific terminology in 2.8 of the self-serve terms is that a disproportionate amount of non-HTML content is not allowed.

If I have a blog, embedding a few images is no issue - maybe even a portfolio website might not be too bad if you're not caching the super high-res variants. But if I had a website dedicated to uploading or sharing images, effectively an image CDN, that would likely get the attention of sales.

Workers don't automatically use that cache outside of the subrequest

fetch()

fetch()

calls I mentioned - you would need to access it yourself using the

cache

cache

API. That's something you're going to want to avoid doing if you will be doing a lot of non-HTML storage/transfers with R2.

https://developers.cloudflare.com/workers/learning/how-the-cache-works/#interacting-with-the-cloudflare-cache

kian•7/12/22, 11:58 PM

Nothing in https://developers.cloudflare.com/r2/examples/demo-worker/ would prompt any cache usage, since there's no

fetch()

fetch()

calls (only using Workers bindings) and no

cache

cache

API usage.

kian•7/13/22, 12:07 AM

The fun, technical side that I might be completely wrong about is that Workers run before

so if I request your Worker, it will always invoke your Worker.

request to just

my-worker

my-worker

never hits the cache 'layer'. If I do a

fetch

fetch

subrequest for (which goes through

cache

cache

origin

origin

) then I would hit the cache layer.

->

cache

cache

origin

origin

->

I can also access

cache

cache

specifically with the API (same cache namespace as

fetch

fetch

requests) or to open a new/existing namespaced cache.

I think custom domains (https://developers.cloudflare.com/workers/platform/routing/custom-domains/) might have cache in-front of them at some point but that's just wishful thinking.

Public buckets/domains for R2 might also do the same at some point in the future? geblobcatshrug

Cat smoocher•7/13/22, 12:16 AM

thanks for the explanation and demo @kian NOTED

! So I can use R2 to store images, serve them with my worker, but i should not manually invoke the cache API or a fetch() on those worker requests to r2 since that would cause some TOS issues while serving the content.

Since there's no caching allowed for that scenario, and since data we upload to r2 is only stored in one location, does that result in users in a different "region" experiencing higher latency than they would otherwise had that content been cached?

Cat smoocher•7/13/22, 12:19 AM

overall, just trying to figure out if r2 is right for my usecase, if I need caching or if im overthinking things, which i tend to do, lol PU_PeepoLaugh

kian•7/13/22, 12:19 AM

Buckets (and on the object-level I believe) is dictated by where the bucket was created, or object was uploaded.

So yeah, whilst the cache exists in every colo (all 350+ of them) and R2 has a lot fewer points of presence, there's lots of performance improvements on the way with R2.

kian•7/13/22, 12:19 AM

If most of your users are in the EU, then create bucket/upload objects from there & that fun stuff

kian•7/13/22, 12:20 AM

r2 is a good write-up by Vitali on the current 'status' of R2 and where it's at with durability/availability/performance with performance being the current focus

kian•7/13/22, 12:20 AM

kian•7/13/22, 12:20 AM

fwiw my betteruptime for Workers R2 bindings

kian•7/13/22, 12:22 AM

R2 is strongly consistent - you write, you read straight after, everyone sees the new value. That does make it a lot more difficult to have global replicas of data than something eventually consistent... like KV!

Workers KV is eventually consistent & cached (not the same cache we've been talking about), so that's good for write-once read-many scenarios if you're OK with global replication possibly taking 60 seconds or more. Cold reads might be around ~130ms, the same I'm getting from my R2 bucket, but once that key is 'warmed up' and in KV's cache then the reads are a lot faster at like ~20ms.

kian•7/13/22, 12:23 AM

They do have different use-cases though (aside from the consistency) - R2 is your normal blob object storage, with object sizes up to 5TB whereas KV is limited to 25MiB.

kian•7/13/22, 12:36 AM

Kate will know how to word it - Kate always knows

Kkian Buckets (and on the object-level I believe) is dictated by where the bucket was ...

Cat smoocher•7/13/22, 12:40 AM

For my usecase, i'd really just need to store and serve user uploaded images. The rest of my data lives in Google Firestore. I like the idea of not having to pay egress and all the other fees that S3/Google Storage would definitely require so r2 seemed like it could be a good fit. I did see that CF also has an Image service aPES_Think

If I just need to exclusively store/serve images, could that be a better solution? What are your thoughts?

Although given the improvements being planned for r2 and the response times you sent, it doesnt seem like the inability to "cache" is a big deal for me...maybe. I really dont know enough about CDN/caching to know if it's going to be a big enough impact to me. I dont have any benchmarks or real data since I'm still building my site out

kian•7/13/22, 12:44 AM

Images: fully managed image pipeline. Images are stored there and you can create variants for resizing, cropping, next-gen formats like WEBP and AVIF, compression. Like Imgix.

Pricing model: per image stored and per image delivered

Image Resizing: Similar to Images except your files can be stored wherever - IR just provides on-the-fly transformations like the above ones that Images can do.

Pricing model: per cache miss on a resized image, IR has its own cache.

KV: eventually consistent global key-value store up to 25MiB, like Redis - would serve your raw image

Pricing model: storage and operations, only realistically usable with a Worker so Worker requests

R2: strongly consistently object storage, like S3 or GCS - would serve your raw image

Pricing model: Ditto KV but public buckets are on the way

kian•7/13/22, 12:45 AM

Features like Polish will already compress and turn images into WEBP on your website - so maybe the transformations aren’t that valuable.

kian•7/13/22, 12:45 AM

It’s all up to you, and also planning for scaling - would you eventually allow arbitrary user uploads which might be big enough to need R2? Will you be OK managing KV for smaller files and R2 for bigger ones, or would you rather merge it?

kian•7/13/22, 12:57 AM

R2 also has a much more generous free tier if that matters to you

kian•7/13/22, 12:57 AM

KV has 1GB, 30k writes, 3.0mil reads (approx) and R2 has 10GB, 1.0mil writes and 10.0mil reads (approx) on their free tiers