Ah interesting... so it would get stored gzipped and downloading it this will happen transparently?
Ah interesting... so it would get stored gzipped and downloading it this will happen transparently?
VSame if I fetch it using the R2 binding on the backend?
VIf so, very cool! I'm surprised I survived so long without understanding all these http details 
Dyep, unless you set
cache-control: no-transform
PHi Community, How to solve this issue for presigned URL
D@Venkat Dinavahi e.g. you can see this fails to decompress automatically, because I have
In some situations, like archival, you don't want files to be modified automatically (because their hash changes).
no-transformIn some situations, like archival, you don't want files to be modified automatically (because their hash changes).
DI actually see a R2 "bug" now lol. The 
Vary: Accept-EncodingCache-Control: no-transformCould you clarify what you mean by "modified automatically"?
VLike cloudflare will transform (in this case un-gzip) the file?
Dyes, just not for the object URL I gave as an example
Dbut again it can be disabled with
Cache-Control: no-transformamazing - then I'm not understanding why we would want to set no-transform
Vwhen is a hash necessary?
Dif you want to set (or "change")
cache-control?response-content-encoding=blahblah
*that looks like a very short and simple URL
Dif you're storing the compressed data in other places, sometimes it can just be easier to not have automatic decompression so you can verify nothing is corrupted.
If you aren't storing stuff in multiple places with different S3 backends, you probably don't care.
If you aren't storing stuff in multiple places with different S3 backends, you probably don't care.
Vah got it, so you can fetch the thing that's actually stored on R2 and check the contents/hash of it
Vwhat would that mean for the checksum? if I want to send a sha1 checksum, is that going to be of the gzipped file or the ungzipped file?
VAlso, funny thing is I'm also dealing with storing OCR data
PHi any one regarding this
Dit'd be the checksum of the gzipped file. But when your client downloads it, it won't have the same checksum as it'll be decompressed.
DAzure is the best I've seen so far, though their API has a lot of pitfails.. that's why we wrote a bunch of Workers stuff to make it sane. We've OCR'd a couple hundred thousand pages iirc with our setup.
VI fell in love with the cloudflare platform.. having everything centralized DNS, workers, storage, etc has been a huge benefit
Vless infrastructure details to think about
Deh, that's presigned S3 URLs for you. The "shorter" version we made with public R2 buckets, a WAF rule, transform rules, and a redirect rule is https://azure-di-source-files.auto.ai.moda/83d6300e2bb85d71b49bedd61f19fc37?cache-control=private%2C+immutable%2C+no-transform%2C+max-age=31536000%2C+stale-while-revalidate=31536000%2C+stale-if-error=31536000%2C+s-maxage=0&expires=Thu%2C+01+Jan+1970+00:00:00+GMT&verify=1712102400-4wZJur5FPt%2FlgOxQfSXl4Yryh6J%2Bjsj0N482RnL7jNc= 
Bif the verify changes, shouldn't this invalidate the cache?
Dit will only invalidate locally on your machine. It won't invalidate the cached R2 object in CF.
Dand to reduce the amount of that happening locally, we also round the timestamps to the current day.
*i see
*i was half joking cause idk what these URL really mean lol
Dit's a fair question.
VAnyone using checksums with R2? It's reliable?
Dyeah it's reliable.
Vok great
Di'll walk you through an example of gzip vs auto decompression and why it can matter for hashes
*i just use cloudflare to hide my webserver IP and to speed things up
D^ with no-transform, this is the output:
You can see the hash of the downloaded file matches the ETag perfectly.
Dbut, with automatic decompression, the downloaded file would have a hash of
a1f2837190f85fdffa030296859eb9b1and if I'm understanding right, the etag means the client can skip requesting the body of the response if the etag matches a previously cached response?
Dyes. If you're opening files in the browser, it'll handle the etag matching stuff for you.
VI think I'm still confused haha. So no-transform is saying "don't ungzip this" for this case which means the md5 is calculated on the bytes that are actually stored.
VLike what is the pipeline with transform vs no-transform
Vr2 => cloudflare => browser
somewhere in there the file is getting unzipped, but that would happen in the browser right?
somewhere in there the file is getting unzipped, but that would happen in the browser right?
Von disk (r2) the file is stored gzipped the header
when browser requests it, the compressed file and the headers are proxied through cloudflare and sent to the browser, browser decompresses it because it has those headers
is my understanding correct?
content-encoding: gzipwhen browser requests it, the compressed file and the headers are proxied through cloudflare and sent to the browser, browser decompresses it because it has those headers
is my understanding correct?
Dno-transform is telling cf "don't ungzip this"
Vahhhh, so cloudflare ungzips it?
Vwouldn't that also have the downside that more data is transferred to the client? like the browser downloads 15mb instead of 500kb?
Dwith no-transform:
r2 (gzip) => cloudflare => browser (ungzip)
without no-transform:
r2 (gzip) => cloudflare (ungzip and possibly recompress depending on the browser) => browser
r2 (gzip) => cloudflare => browser (ungzip)
without no-transform:
r2 (gzip) => cloudflare (ungzip and possibly recompress depending on the browser) => browser
Dit'll recompress it to something like
brif the browser only supports gzip, then iirc it'll just pipe it through