VAnyone using checksums with R2? It's reliable?
Dyeah it's reliable.
Vok great
Di'll walk you through an example of gzip vs auto decompression and why it can matter for hashes
*i just use cloudflare to hide my webserver IP and to speed things up
D^ with no-transform, this is the output:
You can see the hash of the downloaded file matches the ETag perfectly.
Dbut, with automatic decompression, the downloaded file would have a hash of
a1f2837190f85fdffa030296859eb9b1and if I'm understanding right, the etag means the client can skip requesting the body of the response if the etag matches a previously cached response?
Dyes. If you're opening files in the browser, it'll handle the etag matching stuff for you.
VI think I'm still confused haha. So no-transform is saying "don't ungzip this" for this case which means the md5 is calculated on the bytes that are actually stored.
VLike what is the pipeline with transform vs no-transform
Vr2 => cloudflare => browser
somewhere in there the file is getting unzipped, but that would happen in the browser right?
somewhere in there the file is getting unzipped, but that would happen in the browser right?
Von disk (r2) the file is stored gzipped the header
when browser requests it, the compressed file and the headers are proxied through cloudflare and sent to the browser, browser decompresses it because it has those headers
is my understanding correct?
content-encoding: gzipwhen browser requests it, the compressed file and the headers are proxied through cloudflare and sent to the browser, browser decompresses it because it has those headers
is my understanding correct?
Dno-transform is telling cf "don't ungzip this"
Vahhhh, so cloudflare ungzips it?
Vwouldn't that also have the downside that more data is transferred to the client? like the browser downloads 15mb instead of 500kb?
Dwith no-transform:
r2 (gzip) => cloudflare => browser (ungzip)
without no-transform:
r2 (gzip) => cloudflare (ungzip and possibly recompress depending on the browser) => browser
r2 (gzip) => cloudflare => browser (ungzip)
without no-transform:
r2 (gzip) => cloudflare (ungzip and possibly recompress depending on the browser) => browser
Dit'll recompress it to something like
brif the browser only supports gzip, then iirc it'll just pipe it through
Vohhh ok but it's up to CF what to do so it not good to rely on that
Vno-transform says don't touch it just forward it to the client
Vthanks for clarifying that
Dwellllll CF does do a great job imo. It's just really dependent on your use-case. Because OCR stuff is expensive (ish), I want to make sure I don't resubmit the same file that we've already processed because the hash changed.
CData retrieval is charged per GB when data in the Infrequent Access storage class is retrieved and is what allows us to provide storage at a lower price. It reflects the additional computational resources required to fetch data from underlying storage optimized for less frequent access.
Cafaik it's every time for a file in the IA tier
Cwhy we should consider it different from egress pricingdon't think you should, other then for the fact egress pricing is normally talked about as a network concept and GCS for example charges both egress + retrieval for nearline. It is the cost you'll pay for retrieving a file.
Dwhich OCR API are you using?
Vwow perfect timing
Adoes anybody have any idea why im getting a type error when trying to upload an image to R2?
A
A
KWhat is the error?
UIt technically isnt an egress charge so they can get away with it but realistically it is an egress charge yeah.
UFor 99.99% of people you will never use IA, it will only make a difference if you have like petabytes of data because the pricing difference on the storage is very tiny
Uiirc the difference of 1PB is like $15k vs $10k so if you have 1PB of data that you never access you save $5k/mo
UI did some math on it the other week lol would never make sense for me personally
Uhttps://www.cloudflare.com/plans/faq/
Cloudflare provides a SLA for our Business and Enterprise plans.
Cloudflare's Business plan offers a 100% uptime guarantee. In the event of downtime, customers receive a service credit against their monthly fee, in proportion to the respective disruption and affected customer ratio. You can view the full Business plan SLA, here: https://www.cloudflare.com/business-sla/
Cloudflare Enterprise customers receive a 10x credit (included in the Standard Success Offering) against the monthly fee, in proportion to the respective disruption and affected customer ratio. The Premium Success Offering includes 25x reimbursement uptime SLA. You can view the full Enterprise plan SLA, here: https://www.cloudflare.com/enterprise_support_sla/
GDoes anyone know if the api rate limit of (1200 requests per five minutes per user) applies to generating pre-signed urls?
HPresigned URLs shouldn't require API calls at all, so no
Gok, thank you
VUsing one from Google Cloud
VWe migrated off Google Cloud Storage using rclone and it went pretty smoothly
UAs hardlyworkin said presigned doesnt do api calls but the s3 api is not rate limited also
Dah, they require you to upload to GCS first, no?
A2nd image i sent, "second parameter is not an object"
Ai ended up just removing aws4fetch and used the sdk client instead
Aworked