Cyou
Cthey have this note about the aws-sdk
DisablePayloadSigning = true must be passed as Cloudflare R2 does not currently support the Streaming SigV4 implementation used by AWSSDK.S3.
Cand you can see it seems to be trying to use that streaming sigv4 auth
Cmight be possible to disable it if they're just using the aws sdk, might not be something they expose though
CTo respond to your original question, I use Kopia with R2 on my own PC. It's been working fine in my experience and has helped me recover files before when I accidentally replaced and only noticed it a month later.
FI dont see anything like that even in advanced options
DI wrote my own Worker that uses hashes for filenames.
Dhopefully! I feel like I'm gonna have to end up doing a Cron Worker eventually anyway to handle edge cases.
SBut how would you write this worker? Wouldn't this worker have to go through thousands of files to verify the thing you want
DI'd mostly be doing list objects to figure out what's missing. https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html
Returns some or all (up to 1,000) of the objects in a bucket with each request. You can use the request parameters as selection criteria to return a subset of the objects in a bucket. A 200 OK response can contain valid or invalid XML. Make sure to design your application to parse the contents of the response and handle it appropriately. For mor...
Dbut yeah 1,000 at a time is gonna suck...
SYup lmao
SWe're at 10 million files, which each need (not really, but nice to have) 3 variants. So In my case it's even more
SCloudflare images is just to expensive lmao
SWhat do you use to map your bucket? Are there any easy to use programs for that?
SOh lmao
Srclone basically executes the list object command over and over until all files are mapped?
BEspecially when you compare it to Scaleway Glacier which is only 0.002$/Go, that's expensive for glacier.
IYeah, it's more comparable to GCS Nearline
IScaleway's Glacier is definitely cheaper, but also has multi-hour retrieval times.
BWhat's the difference?
KThe blog doesn't talk about any minimum retention periods but I assume there will be some eventually
IThe retrieval time. IA is instant, Glacier is up-to hours.
IThe only Glacier type without retrieval times is GCS' Archive tier, which is cheap in storage, but very expensive in anything else.
SEverything from AWS is
IGCS Archive is cheap storage wise. The issues arrive with retrieval, egress and request costs.
IIt's cheaper than Scaleway's and it's instant retrieval.
IBut you'll need a loan to pay for the retrieval fees themselves + egress, etc.
IPlus 1 year minimum retention.
SFor example for lambda, you need a lot more than just lambda. VPC, EC2, EFS, etc, etc. Just to get an outside connection, to download something from outside lambda.
Workers is so much better. Just hoping they bring an alternative to EFS to Workers.
Workers is so much better. Just hoping they bring an alternative to EFS to Workers.
SYeah that's just S3, right?
JAWS is very cheap(in mentality):lul:
BHow do you backup your r2?
Since there is no retention or immutable object.
Since there is no retention or immutable object.
Bthanks, I'll have a look.
SIsn't Glacier more for big companies that need to store for example files that are required to be stored for a certain period? Foe example in the Netherlands you're required to save all your invoices for about 5 years, in case the IRS needs them. Just push them in there and you're set.
SYeah
RIs there a way to protect R2 content with cookies?
SYou can via workers
RAny documentation how I can do that? 
RFound it, thanks!
Dnot sure I follow how I could do a mapping if all files need to be synced
VHey folks, the content I'm storing on R2 is highly compressible. Like a 500kb vs 15mb.
Will I want to gzip this myself or is there a way to let R2 take care of it for me? And if I don't store the data gzipped, will I be charged?
Will I want to gzip this myself or is there a way to let R2 take care of it for me? And if I don't store the data gzipped, will I be charged?
Dyou gotta upload it gzip'd yourself afaik, pretty easy to do with a Worker (that's what we do).
Dmake sure to set when uploading, that way CF can automatically decompress it for clients that don't support gzip too.
VAh interesting... so it would get stored gzipped and downloading it this will happen transparently?
VSame if I fetch it using the R2 binding on the backend?
VIf so, very cool! I'm surprised I survived so long without understanding all these http details 
Dyep, unless you set , which blocks CF from decompressing it
PHi Community, How to solve this issue for presigned URL
D@Venkat Dinavahi e.g. you can see this fails to decompress automatically, because I have set. https://azure-di.6873d51718aa43a6389a532434554150.r2.cloudflarestorage.com/2024-02-29-preview/prebuilt-read/a45b747b5ed4c37de569213aef10649a.json?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=022cbef64017d6c3154435aeb62aeac1%2F20240403%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20240403T150153Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=b09dd0376defa5ce6ea683f2469304e877bc99d5626bb5c0266b3f3fa8d4a0d1
In some situations, like archival, you don't want files to be modified automatically (because their hash changes).
In some situations, like archival, you don't want files to be modified automatically (because their hash changes).