But how would you write this worker? Wouldn't this worker have to go through thousands of files to v
But how would you write this worker? Wouldn't this worker have to go through thousands of files to verify the thing you want
DI'd mostly be doing list objects to figure out what's missing. https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html
Returns some or all (up to 1,000) of the objects in a bucket with each request. You can use the request parameters as selection criteria to return a subset of the objects in a bucket. A 200 OK response can contain valid or invalid XML. Make sure to design your application to parse the contents of the response and handle it appropriately. For mor...
Dbut yeah 1,000 at a time is gonna suck...
SYup lmao
SWe're at 10 million files, which each need (not really, but nice to have) 3 variants. So In my case it's even more
SCloudflare images is just to expensive lmao
SWhat do you use to map your bucket? Are there any easy to use programs for that?
SOh lmao
Srclone basically executes the list object command over and over until all files are mapped?
BEspecially when you compare it to Scaleway Glacier which is only 0.002$/Go, that's expensive for glacier.
IYeah, it's more comparable to GCS Nearline
IScaleway's Glacier is definitely cheaper, but also has multi-hour retrieval times.
BWhat's the difference?
KThe blog doesn't talk about any minimum retention periods but I assume there will be some eventually
IThe retrieval time. IA is instant, Glacier is up-to hours.
IThe only Glacier type without retrieval times is GCS' Archive tier, which is cheap in storage, but very expensive in anything else.
SEverything from AWS is
IGCS Archive is cheap storage wise. The issues arrive with retrieval, egress and request costs.
IIt's cheaper than Scaleway's and it's instant retrieval.
IBut you'll need a loan to pay for the retrieval fees themselves + egress, etc.
IPlus 1 year minimum retention.
SFor example for lambda, you need a lot more than just lambda. VPC, EC2, EFS, etc, etc. Just to get an outside connection, to download something from outside lambda.
Workers is so much better. Just hoping they bring an alternative to EFS to Workers.
Workers is so much better. Just hoping they bring an alternative to EFS to Workers.
SYeah that's just S3, right?
JAWS is very cheap(in mentality):lul:
BHow do you backup your r2?
Since there is no retention or immutable object.
Since there is no retention or immutable object.
Bthanks, I'll have a look.
SIsn't Glacier more for big companies that need to store for example files that are required to be stored for a certain period? Foe example in the Netherlands you're required to save all your invoices for about 5 years, in case the IRS needs them. Just push them in there and you're set.
SYeah
RIs there a way to protect R2 content with cookies?
SYou can via workers
RAny documentation how I can do that? 
RFound it, thanks!
Dnot sure I follow how I could do a mapping if all files need to be synced
VHey folks, the content I'm storing on R2 is highly compressible. Like a 500kb vs 15mb.
Will I want to gzip this myself or is there a way to let R2 take care of it for me? And if I don't store the data gzipped, will I be charged?
Will I want to gzip this myself or is there a way to let R2 take care of it for me? And if I don't store the data gzipped, will I be charged?
Dyou gotta upload it gzip'd yourself afaik, pretty easy to do with a Worker (that's what we do).
Dmake sure to set
Content-Encoding: gzipAh interesting... so it would get stored gzipped and downloading it this will happen transparently?
VSame if I fetch it using the R2 binding on the backend?
VIf so, very cool! I'm surprised I survived so long without understanding all these http details 
Dyep, unless you set
cache-control: no-transform
PHi Community, How to solve this issue for presigned URL
D@Venkat Dinavahi e.g. you can see this fails to decompress automatically, because I have
In some situations, like archival, you don't want files to be modified automatically (because their hash changes).
no-transformIn some situations, like archival, you don't want files to be modified automatically (because their hash changes).
DI actually see a R2 "bug" now lol. The 
Vary: Accept-EncodingCache-Control: no-transformCould you clarify what you mean by "modified automatically"?
VLike cloudflare will transform (in this case un-gzip) the file?
Dyes, just not for the object URL I gave as an example
Dbut again it can be disabled with
Cache-Control: no-transformamazing - then I'm not understanding why we would want to set no-transform
Vwhen is a hash necessary?
Dif you want to set (or "change")
cache-control?response-content-encoding=blahblah