Is anyone aware of any de-dupe backup software that supports R2?
Is anyone aware of any de-dupe backup software that supports R2?
Cyour backup software doesn't support S3?
Fit does but it breaks with R2
FI tried many times to get it to work
SAlso experienced this at a platform for my database backups. Using Digital Ocean spaces instead, until they get R2 to work.
Chmm, are you trying bucket scoped tokens or any bucket ones? iirc some apps still break on bucket scoped
FNot sure what it uses but we use duplicati
Fit supports B2 & S3
Cwell it's whatever you create. When you make the R2 Token, you either scope it to a specific bucket, or all buckets
Fyeah I tried that iirc
Fbeen a few months now
FGitHub
I have searched open and closed issues for duplicates. I have searched the forum for related topics. Environment info Duplicati version: latest Operating system: Fedora Backend: Cloudflare R2 Descr...
FDuplicati
Somewhat, but with lots of missing functionality, maybe explaining those (501) Not Implemented. Duplicati default logging is mostly some light logging in job database (which gives an advantage of OS independence). One problem with failed backups is it doesn’t make a job log in the usual job logs, but server log at About → Show log → Stored usu...
F¯\_(ツ)_/¯
Fits been open since 2022
Cguessing it's the streaming auth?
Fwho are you responding to?
Cyou
Cthey have this note about the aws-sdk
DisablePayloadSigning = true must be passed as Cloudflare R2 does not currently support the Streaming SigV4 implementation used by AWSSDK.S3.
Cand you can see it seems to be trying to use that streaming sigv4 auth
Cmight be possible to disable it if they're just using the aws sdk, might not be something they expose though
CTo respond to your original question, I use Kopia with R2 on my own PC. It's been working fine in my experience and has helped me recover files before when I accidentally replaced and only noticed it a month later.
FI dont see anything like that even in advanced options
DI wrote my own Worker that uses hashes for filenames.
Dhopefully! I feel like I'm gonna have to end up doing a Cron Worker eventually anyway to handle edge cases.
SBut how would you write this worker? Wouldn't this worker have to go through thousands of files to verify the thing you want
DI'd mostly be doing list objects to figure out what's missing. https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html
Returns some or all (up to 1,000) of the objects in a bucket with each request. You can use the request parameters as selection criteria to return a subset of the objects in a bucket. A 200 OK response can contain valid or invalid XML. Make sure to design your application to parse the contents of the response and handle it appropriately. For mor...
Dbut yeah 1,000 at a time is gonna suck...
SYup lmao
SWe're at 10 million files, which each need (not really, but nice to have) 3 variants. So In my case it's even more
SCloudflare images is just to expensive lmao
SWhat do you use to map your bucket? Are there any easy to use programs for that?
SOh lmao
Srclone basically executes the list object command over and over until all files are mapped?
BEspecially when you compare it to Scaleway Glacier which is only 0.002$/Go, that's expensive for glacier.
IYeah, it's more comparable to GCS Nearline
IScaleway's Glacier is definitely cheaper, but also has multi-hour retrieval times.
BWhat's the difference?
KThe blog doesn't talk about any minimum retention periods but I assume there will be some eventually
IThe retrieval time. IA is instant, Glacier is up-to hours.
IThe only Glacier type without retrieval times is GCS' Archive tier, which is cheap in storage, but very expensive in anything else.
SEverything from AWS is
IGCS Archive is cheap storage wise. The issues arrive with retrieval, egress and request costs.
IIt's cheaper than Scaleway's and it's instant retrieval.
IBut you'll need a loan to pay for the retrieval fees themselves + egress, etc.
IPlus 1 year minimum retention.
SFor example for lambda, you need a lot more than just lambda. VPC, EC2, EFS, etc, etc. Just to get an outside connection, to download something from outside lambda.
Workers is so much better. Just hoping they bring an alternative to EFS to Workers.
Workers is so much better. Just hoping they bring an alternative to EFS to Workers.
SYeah that's just S3, right?
JAWS is very cheap(in mentality):lul:
BHow do you backup your r2?
Since there is no retention or immutable object.
Since there is no retention or immutable object.
Bthanks, I'll have a look.