Hello! Are there any plans to introduce backup, versioning, or similar features for R2? I'm curious

JJohan Hello! Are there any plans to introduce backup, versioning, or similar features ...

S

Sam•4/8/24, 12:02 PM

Also make sure to not test in production, try your scripts in a dev bucket

J

JohanOP•4/8/24, 12:04 PM

Thank you for replying!

Right, so running like a cron-job and using rclone to backup, got it. I guess you'll have to setup and manage some how often to backup it and how long to keep it for.

I guess you could mirror it to some kind of cold storage also I guess.

SSam Also make sure to not test in production, try your scripts in a dev bucket 😉

J

JohanOP•4/8/24, 12:04 PM

Yeah, not touching production until I've tried it extensively locally first

J

JohanOP•4/8/24, 12:06 PM

As a lazy dev who prefer to stay away from infra-stuff as much as possible, it would be great if Cloudflare had a managed service for backup stuff

J

JohanOP•4/8/24, 12:16 PM

Thanks man!

J

JohanOP•4/8/24, 12:51 PM

You have any cheaper cold storage solutions for backups you can recommend?

JJohan You have any cheaper cold storage solutions for backups you can recommend?

H

Hello, I’m Allie!•4/8/24, 12:53 PM

Any of the BigClouds, Scaleway Glacier

Scaleway

Glacier Cold storage | Scaleway

Ideal for deep archived data.

HHello, I’m Allie!Any of the BigClouds, [Scaleway Glacier](https://www.scaleway.com/en/glacier-col...

J

JohanOP•4/8/24, 12:54 PM

Thank you!

Z

Zora•4/8/24, 8:42 PM

I think you have to get enterprise support to get sla on r2

T

TheKaiser•4/8/24, 10:07 PM

anyone know what could be causing this error?

    [cause]: Error [AssertionError]: false == true
        at new AssertionError (node-internal:internal_assertionerror:419:15)
        at assert (node-internal:internal_assert:49:15)
        at ArrayBufferView (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/devalue.ts:95:3)
        at hydrate (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:60:32)
        at hydrate (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:115:17)
        at unflatten (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:132:9)
        at parse (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:16:9)
        at parseWithReadableStreams (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/devalue.ts:313:9)
        at #fetch (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/proxy.worker.ts:237:12)
        at async ProxyServer.fetch (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/proxy.worker.ts:125:11)

    [cause]: Error [AssertionError]: false == true
        at new AssertionError (node-internal:internal_assertionerror:419:15)
        at assert (node-internal:internal_assert:49:15)
        at ArrayBufferView (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/devalue.ts:95:3)
        at hydrate (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:60:32)
        at hydrate (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:115:17)
        at unflatten (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:132:9)
        at parse (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:16:9)
        at parseWithReadableStreams (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/devalue.ts:313:9)
        at #fetch (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/proxy.worker.ts:237:12)
        at async ProxyServer.fetch (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/proxy.worker.ts:125:11)

    [cause]: Error [AssertionError]: false == true
        at new AssertionError (node-internal:internal_assertionerror:419:15)
        at assert (node-internal:internal_assert:49:15)
        at ArrayBufferView (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/devalue.ts:95:3)
        at hydrate (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:60:32)
        at hydrate (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:115:17)
        at unflatten (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:132:9)
        at parse (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:16:9)
        at parseWithReadableStreams (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/devalue.ts:313:9)
        at #fetch (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/proxy.worker.ts:237:12)
        at async ProxyServer.fetch (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/proxy.worker.ts:125:11)

    [cause]: Error [AssertionError]: false == true
        at new AssertionError (node-internal:internal_assertionerror:419:15)
        at assert (node-internal:internal_assert:49:15)
        at ArrayBufferView (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/devalue.ts:95:3)
        at hydrate (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:60:32)
        at hydrate (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:115:17)
        at unflatten (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:132:9)
        at parse (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/.pnpm/devalue@4.3.2/node_modules/devalue/src/parse.js:16:9)
        at parseWithReadableStreams (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/devalue.ts:313:9)
        at #fetch (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/proxy.worker.ts:237:12)
        at async ProxyServer.fetch (file:///home/kai/Code/slp-vault/slpv2/node_modules/.pnpm/miniflare@3.20240329.0/node_modules/miniflare/src/workers/core/proxy.worker.ts:125:11)

im basically doing

            const slpFile = Buffer.from(base64FileData, 'base64');
            const r2Upload = await r2.put(publicId, slpFile);

            const slpFile = Buffer.from(base64FileData, 'base64');
            const r2Upload = await r2.put(publicId, slpFile);

            const slpFile = Buffer.from(base64FileData, 'base64');
            const r2Upload = await r2.put(publicId, slpFile);

            const slpFile = Buffer.from(base64FileData, 'base64');
            const r2Upload = await r2.put(publicId, slpFile);

where base64FileData is a base64 encoded string. The transferring to buffer seems to be working fine, and using the same code to upload strings and the raw base64 seems to work but I just can't get it to work with the binary

T

TheKaiser•4/8/24, 10:07 PM

The file is like 3MB btw

T

TheKaiser•4/8/24, 10:14 PM

lol the I've been stuck on this for 2 days and the magic of posting something has worked again, I just have to upload slpFile.buffer instead of just slpFile

T

TheKaiser•4/8/24, 10:14 PM

bad error message tho

A

Ahmed Eid•4/9/24, 6:33 AM

I would like to do some processing on the file uploaded to R2, however I was not able to decopress it ?

•• Iɴᴅʀᴀᴊɪᴛʜ • ‹ Dᴇᴠ / ›hey, is there any way to list files in r2 using api or smthg?

S

Sam•4/9/24, 10:01 AM

Use list objects via any s3 wrapper

M

Manuel Ogomigo•4/9/24, 4:21 PM

Hi!

I have files stored in R2, images and videos, and accessible through my domain
Is it possible to integrate cloudflare optimization images, so I can crop, transform and optimize it based on my url?

R

Raiyan•4/9/24, 5:14 PM

yes, you can use the images product where you can just use the optimization

R

Raiyan•4/9/24, 5:14 PM

and it is very cheap

G

GALFA•4/9/24, 5:17 PM

hello, can anyone help me using r2. because I just want to use it and want to learn it

I

Isaac McFadyen•4/9/24, 5:36 PM

I know it got blocked but just for future reference, there is over 25 thousand people on this server, so please don't try to ping

@here

@here

@here

@here

V

Virtuous•4/9/24, 8:03 PM

does gcp need my account keys to migrate via super slurper?

V

Virtuous•4/9/24, 8:04 PM

or is it possible to have a private key on a specific bucket in gcp

V

Virtuous•4/9/24, 11:34 PM

cheers brah

B

Bat-Erdene•4/10/24, 2:29 AM

Hey guys! How can I see my R2 objects that is uploaded locally in dev mode?
It seems that uploaded objects are stored in .wrangler/state/v3/r2/my-local-bucket/blobs/example-object.png.
Is it possible to see this image in browser like S3? Or it's not possible and should I create remote preview bucket in my account?

Z

Zxubit•4/10/24, 3:21 AM

Hello hello! I have a feature request...

TL;DR: CopyObject support for Sippy.

For context around my use-case: We are a music distributor and have a large-scale system which transcodes audio. Users upload music to us in various formats. We process it in various ways including transcoding the audio to an industry standard format and end up with a directory structure containing a bunch of XML files (containing metadata), images and audio. We then upload all this data in batches to ~30 partners around the globe (eg Spotify).

There are 2 costly parts to this. 1) Compute for the transcoding process. 2) Egress bandwidth.

We're running this system in GCP, where we store transcoded versions of audio in a 'cache' bucket so that we only need to transcode each audio file once. Due to the fact each of our partners wants batches in varying directory structures; we construct batches by copying the various files from GCS into a temporary 'batch build' bucket and then sync the data to our partners via SFTP with rclone. This relies on GCS cross-bucket copy operation.

We're migrating to R2; and want to use Sippy to pull data into our transcode cache bucket. However; when i'm trying to copy a not-yet-in-R2 file from our transcode bucket to our batch build bucket CopyObject returns a not found error.

So.. if CopyObject was 'sippy aware' and could trigger the pull down of data into the source bucket on request; THEN copy it.. that would be the ideal behaviour. In the meantime my workaround is to first check for the existence of an object in my source bucket and issue a get request if it's not been migrated yet... then the copy operation works.

Hope this makes sense!

T

Tangenius•4/10/24, 7:56 AM

Can you find somewhere the expected cost of what you used since the last billing date or something?

T

Tangenius•4/10/24, 1:03 PM

Wow weird that that doesn't exist

T

Tangenius•4/10/24, 2:38 PM

https://www.cloudflare.com/ips-v4 Does this include R2 servers?

T

Tangenius•4/10/24, 2:40 PM

Nice

J

John Spurlock•4/10/24, 5:20 PM

pricing question: R2 Class A Operations are $4.50 / million requests - does it round up to the nearest million for billing?

C

Chaika•4/10/24, 5:34 PM

I thought it didn't do that before

I know Cache Reserve was that way though

C

Chaika•4/10/24, 5:35 PM

ehh nvm I might have just never passed through the free Class A ops and I'm thinking about the storage costs

U

Unsmart•4/10/24, 5:40 PM

hmm the pricing for R2 doesnt say it rounds thats kinda messed up

U

Unsmart•4/10/24, 5:40 PM

$9 for being 8k over 1 million is rough

J

John Spurlock•4/10/24, 5:42 PM

yea its... surprising (s3 does not do this) - perhaps someone orange can weigh in on if this is official? if true, seems like something that would be useful to document: https://developers.cloudflare.com/r2/pricing/

U

Unsmart•4/10/24, 5:43 PM

I wonder how many products on cf do this now

J

John Spurlock•4/10/24, 5:43 PM

Durable Objects Compute Duration definately does

Original message was deleted

J

John Spurlock•4/10/24, 5:48 PM

it's been undocumented for a while? : )

U

Unsmart•4/10/24, 5:51 PM

since when has it ever been prepaid? :wat:

U

Unsmart•4/10/24, 5:51 PM

Ive never prepaid for it always post

J

John Spurlock•4/10/24, 5:52 PM

yea perhaps the examples on the pricing page would at least implicitly reflect this if all of the example amounts weren't evenly divisible by 1,000,000

J

John Spurlock•4/10/24, 5:52 PM

still think it's worth a mention - as other cloud paygo platforms are not like this

U

Unsmart•4/10/24, 5:52 PM

So if its prepaid then and you paid for 2 million that means on the next month any unused in that 2nd million wont be billed?

U

Unsmart•4/10/24, 5:53 PM

Then its not prepaid