Keep in mind that users watching 1 video will result in a lot more than 1 GetObject request, but oth

Keep in mind that users watching 1 video will result in a lot more than 1 GetObject request, but otherwise yes it should be a lot cheaper.

Pato•2/26/24, 4:41 PM

Hey! Has anyone made a ui to read from logpush to an r2 bucket?

Kkian Keep in mind that users watching 1 video will result in a lot more than 1 GetObj...

Nick•2/26/24, 6:40 PM

mmm can you help me about this?

Nick•2/26/24, 6:47 PM

let's say that one user on my platform wants to watch a 20 minutes video stored on R2. In order to do it, he needs to get a Presigned URL from my backend. How much Class A / B Operations (like GetObject) this operations will consume?

Original message was deleted

Nick•2/26/24, 7:33 PM

can you help me to understand how to do this calculation?

Nick•2/26/24, 7:37 PM

Ok, thanks for your answer, I'm gonna try with the default browser player

Nick•2/26/24, 8:59 PM

ok I watched a 24 minutes (720p / 240MB) on a new bucket and it's about 14 Class B Operations with the default player

Ercan•2/26/24, 9:43 PM

Hi, I have just started running the migration script to migrate all images from Cloudflare Images to R2, but looks like I hit rate limits, and there is no clear docs about what I can do about it. We will need to migrate around 3 million images, and I am kind of following this https://github.com/cloudflare/cloudflare-docs/pull/12615 . Also starting migration caused our prod go down because of rate limit applied for all r2 calls and cloudflare images calls.

GitHub

[R2] Add example on migrating from Cloudflare Images to R2 by nora-...

Added an example of migrating from Cloudflare Images to an R2 bucket via a Worker. Based on my gist @ https://gist.github.com/nora-soderlund/0b30a7edbcf739bc0f6fe421c8d7a6b4.

Ercan•2/26/24, 9:44 PM

Maybe if @nora can also help again on this, we also talked previously here.

Original message was deleted

Ercan•2/26/24, 9:46 PM

Let me try to get the all details for the error

Original message was deleted

Ercan•2/26/24, 9:54 PM

response: {
    status: 429,
    statusText: 'Too Many Requests',
}

response: {
    status: 429,
    statusText: 'Too Many Requests',
}

Cant see more information in the response.

And the rate limiting is happening when trying to get single image buffer data from this api https://api.cloudflare.com/client/v4/accounts/{account_id}/images/v1/{image_id}

Ercan•2/26/24, 9:56 PM

I added delay and even started pulling images grouped by 10, but still at one point it hits the rate limiting, and our production gets affected along with it.

Original message was deleted

Ercan•2/26/24, 9:59 PM

it seems like it is impossible to pull 3 million images without affecting production, is there another way that we can configure temporarily or something?

Original message was deleted

Ercan•2/26/24, 10:04 PM

I also tried with more seconds. Still getting rate limited at like 3k images

Original message was deleted

Ercan•2/26/24, 10:05 PM

I am running it locally, tried worker but worker got another errors for doing parallel requests, and another limits

Original message was deleted

Ercan•2/26/24, 10:07 PM

Along with prod, possible.

Original message was deleted

Ercan•2/26/24, 10:09 PM

Error: Too many subrequestsError: Too many subrequests

Original message was deleted

Ercan•2/26/24, 10:09 PM

The rate limiting is not on R2, it is on Downloading from Cloudflare Images -

Original message was deleted

Ercan•2/26/24, 10:11 PM

No, we are not. But I am not sure how using queues is different than slowing down the requests.

Original message was deleted

Ercan•2/26/24, 10:12 PM

That is what I am already doing, running a local script on my machine.

Isaac McFadyen•2/26/24, 10:13 PM

This ^

Isaac McFadyen•2/26/24, 10:13 PM

The Cloudflare API has very strict limits.

Isaac McFadyen•2/26/24, 10:13 PM

But the actual Images URL likely doesn't

Ercan•2/26/24, 10:14 PM

Hmm, okay that is something I can try now, I will just use the real image urls as if any client is using.

Sam•2/26/24, 10:16 PM

Question about R2 Sippy — I know it’s in beta and not recommended in production:

Currently I have a S3 bucket where content gets uploaded via Lambda, and this needs to stay this way.

When it goes to GA, can this feature handle a few hundreds of request in a short period of time? For example 1k objects are requested in a short period of time, and they don’t exist in R2 yet.
If I request a non-existing object in a Worker using the linked bucket in the “env” variable, will it also request this object from Amazon? Or is this only for HTTP request to my R2 subdomain/custom domain? Already got an answer: it should work in the worker
Is there any ETA for GA?

Ercan•2/26/24, 10:27 PM

@nora @Space - Ping in replies Looks like directly using image urls working fine! already migrated more than 5k images without any error. And not affecting prod apis. This seems to be way to go!

Ercan•2/26/24, 10:29 PM

One more quick questions, there is a Continuation token we are using the retrieve next page of images. If somehow migration stops in the middle, can I use the latest Continuation Token to continue where it left?

Original message was deleted

Ercan•2/26/24, 10:31 PM

Just a note, maybe a good idea to also update your pull request for documentation, in case some other folks get into the same case.

Sumeet•2/27/24, 3:47 AM

hi all - i followed the steps in this cloudflare r2 example and still get an error https://developers.cloudflare.com/workers/tutorials/upload-assets-with-r2/

https://r2test.sumeet-95a.workers.dev

Error on the bolded line from the dev environment

// For example, the request URL my-worker.account.workers.dev/image.png
const url = new URL(request.url);
const key = url.pathname.slice(1);
// Retrieve the key "image.png"

const object = await env.R2TEST_BUCKET.get(key);

if (object === null) {
return new Response('Object Not Found', { status: 404 });
}

Log
{
"outcome": "exception",
"scriptName": "r2test",
"diagnosticsChannelEvents": [],
"exceptions": [
{
"name": "TypeError",
"message": "Cannot read properties of undefined (reading 'get')",
"timestamp": 1709005613189
}
],

Cloudflare Docs

Securely access and upload assets with Cloudflare R2 · Cloudflare W...

This tutorial explains how to create a TypeScript-based Cloudflare Workers project that can securely access files from and upload files to a …

Uucinteractivesl range requests with video files longer than 500mb sometimes works, sometimes don...

Zora•2/27/24, 6:29 AM

You have to set the cache to bypass to always have range requests

Kiecooboi•2/27/24, 8:34 AM

can r2 delete multiple objects in single request? now i have to use a loop function to delete multiple files:NotLikeThis:

Kiecooboi•2/27/24, 8:42 AM

that's exactly my concern:lul:

Thierry Bastian•2/27/24, 10:52 AM

Hello, I am trying to use the AWS c++ SDK to upload files. For technical reasons, I want to use their TransferManager but R2 doesn't recognize the x-amz-checksum-algorithm header and erros out. I am trying to set the checksum algorithm to NOT_SET but I am facing this problem (https://github.com/aws/aws-sdk-cpp/pull/2858). So do you plan on supporting this?

GitHub

Build software better, together

GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.

Keep in mind that users watching 1 video will result in a lot more than 1 GetObject request, but oth

Similar Threads

Similar Threads

Similar Threads