If you use a custom domain and set up caching in front of it, you can conserve class B ops, serving

KKarew If you use a custom domain and set up caching in front of it, you can conserve c...

S

sisifo2•10/8/23, 11:09 PM

I tried that, the first time the cache miss and the next ones hit the cache, but few minutes later if I try again, the cache miss again, reaching R2 and consuming class B ops

K

KarewOP•10/8/23, 11:10 PM

Your file has to be accessed regularly to stay in the cache. Cloudflare’s CDN is very aggressive at evicting unused files

K

KarewOP•10/8/23, 11:10 PM

You can set up a cache rule to try to make it longer-lived, but they will evict unused files

K

KarewOP•10/8/23, 11:11 PM

If your file is popular or being accessed a lot it will stay in the cache and do the actual money saving there

KKarew If your file is popular or being accessed a lot it will stay in the cache and do...

S

sisifo2•10/8/23, 11:13 PM

Thanks Karew for your help. This makes the cost of my app difficult to predict ... will review other approaches

Ssisifo2 Thanks Karew for your help. This makes the cost of my app difficult to predict ....

C

Chaika•10/8/23, 11:21 PM

Class B's are super cheap anyway (0.36/mil), If you were trying to predict your costs, I would just assume all will be uncached in your calculations and then go off real world data. It can vary, depends how spread out your users are as well as access frequency, etc.

but few minutes later if I try again, the cache miss again,

This could also just be because you're hitting a cloudflare multi-colo pop, a cloudflare location made up of multiple datacenters, each with their own cache. Your file might not have been evicted, just your connection expired and you were routed to a diff colo

M

m0jo•10/9/23, 5:40 AM

hi all, is there anyway to find out how much bandwidth you have used per month for your R2 bucket? im currently hosting a bucket that will have up to 200gb worth of files, and i would like to know if there is a way to find out how much you use.

Mm0jo hi all, is there anyway to find out how much bandwidth you have used per month f...

K

KarewOP•10/9/23, 6:37 AM

This is one way (uses the analytics API) https://denoflare.dev/cli/analytics/r2

KKarew This is one way (uses the analytics API) <https://denoflare.dev/cli/analytics/r2...

M

m0jo•10/9/23, 6:37 AM

thank you!

KKarew This is one way (uses the analytics API) <https://denoflare.dev/cli/analytics/r2...

M

m0jo•10/9/23, 6:47 AM

is there any other way? i keep getting this error when trying to execute the command to download denoflare

K

KarewOP•10/9/23, 6:47 AM

I don't really know how to install it on Windows, sorry

KKarew I don't really know how to install it on Windows, sorry

M

m0jo•10/9/23, 6:47 AM

ah ok, thats all good

M

m-reidy•10/9/23, 2:01 PM

Hi all - we're currently testing out an R2 integration and are hitting intermittent CORS issues. One of our three engineers keeps getting CORS errors on a PUT presigned URL operation, myself and our other engineer have no issues at all. As far as I'm aware our CORS policy on the bucket is correctly configured.

M

m-reidy•10/9/23, 2:01 PM

Any chance someone could help me debug?

K

kian•10/9/23, 2:02 PM

Compare the request headers of the failing requests with your CORS policy

M

m-reidy•10/9/23, 2:09 PM

Hi - yes we've done that. There's a fair few differences between the request and response headers across the two clients.

M

m-reidy•10/9/23, 2:09 PM

Successful client -

M

m-reidy•10/9/23, 2:10 PM

Failing client -

K

kian•10/9/23, 2:11 PM

What is the response body on the failing client?

M

m-reidy•10/9/23, 2:12 PM

Afraid it's a cors error so the response isn't accessible

M

m-reidy•10/9/23, 2:12 PM

Here's the relevant CORS policy -

M

m-reidy•10/9/23, 2:13 PM

Differences I've noted anyway - there's no return access-control-allow-origin header on the failing client (or the origin isn't set so we can't see it). There's some difference in the cache request headers but I don't believe that would have an impact

Mm-reidy Afraid it's a cors error so the response isn't accessible

K

kian•10/9/23, 2:13 PM

Copy the request from devtools into curl and send it there

K

kian•10/9/23, 2:14 PM

CORS itself won't cause a traditional 401/403 (like a signature mismatch), so you're probably failing CORS due to the nature of a non-2XX response.

M

m-reidy•10/9/23, 2:17 PM

Ok thanks we'll have a quick crack at that

Mm0jo is there any other way? i keep getting this error when trying to execute the com...

D

digitalpoint•10/9/23, 3:27 PM

Bandwidth isn't billable (as I'm sure you know), but if you want to see it just for sake of curiosity I made a website plugin that lets me see that info for the bucket I'm using as well as for the Cloudflare account as a whole when you hover over the bucket (you can kind of see it in the screenshot). If you have the ability to run GraphQL queries, I can go digging and grab the exact query I'm using for you. But again... bandwidth is free so maybe not even worth bothering setting up a way to do GraphQL queries?

R

r4yan•10/9/23, 10:25 PM

Hi i have the following code to upload images to an r2 bucket but it keeps giving me 401 unauthorized

  const R2_BUCKET = "rep-bucket";
  
  const getR2Client = () => {
    return new S3Client({
        region: "auto",

        endpoint: `https://${sanitizedConfig.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
    
        credentials: {
    
            accessKeyId: sanitizedConfig.R2_ACCESS_KEY,
    
            secretAccessKey: sanitizedConfig.R2_SECRET_KEY,
    
        },
    });
  };
  
  export default class S3 {
    static async get(fileName: string) {
      const file = await getR2Client().send(
        new GetObjectCommand({
          Bucket: R2_BUCKET,
          Key: fileName,
        }),
      );
  
      if (!file) {
        throw new Error("not found.");
      }
  
      return file.Body;
    }
  
    static async put(fileName: string, data: Buffer) {
      const signedUrl = await getSignedUrl(
        getR2Client(),
        new PutObjectCommand({
          Bucket: R2_BUCKET,
          Key: fileName,
        }),
        { expiresIn: 60 },
      );
  
      console.log(signedUrl);
  
      await fetch(signedUrl, {
        method: "PUT",
        body: data,
      });
  
      return `Success`;
    }
  }

  const R2_BUCKET = "rep-bucket";
  
  const getR2Client = () => {
    return new S3Client({
        region: "auto",

        endpoint: `https://${sanitizedConfig.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
    
        credentials: {
    
            accessKeyId: sanitizedConfig.R2_ACCESS_KEY,
    
            secretAccessKey: sanitizedConfig.R2_SECRET_KEY,
    
        },
    });
  };
  
  export default class S3 {
    static async get(fileName: string) {
      const file = await getR2Client().send(
        new GetObjectCommand({
          Bucket: R2_BUCKET,
          Key: fileName,
        }),
      );
  
      if (!file) {
        throw new Error("not found.");
      }
  
      return file.Body;
    }
  
    static async put(fileName: string, data: Buffer) {
      const signedUrl = await getSignedUrl(
        getR2Client(),
        new PutObjectCommand({
          Bucket: R2_BUCKET,
          Key: fileName,
        }),
        { expiresIn: 60 },
      );
  
      console.log(signedUrl);
  
      await fetch(signedUrl, {
        method: "PUT",
        body: data,
      });
  
      return `Success`;
    }
  }

  const R2_BUCKET = "rep-bucket";
  
  const getR2Client = () => {
    return new S3Client({
        region: "auto",

        endpoint: `https://${sanitizedConfig.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
    
        credentials: {
    
            accessKeyId: sanitizedConfig.R2_ACCESS_KEY,
    
            secretAccessKey: sanitizedConfig.R2_SECRET_KEY,
    
        },
    });
  };
  
  export default class S3 {
    static async get(fileName: string) {
      const file = await getR2Client().send(
        new GetObjectCommand({
          Bucket: R2_BUCKET,
          Key: fileName,
        }),
      );
  
      if (!file) {
        throw new Error("not found.");
      }
  
      return file.Body;
    }
  
    static async put(fileName: string, data: Buffer) {
      const signedUrl = await getSignedUrl(
        getR2Client(),
        new PutObjectCommand({
          Bucket: R2_BUCKET,
          Key: fileName,
        }),
        { expiresIn: 60 },
      );
  
      console.log(signedUrl);
  
      await fetch(signedUrl, {
        method: "PUT",
        body: data,
      });
  
      return `Success`;
    }
  }

  const R2_BUCKET = "rep-bucket";
  
  const getR2Client = () => {
    return new S3Client({
        region: "auto",

        endpoint: `https://${sanitizedConfig.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
    
        credentials: {
    
            accessKeyId: sanitizedConfig.R2_ACCESS_KEY,
    
            secretAccessKey: sanitizedConfig.R2_SECRET_KEY,
    
        },
    });
  };
  
  export default class S3 {
    static async get(fileName: string) {
      const file = await getR2Client().send(
        new GetObjectCommand({
          Bucket: R2_BUCKET,
          Key: fileName,
        }),
      );
  
      if (!file) {
        throw new Error("not found.");
      }
  
      return file.Body;
    }
  
    static async put(fileName: string, data: Buffer) {
      const signedUrl = await getSignedUrl(
        getR2Client(),
        new PutObjectCommand({
          Bucket: R2_BUCKET,
          Key: fileName,
        }),
        { expiresIn: 60 },
      );
  
      console.log(signedUrl);
  
      await fetch(signedUrl, {
        method: "PUT",
        body: data,
      });
  
      return `Success`;
    }
  }

i have all the api keys generated with all the permissions and all, anyone could help?

Ddigitalpoint Bandwidth isn't billable (as I'm sure you know), but if you want to see it just ...

M

m0jo•10/10/23, 7:22 AM

is it ok if you can send that plugin? i know bandwidth is nothing but im curious as to how much my guests download

Mm0jo is it ok if you can send that plugin? i know bandwidth is nothing but im curious...

D

digitalpoint•10/10/23, 3:04 PM

It's only for 2 frameworks (WordPress and XenForo). If you site runs WordPress, the R2 functionality is an extension of this plugin: https://wordpress.org/plugins/app-for-cf/

It's probably simpler if you just have a way to run GraphQL queries against your Cloudflare account and I can send you the query you need.

Plugin Directorydigitalpoint

App for Cloudflare®

Utilize Cloudflare for your WordPress site to the maximum extent (caching, settings, rules, analytics, cloud storage, email management, protect admin …

Ddigitalpoint Bandwidth isn't billable (as I'm sure you know), but if you want to see it just ...

I

Isaac McFadyen•10/10/23, 3:13 PM

Is the bandwidth number provided by R2? Just wondering because an object might not be downloaded fully for all requests (i.e. if the request is cancelled or if the client wants partial content)

IIsaac McFadyen Is the bandwidth number provided by R2? Just wondering because an object might n...

D

digitalpoint•10/10/23, 3:20 PM

Honestly, I'm not sure what exactly Cloudflare counts as size as far as reporting. It's using

AccountR2OperationsAdaptiveGroupsSum.responseObjectSize

AccountR2OperationsAdaptiveGroupsSum.responseObjectSize

AccountR2OperationsAdaptiveGroupsSum.responseObjectSize

AccountR2OperationsAdaptiveGroupsSum.responseObjectSize via GraphQL, so I'm just reporting on what Cloudflare gives. What Cloudflare considers "response object size" when factoring in things like canceled requests, I'm not sure (only they would know I suspect).

D

digitalpoint•10/10/23, 3:21 PM

I would think since it's named "response object size", it should factor in partial responses (it's about the response itself, not the object as a whole). But how they handle canceled requests and if they do/don't count the actual bandwidth transfered vs. what was supposed to be sent, I'm not sure

D

digitalpoint•10/10/23, 3:25 PM

I suspect it's just going to be what was supposed to be sent. Things just get complicated if they were trying to handle canceled requests. It would get even more complicated when going the other direction (like what if the client was on a terrible connection and a large number of packets had to be retried?) Getting an "absolute" exact number of bytes transferred in every possible scenario seems like it would be far too much overhead. Like a web server log doesn't tell you how many bytes were received on an HTTP requests, rather how many bytes were intended to be sent.

I

Isaac McFadyen•10/10/23, 3:27 PM

Yeah, that makes sense. At the same time, for cancelled requests in some implementations (i.e. HTTP/1 tcp close, HTTP/2 reset frames) the client tells the server it's cancelled things and no bytes are sent after the server is notified it's been cancelled.

I

Isaac McFadyen•10/10/23, 3:27 PM

I wonder if it would consider that sent or not?

D

digitalpoint•10/10/23, 3:33 PM

It wouldn't be terribly difficult for someone to test... make a bucket with one 100MB object, request it and cancel the request and see what the egress shows. I don't personally care enough, but someone certainly could if they did.

Or ask someone at Cloudflare... maybe they know

I

Isaac McFadyen•10/10/23, 3:42 PM

Yeah I don't really care much either, was just curious.

D

delta•10/10/23, 11:56 PM

Is it a known issue that R2 with custom domain randomly ignoring range header and sending full response?

K

kian•10/11/23, 12:07 AM

If cache is enabled and the file is too big for cache, yes.

D

delta•10/11/23, 12:17 AM

thank you

X

XenoZane•10/11/23, 1:41 AM

Hi, I'm a total noob to web dev and cloud storage, but I want to distribute recordings of my podcast through my website (a static site deployed with cloudflare pages). Would I be able to host these audio files with R2? Is that even a good way to do such a thing?

XXenoZane Hi, I'm a total noob to web dev and cloud storage, but I want to distribute reco...

K

KarewOP•10/11/23, 1:49 AM

Yes and yes, podcasts are a great use-case for R2

X

XenoZane•10/11/23, 1:53 AM

Ok, my next question: when I try to upload the audio files directly from my computer, they always fail. They're all under 300mb. I tried uploading some other smaller files and those worked. Am I doing something wrong or do I need to use Workers to do this?

X

XenoZane•10/11/23, 3:34 AM

Nevermind, it's just a bit finnicky, I think I got it working

X

XenoZane•10/11/23, 4:34 AM

Is there any way I can have my website with cloudflare pages and R2 on the same domain?

If you use a custom domain and set up caching in front of it, you can conserve class B ops, serving

Similar Threads

Similar Threads

Similar Threads