Not `R2Object[]`

G

Gordon•11/2/23, 7:40 AM

Hi everyone, I have a question about R2 and the free egress fee.

If I use R2 to store audio files with public access enabled, would the transfer flow be from R2 to the client? Additionally, in this setup, would I only be charged for storage and the number of requests, with egress to the client being free of cost?

K

kianOP•11/2/23, 7:43 AM

Yes and yes

Kkian Not `R2Object[]`

B

bz_008•11/2/23, 7:59 AM

Thanks for the help, I checked the link, it says it's "An object containing an R2Object array, returned by BUCKET_BINDING.list()." interesting..
So, how to get the R2Object array out?

Kkian Yes and yes

G

Gordon•11/2/23, 8:03 AM

Thanks for the reply. I have a follow-up question.

From the document: https://developers.cloudflare.com/r2/buckets/public-buckets/
It states that to enable public access for R2, I must set up a custom domain, as the r2.dev domain is for development purposes only. However, it appears that after setting up a custom domain, the flow will be: R2 -> CDN -> client. I'm not sure if I've misunderstood anything. Will I be charged for the data transfer from the CDN to the client?

Public buckets · Cloudflare R2 docs

Public Bucket is a feature that allows users to expose the contents of their R2 buckets directly to the Internet. By default, buckets are never …

K

kianOP•11/2/23, 8:03 AM

Nope

K

kianOP•11/2/23, 8:03 AM

The CDN will lower your bill since cached requests don't incur R2 operations

Original message was deleted

B

bz_008•11/2/23, 8:17 AM

Cool, that works.. on R2Objets, thank you!

R

ray.dev•11/2/23, 8:23 AM

Hi kian,
I have the same question today. I am only using R2 without Cloudflare CDN. When I am using a custom domain, the Cloudflare will enable free CDN for me regardless of egress traffic, does that make sense?

If it is the case, say I have 1 pb egress traffic per month, will the Cloudflare charge us by egress traffic rather than orignal plan

R

ray.dev•11/2/23, 8:25 AM

I am still assessing whether to use R2

R

ray.dev•11/2/23, 8:38 AM

Thanks! Do you know any cases that people got charged after exceeding some amount of traffic like 500TB

R

ray.dev•11/2/23, 8:38 AM

or something like that

K

kianOP•11/2/23, 7:04 PM

Are you sure the response isn't cached?

M

Muhammad Rafai•11/2/23, 7:04 PM

hello guys. i was succesfully uploaded a file into R2 bucket from express node JS. but, how i can get the url of the uploaded file?

Rray.dev Hi kian, I have the same question today. I am only using R2 without Cloudflare C...

U

Unsmart•11/2/23, 7:04 PM

You can use an unlimited amount of bandwidth for free without upgrading. Do note that you can get an enterprise agreement where you can negotiate pricing details and normally all bandwidth is charged there but other costs can be decreased by usage amounts.

K

kianOP•11/2/23, 7:05 PM

Do you have a URL that should be showing the headers?

MMuhammad Rafai hello guys. i was succesfully uploaded a file into R2 bucket from express node J...

M

Muhammad Rafai•11/2/23, 7:06 PM

this is my code.

const express = require("express")
const multer = require("multer")
const cors = require("cors")
const {S3Client, PutObjectCommand} = require("@aws-sdk/client-s3")
const dotenv = require("dotenv")

const app = express()
const port = 3000

// middleware
app.use(cors())
dotenv.config()

const storage = multer.memoryStorage()
const upload = multer({
storage,
})

app.get("/", (req, res) => {
res.send("Hello World!")
})

app.post("/upload", upload.single("file"), async (req, res) => {
// console.log(req.file)
// res.send("File Upload")
const S3 = new S3Client({
region: "auto",
endpoint: process.env.ENDPOINT,
credentials: {
accessKeyId: process.env.R2_ACCESS_KEY_ID,
secretAccessKey: process.env.R2_SECRET_ACCESS_KEY,
}
})

await S3.send(
new PutObjectCommand({
Body: req.file.buffer,
Bucket: "fullstack-team",
Key: req.file.originalname,
ContentType: req.file.mimetype
})
)
res.send("File Upload")
})

app.listen(port, () => {
console.log(

Server is lisening on port ${port}

Server is lisening on port ${port}

)
})

EE One hour later still not seeing the `Access-Control-Expose-Headers` header

K

kev-ac•11/2/23, 8:14 PM

As the control plane for many services is degraded right now its quite possible that your changes haven't been applied yet.

J

JesterIruka•11/2/23, 8:55 PM

All of sudden the content-length header is missing too

J

JesterIruka•11/2/23, 8:55 PM

I used to read it on my .js assets (For node js applications)

J

JesterIruka•11/2/23, 8:59 PM

URL for replication =>

https://storage.hydrus.gg/minimal.js?v=5

https://storage.hydrus.gg/minimal.js?v=5

J

JesterIruka•11/2/23, 8:59 PM

HTTP/1.1 200 OK
Date: Thu, 02 Nov 2023 20:58:43 GMT
Content-Type: application/x-javascript
Transfer-Encoding: chunked
Connection: keep-alive
ETag: W/"6b711a097259b80325f9c23545d4f80e"
Last-Modified: Thu, 02 Nov 2023 20:58:20 GMT
Vary: Accept-Encoding
Cache-Control: max-age=14400
CF-Cache-Status: MISS
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=uWU21R8%2BNcWWMMV0FzTKyiQTRXx704CsrYfSrighWma4%2FTPHAVU9U73z%2FlhCxedfVYe6mC4b28ADblbreUfZ7j1CVK8Wt1m6BHgX%2BTtBgL0TVyciumzmQuxNDLU8kvJU6Pz1CdhxbUO8mXRieofFlQ%3D%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Access-Control-Allow-Origin: *
Server: cloudflare
CF-RAY: 81ff4c11da9f028f-GRU
Content-Encoding: br
alt-svc: h3=":443"; ma=86400

HTTP/1.1 200 OK
Date: Thu, 02 Nov 2023 20:58:43 GMT
Content-Type: application/x-javascript
Transfer-Encoding: chunked
Connection: keep-alive
ETag: W/"6b711a097259b80325f9c23545d4f80e"
Last-Modified: Thu, 02 Nov 2023 20:58:20 GMT
Vary: Accept-Encoding
Cache-Control: max-age=14400
CF-Cache-Status: MISS
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=uWU21R8%2BNcWWMMV0FzTKyiQTRXx704CsrYfSrighWma4%2FTPHAVU9U73z%2FlhCxedfVYe6mC4b28ADblbreUfZ7j1CVK8Wt1m6BHgX%2BTtBgL0TVyciumzmQuxNDLU8kvJU6Pz1CdhxbUO8mXRieofFlQ%3D%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Access-Control-Allow-Origin: *
Server: cloudflare
CF-RAY: 81ff4c11da9f028f-GRU
Content-Encoding: br
alt-svc: h3=":443"; ma=86400

HTTP/1.1 200 OK
Date: Thu, 02 Nov 2023 20:58:43 GMT
Content-Type: application/x-javascript
Transfer-Encoding: chunked
Connection: keep-alive
ETag: W/"6b711a097259b80325f9c23545d4f80e"
Last-Modified: Thu, 02 Nov 2023 20:58:20 GMT
Vary: Accept-Encoding
Cache-Control: max-age=14400
CF-Cache-Status: MISS
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=uWU21R8%2BNcWWMMV0FzTKyiQTRXx704CsrYfSrighWma4%2FTPHAVU9U73z%2FlhCxedfVYe6mC4b28ADblbreUfZ7j1CVK8Wt1m6BHgX%2BTtBgL0TVyciumzmQuxNDLU8kvJU6Pz1CdhxbUO8mXRieofFlQ%3D%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Access-Control-Allow-Origin: *
Server: cloudflare
CF-RAY: 81ff4c11da9f028f-GRU
Content-Encoding: br
alt-svc: h3=":443"; ma=86400

HTTP/1.1 200 OK
Date: Thu, 02 Nov 2023 20:58:43 GMT
Content-Type: application/x-javascript
Transfer-Encoding: chunked
Connection: keep-alive
ETag: W/"6b711a097259b80325f9c23545d4f80e"
Last-Modified: Thu, 02 Nov 2023 20:58:20 GMT
Vary: Accept-Encoding
Cache-Control: max-age=14400
CF-Cache-Status: MISS
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=uWU21R8%2BNcWWMMV0FzTKyiQTRXx704CsrYfSrighWma4%2FTPHAVU9U73z%2FlhCxedfVYe6mC4b28ADblbreUfZ7j1CVK8Wt1m6BHgX%2BTtBgL0TVyciumzmQuxNDLU8kvJU6Pz1CdhxbUO8mXRieofFlQ%3D%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Access-Control-Allow-Origin: *
Server: cloudflare
CF-RAY: 81ff4c11da9f028f-GRU
Content-Encoding: br
alt-svc: h3=":443"; ma=86400

J

JesterIruka•11/2/23, 9:01 PM

I mean, now that transfer-encoding is chunked, make a lot of sense, but why it wasn't earlier?

L

lordnikon•11/2/23, 9:38 PM

Anyone using cyberduck with R2?

L

lordnikon•11/2/23, 9:38 PM

just want to confirm it is even working before I beat my head into a wall

L

lordnikon•11/2/23, 9:39 PM

r2

L

lordnikon•11/2/23, 9:43 PM

also, changing a tokens expirey from a date to forver appears to be a bug

L

lordnikon•11/2/23, 9:43 PM

and it never sets, stuck at 1 year

L

lordnikon•11/2/23, 10:09 PM

forever is an option

L

lordnikon•11/2/23, 10:09 PM

and the end date is blank by default, but once you set it, you cant go back

P

polymo1•11/3/23, 1:53 AM

r2 returning 403 forbidden for anyone else

P

polymo1•11/3/23, 2:05 AM

Ppolymo1 Click to see attachment

K

Karew•11/3/23, 4:25 AM

Did your token expire?

I

indicozy•11/3/23, 5:51 AM

I'm new on working R2 and S3, it shows that I am Unauthorized. Is it because the cloudflare having issues or am I doing something wrong?

I

indicozy•11/3/23, 7:45 AM

I don't understand why I cannot upload to R2. My code is:

// getS3Client.ts
import { S3Client } from "@aws-sdk/client-s3";
import { getDotenv } from "./getEnv";

let s3Client: S3Client | undefined = undefined;

const env = getDotenv();

export const getS3Client: () => S3Client = () => {
  console.log(env);
  if (s3Client === undefined) {
    s3Client = new S3Client({
      region: "auto",
      endpoint: `https://${env.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
      credentials: {
        accessKeyId: env.R2_ACCESS_KEY_ID,
        secretAccessKey: env.R2_SECRET_ACCESS_KEY,
      },
    });
    return s3Client;
  }
  return s3Client;
};

// getS3Client.ts
import { S3Client } from "@aws-sdk/client-s3";
import { getDotenv } from "./getEnv";

let s3Client: S3Client | undefined = undefined;

const env = getDotenv();

export const getS3Client: () => S3Client = () => {
  console.log(env);
  if (s3Client === undefined) {
    s3Client = new S3Client({
      region: "auto",
      endpoint: `https://${env.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
      credentials: {
        accessKeyId: env.R2_ACCESS_KEY_ID,
        secretAccessKey: env.R2_SECRET_ACCESS_KEY,
      },
    });
    return s3Client;
  }
  return s3Client;
};

// getS3Client.ts
import { S3Client } from "@aws-sdk/client-s3";
import { getDotenv } from "./getEnv";

let s3Client: S3Client | undefined = undefined;

const env = getDotenv();

export const getS3Client: () => S3Client = () => {
  console.log(env);
  if (s3Client === undefined) {
    s3Client = new S3Client({
      region: "auto",
      endpoint: `https://${env.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
      credentials: {
        accessKeyId: env.R2_ACCESS_KEY_ID,
        secretAccessKey: env.R2_SECRET_ACCESS_KEY,
      },
    });
    return s3Client;
  }
  return s3Client;
};

// getS3Client.ts
import { S3Client } from "@aws-sdk/client-s3";
import { getDotenv } from "./getEnv";

let s3Client: S3Client | undefined = undefined;

const env = getDotenv();

export const getS3Client: () => S3Client = () => {
  console.log(env);
  if (s3Client === undefined) {
    s3Client = new S3Client({
      region: "auto",
      endpoint: `https://${env.R2_ACCOUNT_ID}.r2.cloudflarestorage.com`,
      credentials: {
        accessKeyId: env.R2_ACCESS_KEY_ID,
        secretAccessKey: env.R2_SECRET_ACCESS_KEY,
      },
    });
    return s3Client;
  }
  return s3Client;
};

const s3Client = getS3Client();
const env = getDotenv();

const resizeImage = async (str: string) => {
  // TODO:
  await sharp(str).webp({ quality: 75 }).toBuffer();
};

export const uploadDirectlyOne = async (path: string, body: string) => {
  // TODO: test it
  console.log("uploading:", path);
  const buf = Buffer.from(body, "utf8");

  const command = new PutObjectCommand({
    Bucket: env.S3_BUCKET,
    Key: path,
    Body: buf,
  });
  try {
    const response = await s3Client.send(command);
    console.log("uploaded:", path);
  } catch (err) {
    console.error(err);
    throw err;
  }
};

const s3Client = getS3Client();
const env = getDotenv();

const resizeImage = async (str: string) => {
  // TODO:
  await sharp(str).webp({ quality: 75 }).toBuffer();
};

export const uploadDirectlyOne = async (path: string, body: string) => {
  // TODO: test it
  console.log("uploading:", path);
  const buf = Buffer.from(body, "utf8");

  const command = new PutObjectCommand({
    Bucket: env.S3_BUCKET,
    Key: path,
    Body: buf,
  });
  try {
    const response = await s3Client.send(command);
    console.log("uploaded:", path);
  } catch (err) {
    console.error(err);
    throw err;
  }
};

const s3Client = getS3Client();
const env = getDotenv();

const resizeImage = async (str: string) => {
  // TODO:
  await sharp(str).webp({ quality: 75 }).toBuffer();
};

export const uploadDirectlyOne = async (path: string, body: string) => {
  // TODO: test it
  console.log("uploading:", path);
  const buf = Buffer.from(body, "utf8");

  const command = new PutObjectCommand({
    Bucket: env.S3_BUCKET,
    Key: path,
    Body: buf,
  });
  try {
    const response = await s3Client.send(command);
    console.log("uploaded:", path);
  } catch (err) {
    console.error(err);
    throw err;
  }
};

const s3Client = getS3Client();
const env = getDotenv();

const resizeImage = async (str: string) => {
  // TODO:
  await sharp(str).webp({ quality: 75 }).toBuffer();
};

export const uploadDirectlyOne = async (path: string, body: string) => {
  // TODO: test it
  console.log("uploading:", path);
  const buf = Buffer.from(body, "utf8");

  const command = new PutObjectCommand({
    Bucket: env.S3_BUCKET,
    Key: path,
    Body: buf,
  });
  try {
    const response = await s3Client.send(command);
    console.log("uploaded:", path);
  } catch (err) {
    console.error(err);
    throw err;
  }
};

V

vvo•11/3/23, 11:30 AM

Hey there, we're unable to assign custom domains to R2 buckets. Where is this situation this reflected on your status pages? Thanks.

K

Kirmola•11/3/23, 12:52 PM

Hi. for some reason, my credentials are being reported as unauthroised by boto.

def saveAndPushToR2(self, filename):
        # Use self.engine
        currentDateFileFormat = datetime.now().strftime("%d_%m_%Y")
        # filename = f"{category}_{currentDateFileFormat}.json"
        s3 = boto3.resource('s3',
                            endpoint_url=f'https://{environ.get("ACCOUNT_ID")}.r2.cloudflarestorage.com',
                            aws_access_key_id=environ.get("ACCESS_KEY_ID"),
                            aws_secret_access_key=environ.get(
                                "SECRET_ACCESS_KEY")
                            )
        with open(filename, "r"):
            # s3.Bucket(environ["BUCKET_NAME"]).put_object(
            #     Key=filename, Body=f, ContentType="application/json"
            # )
            s3.Bucket(environ["BUCKET_NAME"]).upload_file(
                filename, filename, ExtraArgs={'ContentType': "application/json"}
            )
            logging.info(
                f"{filename} uploaded successfully to Cloudflare R2.").

def saveAndPushToR2(self, filename):
        # Use self.engine
        currentDateFileFormat = datetime.now().strftime("%d_%m_%Y")
        # filename = f"{category}_{currentDateFileFormat}.json"
        s3 = boto3.resource('s3',
                            endpoint_url=f'https://{environ.get("ACCOUNT_ID")}.r2.cloudflarestorage.com',
                            aws_access_key_id=environ.get("ACCESS_KEY_ID"),
                            aws_secret_access_key=environ.get(
                                "SECRET_ACCESS_KEY")
                            )
        with open(filename, "r"):
            # s3.Bucket(environ["BUCKET_NAME"]).put_object(
            #     Key=filename, Body=f, ContentType="application/json"
            # )
            s3.Bucket(environ["BUCKET_NAME"]).upload_file(
                filename, filename, ExtraArgs={'ContentType': "application/json"}
            )
            logging.info(
                f"{filename} uploaded successfully to Cloudflare R2.").

def saveAndPushToR2(self, filename):
        # Use self.engine
        currentDateFileFormat = datetime.now().strftime("%d_%m_%Y")
        # filename = f"{category}_{currentDateFileFormat}.json"
        s3 = boto3.resource('s3',
                            endpoint_url=f'https://{environ.get("ACCOUNT_ID")}.r2.cloudflarestorage.com',
                            aws_access_key_id=environ.get("ACCESS_KEY_ID"),
                            aws_secret_access_key=environ.get(
                                "SECRET_ACCESS_KEY")
                            )
        with open(filename, "r"):
            # s3.Bucket(environ["BUCKET_NAME"]).put_object(
            #     Key=filename, Body=f, ContentType="application/json"
            # )
            s3.Bucket(environ["BUCKET_NAME"]).upload_file(
                filename, filename, ExtraArgs={'ContentType': "application/json"}
            )
            logging.info(
                f"{filename} uploaded successfully to Cloudflare R2.").

def saveAndPushToR2(self, filename):
        # Use self.engine
        currentDateFileFormat = datetime.now().strftime("%d_%m_%Y")
        # filename = f"{category}_{currentDateFileFormat}.json"
        s3 = boto3.resource('s3',
                            endpoint_url=f'https://{environ.get("ACCOUNT_ID")}.r2.cloudflarestorage.com',
                            aws_access_key_id=environ.get("ACCESS_KEY_ID"),
                            aws_secret_access_key=environ.get(
                                "SECRET_ACCESS_KEY")
                            )
        with open(filename, "r"):
            # s3.Bucket(environ["BUCKET_NAME"]).put_object(
            #     Key=filename, Body=f, ContentType="application/json"
            # )
            s3.Bucket(environ["BUCKET_NAME"]).upload_file(
                filename, filename, ExtraArgs={'ContentType': "application/json"}
            )
            logging.info(
                f"{filename} uploaded successfully to Cloudflare R2.").

K

Kirmola•11/3/23, 12:53 PM

 raise S3UploadFailedError(
boto3.exceptions.S3UploadFailedError: Failed to upload prepositions_03_11_2023.json to production/prepositions_03_11_2023.json: An error occurred (Unauthorized) when calling the PutObject operation: Unauthorized
2023-11-03 18:17:55 [scrapy.extensions.feedexport] INFO: Stored json feed (49 items) in: output.json

 raise S3UploadFailedError(
boto3.exceptions.S3UploadFailedError: Failed to upload prepositions_03_11_2023.json to production/prepositions_03_11_2023.json: An error occurred (Unauthorized) when calling the PutObject operation: Unauthorized
2023-11-03 18:17:55 [scrapy.extensions.feedexport] INFO: Stored json feed (49 items) in: output.json

 raise S3UploadFailedError(
boto3.exceptions.S3UploadFailedError: Failed to upload prepositions_03_11_2023.json to production/prepositions_03_11_2023.json: An error occurred (Unauthorized) when calling the PutObject operation: Unauthorized
2023-11-03 18:17:55 [scrapy.extensions.feedexport] INFO: Stored json feed (49 items) in: output.json

 raise S3UploadFailedError(
boto3.exceptions.S3UploadFailedError: Failed to upload prepositions_03_11_2023.json to production/prepositions_03_11_2023.json: An error occurred (Unauthorized) when calling the PutObject operation: Unauthorized
2023-11-03 18:17:55 [scrapy.extensions.feedexport] INFO: Stored json feed (49 items) in: output.json

this is error

K

Kirmola•11/3/23, 12:54 PM

my code was working perfectly until today, not sure what happened

E

EchoSevenTz•11/3/23, 4:02 PM

Hi,
if I want to access R2 bucket that has custom domain registered for example myr2.com and want to access it from mydomain.com. I set https://mydomain.com in allowed origins in CORS control panel, but still get " Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://my2r.com/Barutana_Model_700k.glb. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing). Status code: 200.

Web Hosting, Domain Name Registration - MyDomain.com

Small business web hosting offering additional business services such as: domain name registrations, email accounts, web services, and various small business solutions.

Similar Threads

Similar Threads

Similar Threads