I haven't gotten an answer about this issue, it's eating unnecessary space (cloudflare reports the f

I haven't gotten an answer about this issue, it's eating unnecessary space (cloudflare reports the file is over 400MB when, it is in fact, around 380MB. In this cases, it is basically inflating 20MB+ on each file...

Windows reports the same file as the CI tool, downloading the file directly from R2 also reports the same size, but, the UI, for some reason reports extra space...

KKarbust I haven't gotten an answer about this issue, it's eating unnecessary space (clou...

Erisa•8/9/23, 12:45 PM

Hi, are you sure this isn't the difference between MiB and MB units? Windows shows MB when it actually means MiB, so the other places may be showing real MB causing a small difference.
(MB is 1,000,000 bytes while MiB is 1,048,576 bytes)

nagyt234•8/9/23, 1:42 PM

I would like to use the S3 PHP API to get the results from Cloudflare logpush. I can successfully list the objects with $client->listObjects(...). However if I want to get the content of an object with
$client->getObject([
'Bucket' => self::BUCKET_NAME,
'Key' => $object['Key']
]);
I get as Body an empy GuzzleHttp\Psr7\Stream instance. How can I get the real content of that gzip-ed object?

EErisa Hi, are you sure this isn't the difference between MiB and MB units? Windows sho...

KarbustOP•8/9/23, 2:04 PM

You are correct, didn't think about that, the file is 401229880 bytes, which is 382MiB and 401MB. Maybe cloudflare could maybe also show in MiB, would be great, TeamCity, Windows and browsers use MB for MiB apparently, this way would have some consistency.

Thank you

Ook Is the "/" needed here?

Unsmart•8/9/23, 11:14 PM

Shouldnt be needed no, but generally people like to think of things as folders where a "/" denotes the folder it is in so normally they include that.

Unsmart•8/9/23, 11:16 PM

Yes it will.

Unsmart•8/9/23, 11:17 PM

R2/S3 in general doesnt actually have any concept of folders its all just prefix / delimiters that can be anything you want

heartz•8/10/23, 12:06 AM

Hi, does anyone know if expired R2 tokens just vanish from the dashboard, or are they listed as "expired"? I have a key which stopped working and I am unable to see which Cloudflare account it was located at, maybe it just disappeared from the account.

Hheartz Hi, does anyone know if expired R2 tokens just vanish from the dashboard, or are...

Erisa•8/10/23, 1:23 AM

They vanish, but if you check the account audit log you should be able to see an entry for the expiration (make sure to tick user level events)

Iizdi Hello, what's the best way to connect to a production R2 database from local mac...

Hello, I’m Allie!•8/10/23, 11:46 AM

Try Rclone

Rclone

Rclone syncs your files to cloud storage: Google Drive, S3, Swift, Dropbox, Google Cloud Storage, Azure, Box and many more.

Iizdi Thanks. I have a worker that fails after accessing R2 objects, I suspect this is...

Hello, I’m Allie!•8/10/23, 11:51 AM

Oh, like that. Have you tried logging the error it spits out?

Single•8/10/23, 1:04 PM

Not sure if directly R2 related but how can i disable the cache for R2? Cloudflare is serving files that i deleted from R2 and i assume it is due to the cache settings.

SmallShen•8/10/23, 1:09 PM

If I delete 20k with the

delete

delete

(Is that possible?) in a single worker request, does it count as one fetch request?

Soham•8/10/23, 1:10 PM

I'm pretty sure delete operations are free

SmallShen•8/10/23, 1:10 PM

Sorry I should make it more clear

Soham•8/10/23, 1:10 PM

SSingle Not sure if directly R2 related but how can i disable the cache for R2? Cloudfla...

Karew•8/10/23, 1:38 PM

If you have a custom domain, set up a Cache Rule to "Bypass Cache" for those file paths or whatever applies

Karew•8/10/23, 1:38 PM

But you still may need to clear old cache items if you got them cached. You can purge your whole cache or clear individual items.

SSmallShen If I delete 20k with the `delete`(Is that possible?) in a single worker request,...

Sid•8/10/23, 1:53 PM

Are you referring to the Workers sub-request limit? If so, yeah one fetch counts as one sub request, regardless of what you do in that request.

SSid Are you referring to the Workers sub-request limit? If so, yeah one fetch counts...

SmallShen•8/10/23, 1:54 PM

Thanks!

SnowRunescape•8/10/23, 5:08 PM

who clean all objects of r2? easy?

SnowRunescape•8/10/23, 5:09 PM

without delete bucket

SSnowRunescape who clean all objects of r2? easy?

Hello, I’m Allie!•8/10/23, 5:11 PM

rclone purge r2:/bucket/

rclone purge r2:/bucket/

Hello, I’m Allie!•8/10/23, 5:11 PM

Though I would probably

--dry-run

--dry-run

it first just in case.

KKarew But you still may need to clear old cache items if you got them cached. You can ...

Single•8/10/23, 7:01 PM

Thank you, i did that and it worked. However, i would love to have a cache but it seems like enabling the cache will deny updates of files in general. I guess i didnt find the right settings.

Erisa•8/10/23, 7:24 PM

yes

acubesa•8/11/23, 2:09 AM

i use "auto", works for me

SSingle Thank you, i did that and it worked. However, i would love to have a cache but i...

Karew•8/11/23, 2:24 AM

You have a lot of control over this. When you update files, you could use the Cloudflare API to bust their cache. You can also set shorter cache lifetimes in the Cache Rules. Alternatively, if you have a database or similar and you need the new file to show up right now, store the file update time and use that the bust the cache in the URL. Alternatively again, change the key of the new file and slate the old file for deletion.

acubesa•8/11/23, 3:06 AM

Have been getting 403 [Error] Origin http://localhost:3000 is not allowed by Access-Control-Allow-Origin. Status code: 403

when trying to PUT to a signedUrl generated from the S3 api

I'm building it this way:

const S3 = new S3Client({
region: 'auto',
endpoint:

https://${process.env.CLOUDFLARE_ACCOUNT_ID}.r2.cloudflarestorage.com

https://${process.env.CLOUDFLARE_ACCOUNT_ID}.r2.cloudflarestorage.com

,
credentials: {
accessKeyId: process.env.CLOUDFLARE_ACCESS_KEY_ID!,
secretAccessKey: process.env.CLOUDFLARE_SECRET_ACCESS_KEY!,
},
});

const response = await getSignedUrl(
S3, new PutObjectCommand(
{
Bucket: process.env.CLOUDFLARE_BUCKET,
Key: fileName,
Metadata: {
"x-amz-meta-origin": "http://localhost:3000"
}
}
),
{ expiresIn: 3600 }
)

And this is CORS policy in R2:

[
{
"AllowedOrigins": [
"http://localhost:3000"
],
"AllowedMethods": [
"GET",
"PUT",
"POST",
"HEAD"
],
"AllowedHeaders": [
"*"
]
}
]

SSid Are you actually using a PUT? POST uploads aren’t supported at the moment (but s...

asp•8/11/23, 10:22 AM

Any ETA for pre-signed POST uploads? I'm setting up an application that, unfortunately, does not support regular pre-signed URLs.

Aasp Any ETA for pre-signed POST uploads? I'm setting up an application that, unfortu...

Sid•8/11/23, 3:19 PM

Unfortunately I can't give you an ETA at the moment, but I can look into the status here. It may not be available in the super short-term though.

Iizdi Hi everyone, I'm looking for a case in documentation regarding `list()` method. ...

Sid•8/11/23, 3:20 PM

That isn't possible right now, but you can prefix your object names with a timestamp, and then list them perhaps?

Zonimi•8/11/23, 9:23 PM

Hello, is R2 already stable to use custom domain with LOTS of traffic without having problems with ToS?
I had problems with this a few months ago and was wondering if everything is okay

ZZonimi Hello, is R2 already stable to use custom domain with LOTS of traffic without ha...

Erisa•8/11/23, 9:28 PM

Yes it will be fine: https://blog.cloudflare.com/updated-tos/

The Cloudflare Blog

Goodbye, section 2.8 and hello to Cloudflare’s new terms of service

We’re excited to announce new updates that will modernize our terms of service and hopefully cut down on customer confusion and frustration.

Erisa•8/11/23, 9:28 PM

How many months ago were you remembering?

EErisa Yes it will be fine: https://blog.cloudflare.com/updated-tos/

Zonimi•8/11/23, 9:33 PM

Thanks!

EErisa How many months ago were you remembering?

Zonimi•8/11/23, 9:33 PM

It was at the beginning of the open beta, it should be a year and a few months in fact

Erisa•8/11/23, 9:34 PM

Quite a while! Yes, there was some issues for a while with incorrect ToS flags for R2 usage. Those are now resolved.

I haven't gotten an answer about this issue, it's eating unnecessary space (cloudflare reports the f

Similar Threads

Similar Threads

Similar Threads