I eventually ditched B2 due to having to retry about 1% of all uploads (they failed or gave me crypt

I eventually ditched B2 due to having to retry about 1% of all uploads (they failed or gave me cryptic Backblaze-only errors). The speed was fine when it worked tho

KKarew I eventually ditched B2 due to having to retry about 1% of all uploads (they fai...

dave•12/28/23, 9:42 PM

so I've heard that before, but somehow I don't seem to have ever ran into it with aws4fetch.

KKarew I eventually ditched B2 due to having to retry about 1% of all uploads (they fai...

dave•12/28/23, 9:43 PM

was your ~1% error rate with a Worker?

KarewOP•12/28/23, 9:43 PM

No, I used the Backblaze API and the S3 interface and a regular Cloudflare CDN endpoint

dave•12/28/23, 9:44 PM

How were you using Cloudflare with B2 without a Worker?

KarewOP•12/28/23, 9:44 PM

It was only on uploads, and it was frustratingly definitely some kind of Backblaze problem (like the error message would say I ran out of "tomes" and stuff like that, which are their low-level storage system objects)

dave•12/28/23, 9:44 PM

oouf.

KarewOP•12/28/23, 9:44 PM

You can just CNAME a Backblaze public bucket with Cloudflare

KKarew It was only on uploads, and it was frustratingly definitely some kind of Backbla...

dave•12/28/23, 9:48 PM

https://www.reddit.com/r/backblaze/comments/186k6um/comment/kbcdepb/?utm_source=share&utm_medium=web2x&context=3

KarewOP•12/28/23, 9:49 PM

Yep, that's what we did. Our upload code would retry like 4 times if it got a Backblaze error before actually giving up. It's p. unacceptable.

dave•12/28/23, 9:50 PM

How big are your files on average? I'm wondering why I haven't ran into it yet.

KarewOP•12/28/23, 9:50 PM

Like 500kb-1mb images, not very frequent

KarewOP•12/28/23, 9:52 PM

You might also see:

"too busy"
"not lucky"
"no master pod"
"no tomes available"
"internal_error"
"incident id"

"too busy"
"not lucky"
"no master pod"
"no tomes available"
"internal_error"
"incident id"

KarewOP•12/28/23, 9:52 PM

Which are all this tome system choking

Chaika•12/28/23, 9:53 PM

"retry 9 more times"

Ddave How big are your files on average? I'm wondering why I haven't ran into it yet.

Chaika•12/28/23, 9:56 PM

could be that your client automatically retries, some of the s3 libraries do I think on 5xx errors

CChaika could be that your client automatically retries, some of the s3 libraries do I t...

dave•12/28/23, 9:58 PM

I usually have retries set to 0 in aws4fetch

Chaika•12/28/23, 9:58 PM

living life on the edge

Chaika•12/28/23, 10:00 PM

B2 has a article here: https://www.backblaze.com/blog/b2-503-500-server-error/
I guess they use 500/503 errors to switch between vaults and such, no LB in front, it gets routed to a vault and if it fails because it's full the request dies. Kind of interesting

As we’ve described in our developer docs, the best decision is to write your integration in a way that it retries in the event of a 500 or 503.

Chaika•12/28/23, 10:02 PM

I guess that's not for S3 though although it can still run into similiar errors

Unsmart•12/28/23, 10:02 PM

Backblaze is the king of being slow and constant errors

Chaika•12/28/23, 10:03 PM

uploads are free because half the time they fail /s

Unsmart•12/28/23, 10:03 PM

real

Chaika•12/28/23, 10:03 PM

really cheap though

Chaika•12/28/23, 10:04 PM

I've wanted to try B2 before but I don't have large enough needs and we get R2 free

Unsmart•12/28/23, 10:12 PM

I used B2 for a little bit for storing on demand video clips from live streams and it was super bad switched to stackpath which was just resale of wasabi or whatever

Unsmart•12/28/23, 10:13 PM

But very good and fast

James•12/28/23, 10:41 PM

We're big B2 users and I'm a fan of their service. You do have to design with 5xxs by design though when using their native APIs at least: https://www.backblaze.com/docs/cloud-storage-integration-checklist

Integration Checklist

Tools that integrate with Backblaze B2 Cloud Storage and are featured on the Backblaze B2 integrations page must follow specific guidelines.

James•12/28/23, 10:41 PM

If you use their S3 compatible API, it's all transparently handled for end-users nowadays, which is very nice

James•12/28/23, 10:42 PM

or at least, in theory it should, lol

JJames If you use their S3 compatible API, it's all transparently handled for end-users...

dave•12/28/23, 10:44 PM

ahhh, that makes sense, I've always be using their S3 API. Never wanted to touch a native cloud storage API

James•12/28/23, 10:45 PM

unfortunately we were about ~18 months early with our integration so the S3 compat API didn't exist

dave•12/28/23, 10:56 PM

Semi-related question, is there any perf difference between virtual and path-style S3 APIs?

dave•12/28/23, 10:56 PM

in practice it looks like both point to the same IP anyway

James•12/28/23, 10:59 PM

I think the path-style is old and planned to be deprecated

James•12/28/23, 10:59 PM

but otherwise I don't think so

James•12/28/23, 11:00 PM

yeah this: https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/

Chaika•12/28/23, 11:01 PM

If you're talking R2 S3-compat specifically I really doubt there would be any diff, already a compat layer on top

JJames I think the path-style is old and planned to be deprecated

dave•12/28/23, 11:02 PM

is it actually gonna be deprecated though?

JJames yeah this: https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-...

Chaika•12/28/23, 11:02 PM

they entirely stopped that though as noted at the top, and said they would provide at least a years heads up before doing it again, so it looks like it'll still be around for a while

James•12/28/23, 11:03 PM

yeah fair, maybe one day

HandyMan•12/28/23, 11:49 PM

Hey all, does anyone have experience in setting up DDNS for bypass rule in Access policy (zero trust)?

I used to have static IP from my ISP and I just bypassed access via that IP but I don't anymore. Wondering if anyone has experience in using DDNS for the bypass rule?

HHandyMan Hey all, does anyone have experience in setting up DDNS for bypass rule in Acces...

Cyb3r-Jak3•12/28/23, 11:56 PM

You’d have to use the API to update the bypass IP but it is doable.

CCyb3r-Jak3 You’d have to use the API to update the bypass IP but it is doable.

HandyMan•12/28/23, 11:57 PM

Thanks. I will checkout the update access policy call.
https://developers.cloudflare.com/api/operations/access-policies-update-an-access-policy

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

CCyb3r-Jak3 You’d have to use the API to update the bypass IP but it is doable.

HandyMan•12/29/23, 1:00 AM

That was easy, thanks for the suggestion

chientrm.com•12/29/23, 6:21 AM

Does Services Binding work for wranglerwrangler local mode?

I get the error: Cannot read properties of undefined (reading 'fetch')Cannot read properties of undefined (reading 'fetch')

chientrm.com•12/29/23, 6:27 AM

oh, local worker can only bind to another local worker

nuwansa.•12/29/23, 8:37 AM

Hi, i'm testing with R2 and aws lambda . aws lambda i hosted on us-east-1 regon. get object latency is very high. around 400 - 500 milliseconds. anyone have tips to improve the peroformance?. thanks

Pprender browser rendering api seems in waitlist? any way to pay for it to get it quickly...

Vero•12/29/23, 10:59 AM

You don't have to pay, you can ask for access in the thread browser-rendering-apiBrowser Rendering API Beta Access

oughex•12/29/23, 1:41 PM

Hi, I have a fairly easy problem to solve. Please could someone help me with that?
I've already logged using wrangler login, but as soon as I start I get the error as following


✘ [ERROR] A request to the Cloudflare API (/accounts/719e33....9acb3c939/d1/database) failed.
  Authentication error [code: 10000]
Getting User settings...
👋 You are logged in with an OAuth Token, associated with the email 
...


✘ [ERROR] A request to the Cloudflare API (/accounts/719e33....9acb3c939/d1/database) failed.
  Authentication error [code: 10000]
Getting User settings...
👋 You are logged in with an OAuth Token, associated with the email 
...

I eventually ditched B2 due to having to retry about 1% of all uploads (they failed or gave me crypt

Similar Threads

Similar Threads

Similar Threads