Yes. The 5GB limit is the max you can proxy through cloudflare on ent

S

Shubham•1/22/24, 2:58 PM

Exactly, before that we were streaming data with 7-8 GB

S

Shubham•1/22/24, 2:59 PM

If this request can be done for increase we will be happy otherwise we need to involve many stakeholders. But Thank you anyways

U

UnsmartOP•1/22/24, 2:59 PM

5GB is the absolute max you have to use multipart

UUnsmart Yes. The 5GB limit is the max you can proxy through cloudflare on ent

E

Erisa•1/22/24, 3:22 PM

Proxied upload can be increased above 5GB on Ent (and the max without contacting support/success is 500MB), but you start running into tradeoffs and needing to disable body buffering when you get that high so its usually not worth it.

S

Scruffy•1/22/24, 5:36 PM

I've got a corrupted file upload issue. On the web UI the files are listed with their respective upload dates and sizes. When I open a specific object in the web UI it shows as having 0 Bytes in size and any attempt to access it via the public URL returns a 404

S

Scruffy•1/22/24, 5:39 PM

S

Scruffy•1/22/24, 5:40 PM

To be clear my bucket does have public access enabled and it works for all my other objects

SScruffy I've got a corrupted file upload issue. On the web UI the files are listed with ...

M

Matt Silverlock•1/22/24, 5:42 PM

Can you please open a Support ticket, including the account ID and bucket ID? That's the best place for us to investigate this — any suggestion of "corruption" is taken very seriously

MMatt Silverlock Can you please open a Support ticket, including the account ID and bucket ID? Th...

S

Scruffy•1/22/24, 5:42 PM

Sure i dont actually know how to open a support ticket

U

UnsmartOP•1/22/24, 5:42 PM

The issue is the hashtag... when accessing by url you need to percent encode

%23

%23

U

UnsmartOP•1/22/24, 5:43 PM

https://support.google.com/richmedia/answer/190941?hl=en

UUnsmart <https://support.google.com/richmedia/answer/190941?hl=en>

S

Scruffy•1/22/24, 5:43 PM

yep thats correct

S

Scruffy•1/22/24, 5:43 PM

ty

S

Scruffy•1/22/24, 5:44 PM

would suggest fixing it on the cloudflare UI

S

Scruffy•1/22/24, 5:44 PM

V

Venkat Dinavahi•1/22/24, 6:26 PM

Hi has anyone succeeded in uploading to cloudflare R2 with a presigned url?

V

Venkat Dinavahi•1/22/24, 6:26 PM

If so, are there any working examples online for this? i've been struggling with a CORS issue. Seems to work for GET requests but PUT requests don't work.

V

Venkat Dinavahi•1/22/24, 6:28 PM

Or my ultimate goal is to be able to support uploads of up to 2GB but it appears that uploading to my cloudflare worker has request body limits.

SShubham We are facing some issue while uploading file larger than 5Gb in the R2. We have...

V

Venkat Dinavahi•1/22/24, 6:30 PM

How are you able to upload files even 2GB? I'm not able to figure out how to do this.

H

Hexstream•1/22/24, 7:47 PM

Hello, I'm getting unexpected errors while trying to retrieve stuff from R2, bizarrely cloudflarestatus does not seem to know about this at all...
Even downloading files directly from the R2 dashboard doesn't work.

H

Hexstream•1/22/24, 7:49 PM

get: The specified method is not allowed against this resource (10060)

H

Hexstream•1/22/24, 7:49 PM

Discord messages are too short for the full stacktrace, unfortunately.

H

Hexstream•1/22/24, 7:51 PM

Hum, I think redirects may be handled more strictly than before?

https://developers.cloudflare.com/support/troubleshooting/cloudflare-errors/troubleshooting-cloudflare-10xxx-errors/#error-10060-missing-scheme-in-redirect-target-url

H

Hexstream•1/22/24, 7:53 PM

Stuff is normally accessed through a trivial worker that just retrieves the R2 stuff, but if I try to download stuff from the dashboard that doesn't work either.

H

Hexstream•1/22/24, 7:54 PM

I could post a worker URL, but that could be seen as a bit spammy I guess...

U

UnsmartOP•1/22/24, 7:54 PM

You can post the URL. Everything works fine for me so seems its most likely an issue on your end so code would be helpful also

H

Hexstream•1/22/24, 7:55 PM

https://ponies.hexstream.net/videos/brony-originals/misc/buttons-adventures-episode-pilot.mp4 would be an example URL.

U

UnsmartOP•1/22/24, 7:55 PM

and whats the code for how you are getting the object?

H

Hexstream•1/22/24, 7:55 PM

I haven't changed anything related to that recently, so this must be a new incompatibility of some sort.

H

Hexstream•1/22/24, 7:57 PM

Yeah I'll fetch the code, I haven't touched that one in a while...

H

Hexstream•1/22/24, 7:58 PM

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

H

Hexstream•1/22/24, 8:01 PM

But logically this is probably not related to a problem with that code, since it hasn't changed in ages and I can't download the file directly from Cloudflare's R2 dashboard either, unless that's an unrelated issue...

U

UnsmartOP•1/22/24, 8:02 PM

whats the error you get when trying to download from the dashboard?

H

Hexstream•1/22/24, 8:03 PM

It just says "Download failed. Please try again.".

U

UnsmartOP•1/22/24, 8:04 PM

Not sure I would open a support ticket

H

Hexstream•1/22/24, 8:05 PM

I'm on the free plan. The support wizard told me to post about this here.

H

Hexstream•1/22/24, 8:06 PM

H

Hexstream•1/22/24, 8:07 PM

Perhaps I could try posting in the community support forum thingy...

H

Hexstream•1/22/24, 8:07 PM

I would be pretty shocked if I'm the only one affected by this...

U

UnsmartOP•1/22/24, 8:09 PM

I'd find it very likely you are the only one otherwise I would expect a lot of reports here and on the community yet there are none and things work just fine for me on all of my buckets. Probably some configuration option set to a bad value but not sure

H

Hexstream•1/22/24, 8:10 PM

That's the weird thing, nothing changed, basically...
These errors just started popping up out of nowhere.

U

UnsmartOP•1/22/24, 8:10 PM

R2 employees do read this channel from time to time so they would best be able to assist when they see it

H

Hexstream•1/22/24, 8:12 PM

Yeah, normally I would use my tmp bucket but that won't work right now... I'll upload the full stacktrace somewhere.
But again, this also fails from the dashboard directly, so clearly unrelated to the worker trying to fetch.
(Unless the dashboard download error is unrelated...)

H

Hexstream•1/22/24, 8:13 PM

No, since yesterday at least.

H

Hexstream•1/22/24, 8:15 PM

Great question! I should try that, but I almost never use rclone so figuring out the incantation could take a while.

H

Hexstream•1/22/24, 8:16 PM

Thanks for the help btw, usually my strategy is waiting for this kind of stuff to get fixed, but seems like that might not happen on its own this time...

UUnsmart I'd find it very likely you are the only one otherwise I would expect a lot of r...

C

Chaika•1/22/24, 8:21 PM

I mean there was this just recently: https://www.cloudflarestatus.com/incidents/cxyckjngxnqk
strange though

R2 Availability Impact on object retrieval

U

UnsmartOP•1/22/24, 8:21 PM

Pretty different issue though

C

Chaika•1/22/24, 8:22 PM

Tailing the worker to see the exception its throwing might also not be a bad idea, or using dev tools (ctrl+shift+i -> Network tab) and seeing the exact response trying to download the item from the bucket gives you other then failure

CChaika I mean there was this just recently: https://www.cloudflarestatus.com/incidents/...

H

Hexstream•1/22/24, 8:27 PM

Yeah, I saw that, I would have thought that could be related, but this is apparently resolved...

Yes. The 5GB limit is the max you can proxy through cloudflare on ent

Similar Threads

Similar Threads

Similar Threads