Mmm I'm only passing the request.body to the put function via the worker

Cconstant1894 Mmm I'm only passing the request.body to the put function via the worker

H

Hello, I’m Allie!•1/22/24, 1:06 PM

Yeah, so adding the content-type there might help

HHello, I’m Allie!Yeah, so adding the content-type there might help

C

constant1894OP•1/22/24, 1:07 PM

I'll give it a go, but it's weird because all my other content types work.

Original message was deleted

H

Hello, I’m Allie!•1/22/24, 1:08 PM

If it is cached, no. Otherwise, yes

C

constant1894OP•1/22/24, 1:09 PM

@HardlyWorkin' Sorry for the dumb question, if my current worker api looks like this: await env.Testing_Bucket.put(key, request.body); How would i add the content type?

Cconstant1894 @HardlyWorkin' Sorry for the dumb question, if my current worker api looks like ...

H

Hello, I’m Allie!•1/22/24, 1:11 PM

await env.Testing_bucket.put(key, request.body, request.headers);

await env.Testing_bucket.put(key, request.body, request.headers);

await env.Testing_bucket.put(key, request.body, request.headers);

await env.Testing_bucket.put(key, request.body, request.headers);

should be fine

Original message was deleted

H

Hello, I’m Allie!•1/22/24, 1:12 PM

Yes

Original message was deleted

H

Hello, I’m Allie!•1/22/24, 1:12 PM

Should definitely be doable. Mine is slow, but it works: https://maps.goalastair.com

H

Hello, I’m Allie!•1/22/24, 1:12 PM

Based on Protomaps

HHello, I’m Allie!```ts await env.Testing_bucket.put(key, request.body, request.headers); ```shoul...

C

constant1894OP•1/22/24, 1:13 PM

Thanks, just tested in, but still getting the same issue as before.

S

Shubham•1/22/24, 1:43 PM

We are facing some issue while uploading file larger than 5Gb in the R2. We have migrated to R2 and we started facing issue, we cannot ask client to update their code to use multipart upload, also multipart upload is not that straight forward. We have already submitted the request from https://docs.google.com/forms/d/e/1FAIpQLSd_fwAVOboH9SlutMonzbhCxuuuOmiU1L_I5O2CFbXf_XXMRg/viewform?pli=1&pli=1 but no reply yet. We are facing too much issue due to this. We have use case of upload upto 15Gb. Could anybody from cloudflare team can help on this?

@Skye @Matt

Google Docs

Cloudflare Developer Platform: Limit Increase Request Form

Want to increase the limits of your current plan? Let us know what your requirements are here.

SShubham We are facing some issue while uploading file larger than 5Gb in the R2. We have...

S

Shubham•1/22/24, 1:56 PM

Account ID: 83025b28472d6aa2bf5ae59f3724aa78

E

Erisa•1/22/24, 2:15 PM

Please don't ping people for this. If you need urgent help you will need to open a support ticket, Discord server assistance is community supported and best effort.
To my knowledge 5GB is a fixed limit that should not be too unusual for an S3-compatible service and S3 multipart uploads are effectively an industry standard when handling large uploads.

S

Shubham•1/22/24, 2:49 PM

@Erisa | Support Engineer I raised the request using the request form and I tried opening support ticket but it is redirecting to discord for any R2 related issue. I understand that but we have migrated around 1TB of data to R2 and we are facing now issue which we were not facing with other provider. It was just a humble request to look into the request raised by Form as the business is getting impacted.

S

Shubham•1/22/24, 2:55 PM

@Space - Ping in replies I am asking because there was option for that request, we were using already another provider where we didn't had issue and after migration it is very difficult to move back more that 1TB data. If it cannot be done then we will be stuck with multi part option only that I am aware.

U

Unsmart•1/22/24, 2:57 PM

Workers would have the same 5GB limit. Probably lower if they aren’t enterprise

U

Unsmart•1/22/24, 2:58 PM

Yes. The 5GB limit is the max you can proxy through cloudflare on ent

S

Shubham•1/22/24, 2:58 PM

Exactly, before that we were streaming data with 7-8 GB

S

Shubham•1/22/24, 2:59 PM

If this request can be done for increase we will be happy otherwise we need to involve many stakeholders. But Thank you anyways

U

Unsmart•1/22/24, 2:59 PM

5GB is the absolute max you have to use multipart

UUnsmart Yes. The 5GB limit is the max you can proxy through cloudflare on ent

E

Erisa•1/22/24, 3:22 PM

Proxied upload can be increased above 5GB on Ent (and the max without contacting support/success is 500MB), but you start running into tradeoffs and needing to disable body buffering when you get that high so its usually not worth it.

S

Scruffy•1/22/24, 5:36 PM

I've got a corrupted file upload issue. On the web UI the files are listed with their respective upload dates and sizes. When I open a specific object in the web UI it shows as having 0 Bytes in size and any attempt to access it via the public URL returns a 404

S

Scruffy•1/22/24, 5:39 PM

S

Scruffy•1/22/24, 5:40 PM

To be clear my bucket does have public access enabled and it works for all my other objects

SScruffy I've got a corrupted file upload issue. On the web UI the files are listed with ...

M

Matt Silverlock•1/22/24, 5:42 PM

Can you please open a Support ticket, including the account ID and bucket ID? That's the best place for us to investigate this — any suggestion of "corruption" is taken very seriously

MMatt Silverlock Can you please open a Support ticket, including the account ID and bucket ID? Th...

S

Scruffy•1/22/24, 5:42 PM

Sure i dont actually know how to open a support ticket

U

Unsmart•1/22/24, 5:42 PM

The issue is the hashtag... when accessing by url you need to percent encode

%23

%23

%23

%23

U

Unsmart•1/22/24, 5:43 PM

https://support.google.com/richmedia/answer/190941?hl=en

UUnsmart <https://support.google.com/richmedia/answer/190941?hl=en>

S

Scruffy•1/22/24, 5:43 PM

yep thats correct

S

Scruffy•1/22/24, 5:43 PM

ty

S

Scruffy•1/22/24, 5:44 PM

would suggest fixing it on the cloudflare UI

S

Scruffy•1/22/24, 5:44 PM

V

Venkat Dinavahi•1/22/24, 6:26 PM

Hi has anyone succeeded in uploading to cloudflare R2 with a presigned url?

V

Venkat Dinavahi•1/22/24, 6:26 PM

If so, are there any working examples online for this? i've been struggling with a CORS issue. Seems to work for GET requests but PUT requests don't work.

V

Venkat Dinavahi•1/22/24, 6:28 PM

Or my ultimate goal is to be able to support uploads of up to 2GB but it appears that uploading to my cloudflare worker has request body limits.

SShubham We are facing some issue while uploading file larger than 5Gb in the R2. We have...

V

Venkat Dinavahi•1/22/24, 6:30 PM

How are you able to upload files even 2GB? I'm not able to figure out how to do this.

H

Hexstream•1/22/24, 7:47 PM

Hello, I'm getting unexpected errors while trying to retrieve stuff from R2, bizarrely cloudflarestatus does not seem to know about this at all...
Even downloading files directly from the R2 dashboard doesn't work.

H

Hexstream•1/22/24, 7:49 PM

get: The specified method is not allowed against this resource (10060)

H

Hexstream•1/22/24, 7:49 PM

Discord messages are too short for the full stacktrace, unfortunately.

H

Hexstream•1/22/24, 7:51 PM

Hum, I think redirects may be handled more strictly than before?

https://developers.cloudflare.com/support/troubleshooting/cloudflare-errors/troubleshooting-cloudflare-10xxx-errors/#error-10060-missing-scheme-in-redirect-target-url

H

Hexstream•1/22/24, 7:53 PM

Stuff is normally accessed through a trivial worker that just retrieves the R2 stuff, but if I try to download stuff from the dashboard that doesn't work either.

H

Hexstream•1/22/24, 7:54 PM

I could post a worker URL, but that could be seen as a bit spammy I guess...

U

Unsmart•1/22/24, 7:54 PM

You can post the URL. Everything works fine for me so seems its most likely an issue on your end so code would be helpful also

H

Hexstream•1/22/24, 7:55 PM

https://ponies.hexstream.net/videos/brony-originals/misc/buttons-adventures-episode-pilot.mp4 would be an example URL.

U

Unsmart•1/22/24, 7:55 PM

and whats the code for how you are getting the object?

H

Hexstream•1/22/24, 7:55 PM

I haven't changed anything related to that recently, so this must be a new incompatibility of some sort.

H

Hexstream•1/22/24, 7:57 PM

Yeah I'll fetch the code, I haven't touched that one in a while...

H

Hexstream•1/22/24, 7:58 PM

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

H

Hexstream•1/22/24, 8:01 PM

But logically this is probably not related to a problem with that code, since it hasn't changed in ages and I can't download the file directly from Cloudflare's R2 dashboard either, unless that's an unrelated issue...

U

Unsmart•1/22/24, 8:02 PM

whats the error you get when trying to download from the dashboard?

H

Hexstream•1/22/24, 8:03 PM

It just says "Download failed. Please try again.".

Mmm I'm only passing the request.body to the put function via the worker

Similar Threads

Similar Threads

Similar Threads