I've got a corrupted file upload issue. On the web UI the files are listed with their respective upl

S

ScruffyOP•1/22/24, 5:39 PM

S

ScruffyOP•1/22/24, 5:40 PM

To be clear my bucket does have public access enabled and it works for all my other objects

SScruffy I've got a corrupted file upload issue. On the web UI the files are listed with ...

M

Matt Silverlock•1/22/24, 5:42 PM

Can you please open a Support ticket, including the account ID and bucket ID? That's the best place for us to investigate this — any suggestion of "corruption" is taken very seriously

MMatt Silverlock Can you please open a Support ticket, including the account ID and bucket ID? Th...

S

ScruffyOP•1/22/24, 5:42 PM

Sure i dont actually know how to open a support ticket

U

Unsmart•1/22/24, 5:42 PM

The issue is the hashtag... when accessing by url you need to percent encode

%23

%23

%23

%23

U

Unsmart•1/22/24, 5:43 PM

https://support.google.com/richmedia/answer/190941?hl=en

UUnsmart <https://support.google.com/richmedia/answer/190941?hl=en>

S

ScruffyOP•1/22/24, 5:43 PM

yep thats correct

S

ScruffyOP•1/22/24, 5:43 PM

ty

S

ScruffyOP•1/22/24, 5:44 PM

would suggest fixing it on the cloudflare UI

S

ScruffyOP•1/22/24, 5:44 PM

V

Venkat Dinavahi•1/22/24, 6:26 PM

Hi has anyone succeeded in uploading to cloudflare R2 with a presigned url?

V

Venkat Dinavahi•1/22/24, 6:26 PM

If so, are there any working examples online for this? i've been struggling with a CORS issue. Seems to work for GET requests but PUT requests don't work.

V

Venkat Dinavahi•1/22/24, 6:28 PM

Or my ultimate goal is to be able to support uploads of up to 2GB but it appears that uploading to my cloudflare worker has request body limits.

SShubham We are facing some issue while uploading file larger than 5Gb in the R2. We have...

V

Venkat Dinavahi•1/22/24, 6:30 PM

How are you able to upload files even 2GB? I'm not able to figure out how to do this.

H

Hexstream•1/22/24, 7:47 PM

Hello, I'm getting unexpected errors while trying to retrieve stuff from R2, bizarrely cloudflarestatus does not seem to know about this at all...
Even downloading files directly from the R2 dashboard doesn't work.

H

Hexstream•1/22/24, 7:49 PM

get: The specified method is not allowed against this resource (10060)

H

Hexstream•1/22/24, 7:49 PM

Discord messages are too short for the full stacktrace, unfortunately.

H

Hexstream•1/22/24, 7:51 PM

Hum, I think redirects may be handled more strictly than before?

https://developers.cloudflare.com/support/troubleshooting/cloudflare-errors/troubleshooting-cloudflare-10xxx-errors/#error-10060-missing-scheme-in-redirect-target-url

H

Hexstream•1/22/24, 7:53 PM

Stuff is normally accessed through a trivial worker that just retrieves the R2 stuff, but if I try to download stuff from the dashboard that doesn't work either.

H

Hexstream•1/22/24, 7:54 PM

I could post a worker URL, but that could be seen as a bit spammy I guess...

U

Unsmart•1/22/24, 7:54 PM

You can post the URL. Everything works fine for me so seems its most likely an issue on your end so code would be helpful also

H

Hexstream•1/22/24, 7:55 PM

https://ponies.hexstream.net/videos/brony-originals/misc/buttons-adventures-episode-pilot.mp4 would be an example URL.

U

Unsmart•1/22/24, 7:55 PM

and whats the code for how you are getting the object?

H

Hexstream•1/22/24, 7:55 PM

I haven't changed anything related to that recently, so this must be a new incompatibility of some sort.

H

Hexstream•1/22/24, 7:57 PM

Yeah I'll fetch the code, I haven't touched that one in a while...

H

Hexstream•1/22/24, 7:58 PM

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

H

Hexstream•1/22/24, 8:01 PM

But logically this is probably not related to a problem with that code, since it hasn't changed in ages and I can't download the file directly from Cloudflare's R2 dashboard either, unless that's an unrelated issue...

U

Unsmart•1/22/24, 8:02 PM

whats the error you get when trying to download from the dashboard?

H

Hexstream•1/22/24, 8:03 PM

It just says "Download failed. Please try again.".

U

Unsmart•1/22/24, 8:04 PM

Not sure I would open a support ticket

H

Hexstream•1/22/24, 8:05 PM

I'm on the free plan. The support wizard told me to post about this here.

H

Hexstream•1/22/24, 8:06 PM

H

Hexstream•1/22/24, 8:07 PM

Perhaps I could try posting in the community support forum thingy...

H

Hexstream•1/22/24, 8:07 PM

I would be pretty shocked if I'm the only one affected by this...

U

Unsmart•1/22/24, 8:09 PM

I'd find it very likely you are the only one otherwise I would expect a lot of reports here and on the community yet there are none and things work just fine for me on all of my buckets. Probably some configuration option set to a bad value but not sure

H

Hexstream•1/22/24, 8:10 PM

That's the weird thing, nothing changed, basically...
These errors just started popping up out of nowhere.

U

Unsmart•1/22/24, 8:10 PM

R2 employees do read this channel from time to time so they would best be able to assist when they see it

H

Hexstream•1/22/24, 8:12 PM

Yeah, normally I would use my tmp bucket but that won't work right now... I'll upload the full stacktrace somewhere.
But again, this also fails from the dashboard directly, so clearly unrelated to the worker trying to fetch.
(Unless the dashboard download error is unrelated...)

H

Hexstream•1/22/24, 8:13 PM

No, since yesterday at least.

H

Hexstream•1/22/24, 8:15 PM

Great question! I should try that, but I almost never use rclone so figuring out the incantation could take a while.

H

Hexstream•1/22/24, 8:16 PM

Thanks for the help btw, usually my strategy is waiting for this kind of stuff to get fixed, but seems like that might not happen on its own this time...

UUnsmart I'd find it very likely you are the only one otherwise I would expect a lot of r...

C

Chaika•1/22/24, 8:21 PM

I mean there was this just recently: https://www.cloudflarestatus.com/incidents/cxyckjngxnqk
strange though

R2 Availability Impact on object retrieval

U

Unsmart•1/22/24, 8:21 PM

Pretty different issue though

C

Chaika•1/22/24, 8:22 PM

Tailing the worker to see the exception its throwing might also not be a bad idea, or using dev tools (ctrl+shift+i -> Network tab) and seeing the exact response trying to download the item from the bucket gives you other then failure

CChaika I mean there was this just recently: https://www.cloudflarestatus.com/incidents/...

H

Hexstream•1/22/24, 8:27 PM

Yeah, I saw that, I would have thought that could be related, but this is apparently resolved...

CChaika Tailing the worker to see the exception its throwing might also not be a bad ide...

H

Hexstream•1/22/24, 8:29 PM

I'm getting a 405 Method Not Allowed in DevTools when trying to download from the dashboard.

HHexstream > get: The specified method is not allowed against this resource (10060)

H

Hexstream•1/22/24, 8:30 PM

This seems consistent with the worker error...

H

Hexstream•1/22/24, 8:38 PM

Here is the full stack trace from the worker:
https://gist.github.com/Hexstream/801cee304a4d552959241858bb8e6dc7

Gist

Cannot fetch from R2: full stacktrace

Cannot fetch from R2: full stacktrace. GitHub Gist: instantly share code, notes, and snippets.

H

Hexstream•1/22/24, 8:44 PM

H

Hexstream•1/22/24, 8:50 PM

I've got a corrupted file upload issue. On the web UI the files are listed with their respective upl

Similar Threads

Similar Threads

Similar Threads