How are you able to upload files even 2GB? I'm not able to figure out how to do this.

H

Hexstream•1/22/24, 7:47 PM

Hello, I'm getting unexpected errors while trying to retrieve stuff from R2, bizarrely cloudflarestatus does not seem to know about this at all...
Even downloading files directly from the R2 dashboard doesn't work.

H

Hexstream•1/22/24, 7:49 PM

get: The specified method is not allowed against this resource (10060)

H

Hexstream•1/22/24, 7:49 PM

Discord messages are too short for the full stacktrace, unfortunately.

H

Hexstream•1/22/24, 7:51 PM

Hum, I think redirects may be handled more strictly than before?

https://developers.cloudflare.com/support/troubleshooting/cloudflare-errors/troubleshooting-cloudflare-10xxx-errors/#error-10060-missing-scheme-in-redirect-target-url

H

Hexstream•1/22/24, 7:53 PM

Stuff is normally accessed through a trivial worker that just retrieves the R2 stuff, but if I try to download stuff from the dashboard that doesn't work either.

H

Hexstream•1/22/24, 7:54 PM

I could post a worker URL, but that could be seen as a bit spammy I guess...

U

Unsmart•1/22/24, 7:54 PM

You can post the URL. Everything works fine for me so seems its most likely an issue on your end so code would be helpful also

H

Hexstream•1/22/24, 7:55 PM

https://ponies.hexstream.net/videos/brony-originals/misc/buttons-adventures-episode-pilot.mp4 would be an example URL.

U

Unsmart•1/22/24, 7:55 PM

and whats the code for how you are getting the object?

H

Hexstream•1/22/24, 7:55 PM

I haven't changed anything related to that recently, so this must be a new incompatibility of some sort.

H

Hexstream•1/22/24, 7:57 PM

Yeah I'll fetch the code, I haven't touched that one in a while...

H

Hexstream•1/22/24, 7:58 PM

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

function retrieve (url, env) {
    const path = url.hostname + url.pathname;
    const split = path.split("/");
    const tmpIndex = split.indexOf("tmp");
    if (tmpIndex >= 0) {
        split.splice(tmpIndex, 1);
        const bucket = env.TMP_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    } else {
        const bucket = env.BIG_BUCKET;
        const key = split.join("/");
        return bucket.get(key);
    }
}

export default {
  async fetch (request, env, context) {
      const cache = await caches.open("r2");
      const cachedResponse = await cache.match(request);
      if (cachedResponse)
          return cachedResponse;
      const object = await retrieve(new URL(request.url), env);
      if (object === null)
          return fetch(request);
      const body = await object.body;
      const headers = new Headers();
      object.writeHttpMetadata(headers);
      headers.set('etag', object.httpEtag);
      headers.append('Cache-Control', 's-maxage=691200');
      const response = new Response(body, { headers });
      const range = request.headers.get("Range");
      if (range === null) {
          context.waitUntil(cache.put(request, response.clone()));
          return response;
      } else {
          await cache.put(request, response);
          return cache.match(request);
      }
  }
};

H

Hexstream•1/22/24, 8:01 PM

But logically this is probably not related to a problem with that code, since it hasn't changed in ages and I can't download the file directly from Cloudflare's R2 dashboard either, unless that's an unrelated issue...

U

Unsmart•1/22/24, 8:02 PM

whats the error you get when trying to download from the dashboard?

H

Hexstream•1/22/24, 8:03 PM

It just says "Download failed. Please try again.".

U

Unsmart•1/22/24, 8:04 PM

Not sure I would open a support ticket

H

Hexstream•1/22/24, 8:05 PM

I'm on the free plan. The support wizard told me to post about this here.

H

Hexstream•1/22/24, 8:06 PM

H

Hexstream•1/22/24, 8:07 PM

Perhaps I could try posting in the community support forum thingy...

H

Hexstream•1/22/24, 8:07 PM

I would be pretty shocked if I'm the only one affected by this...

U

Unsmart•1/22/24, 8:09 PM

I'd find it very likely you are the only one otherwise I would expect a lot of reports here and on the community yet there are none and things work just fine for me on all of my buckets. Probably some configuration option set to a bad value but not sure

H

Hexstream•1/22/24, 8:10 PM

That's the weird thing, nothing changed, basically...
These errors just started popping up out of nowhere.

U

Unsmart•1/22/24, 8:10 PM

R2 employees do read this channel from time to time so they would best be able to assist when they see it

H

Hexstream•1/22/24, 8:12 PM

Yeah, normally I would use my tmp bucket but that won't work right now... I'll upload the full stacktrace somewhere.
But again, this also fails from the dashboard directly, so clearly unrelated to the worker trying to fetch.
(Unless the dashboard download error is unrelated...)

H

Hexstream•1/22/24, 8:13 PM

No, since yesterday at least.

H

Hexstream•1/22/24, 8:15 PM

Great question! I should try that, but I almost never use rclone so figuring out the incantation could take a while.

H

Hexstream•1/22/24, 8:16 PM

Thanks for the help btw, usually my strategy is waiting for this kind of stuff to get fixed, but seems like that might not happen on its own this time...

UUnsmart I'd find it very likely you are the only one otherwise I would expect a lot of r...

C

Chaika•1/22/24, 8:21 PM

I mean there was this just recently: https://www.cloudflarestatus.com/incidents/cxyckjngxnqk
strange though

R2 Availability Impact on object retrieval

U

Unsmart•1/22/24, 8:21 PM

Pretty different issue though

C

Chaika•1/22/24, 8:22 PM

Tailing the worker to see the exception its throwing might also not be a bad idea, or using dev tools (ctrl+shift+i -> Network tab) and seeing the exact response trying to download the item from the bucket gives you other then failure

CChaika I mean there was this just recently: https://www.cloudflarestatus.com/incidents/...

H

Hexstream•1/22/24, 8:27 PM

Yeah, I saw that, I would have thought that could be related, but this is apparently resolved...

CChaika Tailing the worker to see the exception its throwing might also not be a bad ide...

H

Hexstream•1/22/24, 8:29 PM

I'm getting a 405 Method Not Allowed in DevTools when trying to download from the dashboard.

HHexstream > get: The specified method is not allowed against this resource (10060)

H

Hexstream•1/22/24, 8:30 PM

This seems consistent with the worker error...

H

Hexstream•1/22/24, 8:38 PM

Here is the full stack trace from the worker:
https://gist.github.com/Hexstream/801cee304a4d552959241858bb8e6dc7

Gist

Cannot fetch from R2: full stacktrace

Cannot fetch from R2: full stacktrace. GitHub Gist: instantly share code, notes, and snippets.

H

Hexstream•1/22/24, 8:44 PM

H

Hexstream•1/22/24, 8:50 PM

HHexstream Here is the full stack trace from the worker: https://gist.github.com/Hexstream/...

C

Chaika•1/22/24, 9:07 PM

that's weird, almost makes me think it's a T&S action, not sure though. You don't have any overdue/partially paid invoices though, right? Worth a quick check
Otherwise what is your account id (visible on right side of R2 page, not a secret), and the bucket name?

H

Hexstream•1/22/24, 9:23 PM

I'm assuming T&S actions without so much as an email is simply not a thing?
No unpaid stuff whatsoever...

Account ID: 4337027168198b052256d6acbf1adf57
Bucket name: big

H

Hexstream•1/22/24, 9:28 PM

Oh wow, download from other buckets does seem to work...

H

Hexstream•1/22/24, 10:30 PM

The 20 january date in my last screenshot above does weirdly correspond with this R2 incident, perhaps this is not fully resolved after all?
https://www.cloudflarestatus.com/incidents/cxyckjngxnqk

R2 Availability Impact on object retrieval

BBig Al I am using AWS SDK V3 to upload/fetch files from R2. I am testing out the Create...

B

Big Al•1/22/24, 10:33 PM

I figured this out, I had entered the endpoint into my .env with the bucketname on the end... so it was trying to create a bucket inside a bucket and list the buckets inside a bucket. Which weirdly still responded with status code 200s. Anyway, dumb mistake that took forever to find.

K

kian•1/22/24, 10:35 PM

Probably just created a file in the bucket

K

kian•1/23/24, 1:51 AM

There's no more metadata.json or a folder, it's SQLite and a folder of 'blobs' which are the file contents

P

Potassium•1/23/24, 5:00 AM

how do i make my bucket become public without r2.dev ? I need to buy domain, then setup cloudflare zone, then what is the next step?

PPotassium how do i make my bucket become public without r2.dev ? I need to buy domain, the...

K

Karew•1/23/24, 5:18 AM

(1) Get a domain (2) Add the domain to your Cloudflare account (3) In the settings for your R2 bucket, add a custom (sub)domain

P

Potassium•1/23/24, 5:19 AM

Just that ?

P

Potassium•1/23/24, 5:20 AM

how do i use the same domain for multiple bucket ? For example, if abc.com is already used by other bucket, how do i use the domain again for another bucket?

U

Unsmart•1/23/24, 5:28 AM

you would use foo.abc.com

VVenkat Dinavahi How are you able to upload files even 2GB? I'm not able to figure out how to do ...

S

Shubham•1/23/24, 5:40 AM

We are using presigned URL and using that URL to upload with timeout of 15min, we are using files of around 8-10 GB.

How are you able to upload files even 2GB? I'm not able to figure out how to do this.

Similar Threads

Similar Threads

Similar Threads