It's been about a day for me: https://pub-0177eb9df0874e6e892a9e84263b9075.r2.dev/sprites.json. Just

It's been about a day for me: https://pub-0177eb9df0874e6e892a9e84263b9075.r2.dev/sprites.json. Just to make sure though, it should send

ACAO

ACAO

even if it isn't an

OPTIONS

OPTIONS

request, right?

HHello, I’m Allie!No errors or anything. It just gets stuck

Sid•2/20/23, 3:54 PM

Try adding a

-vv

-vv

argument to see what it's doing?

HHello, I’m Allie!It's been about a day for me: https://pub-0177eb9df0874e6e892a9e84263b9075.r2.de...

DaniFoldi•2/20/23, 3:55 PM

Same here, I've added an origin yesterday and cors headers are still not present on the actual response

SSid Try adding a `-vv` argument to see what it's doing?

Hello, I’m Allie!OP•2/20/23, 3:55 PM

2023/02/20 16:55:05 DEBUG : rclone: Version "v1.53.3-DEV" starting with parameters ["rclone" "copy" "planet.pmtiles" "r2:/protomap" "-P" "--s3-upload-concurrency=3" "-vv"]
2023/02/20 16:55:05 DEBUG : Creating backend with remote "planet.pmtiles"
2023/02/20 16:55:05 DEBUG : Using config file from "/home/alastair/.config/rclone/rclone.conf"
2023/02/20 16:55:05 DEBUG : fs cache: adding new entry for parent of "planet.pmtiles", "/mnt"
2023/02/20 16:55:05 DEBUG : Creating backend with remote "r2:/protomap"
2023/02/20 16:55:05 DEBUG : fs cache: renaming cache item "r2:/protomap" to be canonical "r2:protomap"
2023-02-20 16:55:05 DEBUG : planet.pmtiles: Need to transfer - File not found at Destination
Transferred:             0 / 54.779 GBytes, 0%, 0 Bytes/s, ETA -
Transferred:            0 / 1, 0%
Elapsed time:        24.0s
Transferring:
 *                                planet.pmtiles:  0% /54.779G, 0/s, -

2023/02/20 16:55:05 DEBUG : rclone: Version "v1.53.3-DEV" starting with parameters ["rclone" "copy" "planet.pmtiles" "r2:/protomap" "-P" "--s3-upload-concurrency=3" "-vv"]
2023/02/20 16:55:05 DEBUG : Creating backend with remote "planet.pmtiles"
2023/02/20 16:55:05 DEBUG : Using config file from "/home/alastair/.config/rclone/rclone.conf"
2023/02/20 16:55:05 DEBUG : fs cache: adding new entry for parent of "planet.pmtiles", "/mnt"
2023/02/20 16:55:05 DEBUG : Creating backend with remote "r2:/protomap"
2023/02/20 16:55:05 DEBUG : fs cache: renaming cache item "r2:/protomap" to be canonical "r2:protomap"
2023-02-20 16:55:05 DEBUG : planet.pmtiles: Need to transfer - File not found at Destination
Transferred:             0 / 54.779 GBytes, 0%, 0 Bytes/s, ETA -
Transferred:            0 / 1, 0%
Elapsed time:        24.0s
Transferring:
 *                                planet.pmtiles:  0% /54.779G, 0/s, -

johtso•2/20/23, 3:57 PM

I tested the url here and it seems to be fine https://cors-test.codehappy.dev/?url=https://data.infinitenovelty.com/db.4.sqlite&origin=https://infinitenovelty.com&method=get

accept-ranges: bytes
access-control-allow-origin: https://infinitenovelty.com
cf-cache-status: DYNAMIC
cf-ray: 79c86f81f22c71c6-LHR
connection: keep-alive
content-length: 732532736
content-type: application/octet-stream
date: Mon, 20 Feb 2023 15:57:01 GMT
etag: "bca688cff8901d77164f486ca2cc0cd5"
last-modified: Mon, 20 Feb 2023 15:31:10 GMT
server: cloudflare
vary: Origin, Accept-Encoding

accept-ranges: bytes
access-control-allow-origin: https://infinitenovelty.com
cf-cache-status: DYNAMIC
cf-ray: 79c86f81f22c71c6-LHR
connection: keep-alive
content-length: 732532736
content-type: application/octet-stream
date: Mon, 20 Feb 2023 15:57:01 GMT
etag: "bca688cff8901d77164f486ca2cc0cd5"
last-modified: Mon, 20 Feb 2023 15:31:10 GMT
server: cloudflare
vary: Origin, Accept-Encoding

johtso•2/20/23, 3:58 PM

but when actually trying to load the file on my site it gives the CORS error.. https://infinitenovelty.com

johtso•2/20/23, 3:58 PM

HHello, I’m Allie!It's been about a day for me: https://pub-0177eb9df0874e6e892a9e84263b9075.r2.de...

Sid•2/20/23, 4:00 PM

It should, yep

Jjohtso I tested the url here and it seems to be fine https://cors-test.codehappy.dev/?u...

Sid•2/20/23, 4:02 PM

Yeah on that CORS testing site I see a warning, so this does look like an issue

johtso•2/20/23, 4:03 PM

isn't that just saying that I have an explicitly allowed domain?

johtso•2/20/23, 4:03 PM

which is fine, I have one domain where the resource will be used

Sid•2/20/23, 4:05 PM

Yeah NVM I'm an idiot, I thought this meant that this header was missing (when I should have read the line right above the code block)

johtso•2/20/23, 4:07 PM

Ah, figured it out, I needed to allow HEAD!

Sid•2/20/23, 4:10 PM

For others who were seeing similar CORS issues, I'm guessing y'alls config is correct, right?

HHello, I’m Allie!It's been about a day for me: https://pub-0177eb9df0874e6e892a9e84263b9075.r2.de...

Sid•2/20/23, 4:10 PM

This one seems to be working fine for me as well now!

SSid For others who were seeing similar CORS issues, I'm guessing y'alls config is co...

johtso•2/20/23, 4:23 PM

the problem is, devtools doesn't tell you what kind of request is being made if there is a CORS error

Sid•2/20/23, 4:23 PM

The Network tab should, no?

johtso•2/20/23, 4:24 PM

kian•2/20/23, 4:25 PM

One of the queries that Synology makes results in a

NextMarker

NextMarker

loop - forever

SSid This one seems to be working fine for me as well now!

Hello, I’m Allie!OP•2/20/23, 4:26 PM

Here's what I get for headers, live, with cache disabled locally:

HTTP/1.1 200 OK
Date: Mon, 20 Feb 2023 16:25:18 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
ETag: W/"1590706038a3703c37b0c771e68725b1"
Last-Modified: Mon, 20 Feb 2023 13:14:46 GMT
Vary: Accept-Encoding
Server: cloudflare
CF-RAY: 79c898efea12be4c-CPH
Content-Encoding: gzip

HTTP/1.1 200 OK
Date: Mon, 20 Feb 2023 16:25:18 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
ETag: W/"1590706038a3703c37b0c771e68725b1"
Last-Modified: Mon, 20 Feb 2023 13:14:46 GMT
Vary: Accept-Encoding
Server: cloudflare
CF-RAY: 79c898efea12be4c-CPH
Content-Encoding: gzip

Is it maybe that it is cached without the CORS headers?

Sid•2/20/23, 4:28 PM

So CORS rules aren't cached, they're distributed globally like all other CF config, and so we don't promise strong consistency there

SSid So CORS rules aren't _cached_, they're distributed globally like all other CF co...

Hello, I’m Allie!OP•2/20/23, 4:29 PM

No, I meant that it was cached on the Response via the CF Cache. But now I don't remember if R2 public domains even have cache...

Sid•2/20/23, 4:30 PM

They don't, out of the box. That's one of the reasons why we encourage you to use a custom domain, so you can configure whatever makes sense to your app

Ggaikwmsm Hi everyone, I have a few questions about R2. Thank you in advance for your assi...

Hello, I’m Allie!OP•2/20/23, 4:33 PM

#1: Not at the moment
#2: Caching is free on Cloudflare. The max size for a cached object is determined by the plan that the custom domain has.

Ggaikwmsm Anyone with any insight on this would be greatly appreciated

Sid•2/20/23, 4:34 PM

Also see https://developers.cloudflare.com/workers/platform/limits/#cache-api

Limits · Cloudflare Workers docs

You can request adjustments to limits that conflict with your project goals by contacting Cloudflare. To increase a limit, complete the Limit Increase …

Ggaikwmsm thanks

Hello, I’m Allie!OP•2/20/23, 4:34 PM

Happy to help!

Ggaikwmsm thanks

Hello, I’m Allie!OP•2/20/23, 4:47 PM

Just to clarify, because I misspoke, the max size for a cached object is determined by the plan that the custom domain has, and it isn't applied across your entire account.

johtso•2/20/23, 6:06 PM

what's the expected behaviour when making a range request on something in R2 with caching enabled?

johtso•2/20/23, 6:06 PM

currently the requests completely stall if the cache is cold

johtso•2/20/23, 6:19 PM

it's been about 20 minutes since I enabled caching on my custom r2 domain, and any range request that doesn't start at byte 0 still just stalls indefinitely

kian•2/20/23, 6:21 PM

Can't replicate it personally - do you have a URL to hand?

Kkian Can't replicate it personally - do you have a URL to hand?

johtso•2/20/23, 7:44 PM

https://infinitenovelty.com

johtso•2/20/23, 7:44 PM

you'll see it's trying to fetch ranges of https://data.infinitenovelty.com/db.4.sqlite

johtso•2/20/23, 7:44 PM

which is a 700MB file

Original message was deleted

Hello, I’m Allie!OP•2/20/23, 7:53 PM

Depends on the tier

johtso•2/20/23, 7:58 PM

where are the cache size limits described?

johtso•2/20/23, 7:58 PM

I'm on the free tier

johtso•2/20/23, 7:59 PM

shouldn't files that exceed the limit be served uncached?

Jjohtso I'm on the free tier

Hello, I’m Allie!OP•2/20/23, 8:02 PM

100 MB

johtso•2/20/23, 8:04 PM

thanks, I'll look into chunking it

johtso•2/20/23, 8:04 PM

but I'm still concerned that it behaves like this

Jjohtso thanks, I'll look into chunking it

Hello, I’m Allie!OP•2/20/23, 8:05 PM

Never mind:

johtso•2/20/23, 8:05 PM

if the resource isn't cached, it just breaks

2023/02/20 16:55:05 DEBUG : rclone: Version "v1.53.3-DEV" starting with parameters ["rclone" "copy" "planet.pmtiles" "r2:/protomap" "-P" "--s3-upload-concurrency=3" "-vv"] 2023/02/20 16:55:05 DEBUG : Creating backend with remote "planet.pmtiles" 2023/02/20 16:55:05 DEBUG : Using config file from "/home/alastair/.config/rclone/rclone.conf" 2023/02/20 16:55:05 DEBUG : fs cache: adding new entry for parent of "planet.pmtiles", "/mnt" 2023/02/20 16:55:05 DEBUG : Creating backend with remote "r2:/protomap" 2023/02/20 16:55:05 DEBUG : fs cache: renaming cache item "r2:/protomap" to be canonical "r2:protomap" 2023-02-20 16:55:05 DEBUG : planet.pmtiles: Need to transfer - File not found at Destination Transferred: 0 / 54.779 GBytes, 0%, 0 Bytes/s, ETA - Transferred: 0 / 1, 0% Elapsed time: 24.0s Transferring: * planet.pmtiles: 0% /54.779G, 0/s, -

accept-ranges: bytes access-control-allow-origin: https://infinitenovelty.com cf-cache-status: DYNAMIC cf-ray: 79c86f81f22c71c6-LHR connection: keep-alive content-length: 732532736 content-type: application/octet-stream date: Mon, 20 Feb 2023 15:57:01 GMT etag: "bca688cff8901d77164f486ca2cc0cd5" last-modified: Mon, 20 Feb 2023 15:31:10 GMT server: cloudflare vary: Origin, Accept-Encoding

HTTP/1.1 200 OK Date: Mon, 20 Feb 2023 16:25:18 GMT Content-Type: application/json Transfer-Encoding: chunked Connection: keep-alive ETag: W/"1590706038a3703c37b0c771e68725b1" Last-Modified: Mon, 20 Feb 2023 13:14:46 GMT Vary: Accept-Encoding Server: cloudflare CF-RAY: 79c898efea12be4c-CPH Content-Encoding: gzip