Migrated my Minecraft Overviewer maps to R2+B2 (instead of Spaces+B2) and wow, it's so fast!! Loving

Migrated my Minecraft Overviewer maps to R2+B2 (instead of Spaces+B2) and wow, it's so fast!! Loving the low latency https://mc-maps.geostyx.com/maps/servers/ftb-ultimate-ae-2022/index.html#/-4864/64/2560/-7/FTBUltimateAE2022%20-%20overworld/survivalday
Here's the handler for it that races R2 and B2 so I can arbitrarily store data in both storage providers: https://gist.github.com/jahands/a9625d2ac6553cce6a86aeb6d9cb8a18

JacobOP•7/7/22, 2:50 PM

It's loading about as fast as Google Cloud Storage, which is fantastic

JJacob Migrated my Minecraft Overviewer maps to R2+B2 (instead of Spaces+B2) and wow, i...

James•7/7/22, 3:41 PM

This is sweet

Mmusicman123 — FJLaboratories More just thinking of the "time till everything's back to operational" or "time ...

beelzabub•7/7/22, 4:58 PM

Externally that’s visible as availability not durability and is something for us to manage. Replicating in geographically distant regions is just one strategy.

JJacob Migrated my Minecraft Overviewer maps to R2+B2 (instead of Spaces+B2) and wow, i...

MyZeD•7/7/22, 5:19 PM

I use a external small Webhosting + Vector Tiles Maps (Not Minecraft) and cache it through Proxyed CF.
So i dont need Workers

JacobOP•7/7/22, 5:48 PM

Nice! I've found it's a challenge to efficiently host Leaflet maps at scale. I have about 37 million images taking up about 2TB of space. I used to host on a VPS, but quickly had issues with speed in different regions, and SSDs get expensive

funmiles.•7/7/22, 5:51 PM

All the examples in Cloudflare's R2 documentation for javascript use aws-sdk V2. Is there any example with V3 of the AWS SDK?

MMyZeD I use a external small Webhosting + Vector Tiles Maps (Not Minecraft) and cache ...

funmiles.•7/7/22, 5:52 PM

Which tile map source do you use? Some forbid caching.

JJacob Nice! I've found it's a challenge to efficiently host Leaflet maps at scale. I h...

MyZeD•7/7/22, 5:55 PM

this is another thing ;)

Ffunmiles.Which tile map source do you use? Some forbid caching.

MyZeD•7/7/22, 5:55 PM

If i pay them, so it can't be forbidden.
but i check that, thanks

Ffunmiles.Which tile map source do you use? Some forbid caching.

MyZeD•7/7/22, 5:56 PM

Amazon Location Service

funmiles.•7/7/22, 5:59 PM

@MyZeD I have an app where I wanted to use maps with vector tiles. I also wanted to tune the styling. I initially used mapbox, then maplibre but reading the licenses, it seems they don't want you to cache . Right now, I'm just using a 3D view of the world I made with free NASA images.

funmiles.•7/7/22, 6:03 PM

I actually want tiles up to level 9 or 10. I'm willing to pay for building all the tiles from openstreetmap data once and keep using them.

MMyZeD If i pay them, so it can't be forbidden. but i check that, thanks

funmiles.•7/7/22, 6:06 PM

I think their logic is you pay per a certain amount of tiles you retrieve. If you have a lot of user, they want you to pay essentially something for each user. I've read some of the agreement and some specifically say you can only serve the tiles from one retrieval to one user...

funmiles.•7/7/22, 6:12 PM

https://aws.amazon.com/service-terms/ Section 82.3 specifically says you cannot cache most of the data.

Amazon Web Services, Inc.

AWS Service Terms

Ffunmiles.All the examples in Cloudflare's R2 documentation for javascript use aws-sdk V2....

beelzabub•7/7/22, 6:17 PM

Not yet. Internally we use v2 for all our tests so that's why we have v2 examples everywhere. We're happy to take contributions to add v3 examples.

Bbeelzabub Not yet. Internally we use v2 for all our tests so that's why we have v2 example...

funmiles.•7/7/22, 6:19 PM

I could not find an example to store or retrieve data. Can you point me to an example?

funmiles.•7/7/22, 6:20 PM

Or can I use my code from AWS and do the same, except for the region argument?

funmiles.•7/7/22, 6:20 PM

my AWS v3 code.

kian•7/7/22, 6:22 PM

You’ll need to specify the R2 endpoint as your bucketEndpoint in the S3 client constructor’s config

kian•7/7/22, 6:22 PM

That’s all most the examples are doing - the code beyond that is the same as you’d use for S3, B2, GCS, etc for the most part

kian•7/7/22, 6:23 PM

I never made a V3 example since there was already the V2 one personally

kian•7/7/22, 6:25 PM

tbh there was one SDK that rejected my R2 token since it’s a different length to the AWS tokens but I wouldn’t be able to tell you which it was since I never looked into how to get around it and just used another SDK

funmiles.•7/7/22, 6:33 PM

OK I'll just try. BTW is it normal that with the V2 SDK, a listBuckets() takes about 2 seconds?

kian•7/7/22, 6:36 PM

import {S3Client} from "@aws-sdk/client-s3"
import {GetObjectCommand} from "@aws-sdk/client-s3"

const ACCOUNT_ID = 'x'
const ACCESS_KEY_ID = 'x'
const SECRET_ACCESS_KEY = 'x'

const S3 = new S3Client({
    region: "auto",
    endpoint: `https://${ACCOUNT_ID}.r2.cloudflarestorage.com`,
    credentials: {
        accessKeyId: ACCESS_KEY_ID,
        secretAccessKey: SECRET_ACCESS_KEY,
    }
});

console.log(
    await S3.send(new GetObjectCommand({
        Bucket: "sdk-example",
        Key: "ferriswasm.png"
    }))
)

import {S3Client} from "@aws-sdk/client-s3"
import {GetObjectCommand} from "@aws-sdk/client-s3"

const ACCOUNT_ID = 'x'
const ACCESS_KEY_ID = 'x'
const SECRET_ACCESS_KEY = 'x'

const S3 = new S3Client({
    region: "auto",
    endpoint: `https://${ACCOUNT_ID}.r2.cloudflarestorage.com`,
    credentials: {
        accessKeyId: ACCESS_KEY_ID,
        secretAccessKey: SECRET_ACCESS_KEY,
    }
});

console.log(
    await S3.send(new GetObjectCommand({
        Bucket: "sdk-example",
        Key: "ferriswasm.png"
    }))
)

kian•7/7/22, 6:37 PM

kian•7/7/22, 6:37 PM

That's an example

Ffunmiles.https://aws.amazon.com/service-terms/ Section 82.3 specifically says you cannot ...

MyZeD•7/7/22, 6:39 PM

Thank you for this information

Kkian ```js import {S3Client} from "@aws-sdk/client-s3" import {GetObjectCommand} from...

funmiles.•7/7/22, 6:43 PM

That looks like V3 to me. The import for V2 is import S3 from 'aws-sdk/clients/s3.js';import S3 from 'aws-sdk/clients/s3.js'; from my understanding.

funmiles.•7/7/22, 6:44 PM

So your example is quite useful

kian•7/7/22, 6:44 PM

Yeah, it's a V3 example

kian•7/7/22, 6:44 PM

There's already V2 examples in the docs

Ffunmiles.All the examples in Cloudflare's R2 documentation for javascript use aws-sdk V2....

kian•7/7/22, 6:44 PM

It was in reply to this

funmiles.•7/7/22, 6:46 PM

Ah thanks a bunch!

funmiles.•7/7/22, 6:58 PM

I was able to store my first file.

Kkian Yeah, it's a V3 example

beelzabub•7/7/22, 7:05 PM

Mind throwing that up in the docs so that we have both? Can be part of the JS page

kian•7/7/22, 7:06 PM

Sounds good to me

kian•7/7/22, 7:06 PM

Will PR it in a bit

funmiles.•7/7/22, 7:08 PM

For me what I was missing compared to AWS are the region: 'auto'region: 'auto' and even more the `endpoint: https://${ACCOUNT_ID}.r2.cloudflarestorage.com\https://${ACCOUNT_ID}.r2.cloudflarestorage.com\ parts

kian•7/7/22, 7:11 PM

It works without the region to be fair - R2 accepts an empty string and

us-east-1

us-east-1

as aliases to

auto

auto

but for completeness, it's best to use

auto

auto

funmiles.•7/7/22, 7:25 PM

Thanks for the clarification. Still end endpointendpoint is necessary, right? Without it I had an error telling me the account was wrong, or something of the sort.

kian•7/7/22, 7:25 PM

Yeah