if i have lets say 5 servers then i get 5x the traffic with this method

T

tobiOP•5/24/23, 5:37 PM

yea, basically just "forwarding" the request somewhere, right?

Ttobi if i have lets say 5 servers then i get 5x the traffic with this method

D

dave•5/24/23, 5:37 PM

Sure, but you can cache the result too if it's problem.

T

tobiOP•5/24/23, 5:38 PM

depends on what one wants to accomplish.

D

dave•5/24/23, 5:38 PM

It's really nice since you can do "advanced" ish load balancing too if want.

e.g. can see the round-trip latency by doing:

curl -s "https://untrusted.files.ai.moda/api/v1.0.0-alpha/ping" | jq

curl -s "https://untrusted.files.ai.moda/api/v1.0.0-alpha/ping" | jq

curl -s "https://untrusted.files.ai.moda/api/v1.0.0-alpha/ping" | jq

curl -s "https://untrusted.files.ai.moda/api/v1.0.0-alpha/ping" | jq

T

tobiOP•5/24/23, 5:39 PM

if i, lets say want to build some kind of "low-latency" storage for small files, that is kinda guaranteed to have sub <50-60 ms latency, i could make that with less (or no) js on the user side, while still being able to serve the data quickly.

Ddave It's really nice since you can do "advanced" ish load balancing too if want. e...

T

tobiOP•5/24/23, 5:40 PM

what do you mean with that?

Ttobi if i, lets say want to build some kind of "low-latency" storage for small files,...

D

dave•5/24/23, 5:40 PM

https://untrusted.files.ai.moda/api/v1.0.0-alpha/url/generate_one_time

https://untrusted.files.ai.moda/api/v1.0.0-alpha/url/generate_one_time

https://untrusted.files.ai.moda/api/v1.0.0-alpha/url/generate_one_time

https://untrusted.files.ai.moda/api/v1.0.0-alpha/url/generate_one_time

^ we have a similar ish use-case, and made this to return the endpoint with the estimated lowest latency for the user.

T

tobiOP•5/24/23, 5:41 PM

ahhh

T

tobiOP•5/24/23, 5:41 PM

basically just a request to where the client should then upload to. so same thing accomplished, just not intransparent to the user

D

dave•5/24/23, 5:41 PM

sure it can be

T

tobiOP•5/24/23, 5:42 PM

uh, nearest datacenter hit is london?

Ttobi uh, nearest datacenter hit is london?

D

dave•5/24/23, 5:43 PM

that sounds right, since that uses an S3 backend with buckets in Frankfurt, Ireland, London, Montreal, San Jose, and Virginia.

T

tobiOP•5/24/23, 5:43 PM

well, then from austria it should have hit frankfurt

Ttobi basically just a request to where the client should then upload to. so same thin...

D

dave•5/24/23, 5:44 PM

if it's <100MB, you can just accept the request data body directly in the Worker and POST it to an endpoint elsewhere without needing to make the client wait.

Ttobi well, then from austria it should have hit frankfurt 😛 😉

D

dave•5/24/23, 5:45 PM

ah so this crappy S3 provider has terrible uptime, and their Frankfurt server has crazy reliability issues (hours of downtime per month). That's why this Worker is really nice, because it actually checks if the S3 endpoint is working or not.

D

dave•5/24/23, 5:46 PM

(off-topic but are only using this unreliable S3 provider temporarily mainly because of the lack of per-bucket API tokens for R2.)

T

tobiOP•5/24/23, 5:47 PM

ahhh

thanks for the insight, always interresting to hear what others are building.

Ttobi if i, lets say want to build some kind of "low-latency" storage for small files,...

D

dave•5/24/23, 5:47 PM

Will your users be using a browser, or is this going to be used by another backend?

T

tobiOP•5/24/23, 5:49 PM

mainly browser.

Ttobi mainly browser.

D

dave•5/24/23, 5:51 PM

we do a lot of preloading on our website. e.g. we have this at the bottom of our head tags:

<link fetchpriority="low" rel="preload" href="/geolocation.json" as="fetch" crossorigin="anonymous" />

<link fetchpriority="low" rel="preload" href="/geolocation.json" as="fetch" crossorigin="anonymous" />

<link fetchpriority="low" rel="preload" href="/geolocation.json" as="fetch" crossorigin="anonymous" />

<link fetchpriority="low" rel="preload" href="/geolocation.json" as="fetch" crossorigin="anonymous" />

That way when we fetch

/geolocation.json

/geolocation.json

/geolocation.json

/geolocation.json later, it's already loaded.

D

dave•5/24/23, 5:56 PM

if it's something that could change, then IMO I would do like:

setInterval(() => {
  const link = document.createElement("link");
  link.setAttribute("fetchpriority", "low");
  link.setAttribute("rel", "preload");
  link.setAttribute("href", `/geolocation.json`);
  link.setAttribute("as", "fetch");
  link.setAttribute("crossorigin", "anonymous");
  document.head.appendChild(link);
}, 60000)

setInterval(() => {
  const link = document.createElement("link");
  link.setAttribute("fetchpriority", "low");
  link.setAttribute("rel", "preload");
  link.setAttribute("href", `/geolocation.json`);
  link.setAttribute("as", "fetch");
  link.setAttribute("crossorigin", "anonymous");
  document.head.appendChild(link);
}, 60000)

setInterval(() => {
  const link = document.createElement("link");
  link.setAttribute("fetchpriority", "low");
  link.setAttribute("rel", "preload");
  link.setAttribute("href", `/geolocation.json`);
  link.setAttribute("as", "fetch");
  link.setAttribute("crossorigin", "anonymous");
  document.head.appendChild(link);
}, 60000)

setInterval(() => {
  const link = document.createElement("link");
  link.setAttribute("fetchpriority", "low");
  link.setAttribute("rel", "preload");
  link.setAttribute("href", `/geolocation.json`);
  link.setAttribute("as", "fetch");
  link.setAttribute("crossorigin", "anonymous");
  document.head.appendChild(link);
}, 60000)

You should probably remove the old link so you don't end up with hundreds of elements in your header over time

T

tobiOP•5/24/23, 5:57 PM

hmm, it could change, but you wouldn't expect the user to suddenly change country xD

K

kian•5/24/23, 6:19 PM

since Chrome 103 meowwink

K

kian•5/24/23, 6:19 PM

oh hey Edge does too - I guess by extension of being Chromium

K

kian•5/24/23, 6:19 PM

Safari is the one I'll be hoping for - Safari on iOS is a big chunk of our users

S

sakuta•5/24/23, 7:22 PM

what permissions does my api token need for

wrangler deploy

wrangler deploy

wrangler deploy

wrangler deploy? i'm using only

Workers Scripts:Edit

Workers Scripts:Edit

Workers Scripts:Edit

Workers Scripts:Edit permission at the moment and get an "unauthorized" error in my logs...

NNeiki i cant find anything to workers or crons in the notifications tab, only about pa...

N

Neiki•5/24/23, 7:26 PM

bump

P

Peter Belbin•5/24/23, 8:00 PM

hey there folks. question: https://github.com/cloudflare/wrangler-legacy/pull/2221 talks about being able to mark assets for future expiration of cached content to avoid users getting 404 errors when a new deployment happens for Wrangler workers. It also talks about providing something similar for 2.x (and now we're on 3.x). What's the story with this currently (particularly for 3.x). How does KV content get held over until the upgrade is done currently? Really want our users to not see 404 errors because they've been cuaght mid-update.

GitHub

fix: expire unused assets on site upload by caass · Pull Request #2...

backport of cloudflare/workers-sdk#587
currently, we just delete unused assets when uploading a new version
of a workers-sites project. this can cause problems when a user
accessing an old site get...

J

James•5/24/23, 8:05 PM

I believe in v2/v3 with Workers Sites, old assets are deleted as soon as new ones are published (ref: https://github.com/cloudflare/workers-sdk/blob/4082cfcbdf08740d4a608d3d87df22e51ad0ce4a/packages/wrangler/src/sites.ts#L319). The holistic solution these days would be to migrate to Cloudflare Pages (https://pages.cloudflare.com/) where development efforts are focused instead of Workers Sites

NNeiki bump

C

Cyb3r-Jak3•5/24/23, 8:10 PM

Workers doesn’t have error notifications. You have to put those in your script

CCyb3r-Jak3 Workers doesn’t have error notifications. You have to put those in your script

N

Neiki•5/24/23, 8:18 PM

but i cant catch the "Exceeded Resources" thing, it just happens sometimes i cant really say why its weird (only happend twice so far) but i thought maybe i can let me notify or something

K

kian•5/24/23, 8:25 PM

https://developers.cloudflare.com/workers/platform/tail-workers/ if you're on a paid plan

Tail Workers · Cloudflare Workers docs

A Tail Worker receives information about the execution of other Workers, such as HTTP statuses, data passed to console.log() or uncaught exceptions. …

N

Neiki•5/24/23, 8:26 PM

thanks and yes

NNeiki thanks and yes

N

Neiki•5/24/23, 8:48 PM

nvm sorry i found it, sorry for ping

Original message was deleted

D

dave•5/24/23, 9:11 PM

Correct. It's a bit annoying to have to have different preloading implementations for Chrome vs Safari though. For ease of use, we just stick with doing preloading client side for now.

Kkian Safari is the one I'll be hoping for - Safari on iOS is a big chunk of our users

D

dave•5/24/23, 9:12 PM

and those are the ones with the slow interwebz who you care about usually

D

dave•5/24/23, 9:17 PM

@sathoro btw Supabase staff will turn off rate limiting for your project if you ask them and say it's because you're using a Worker. Don't think you were running into issues yet, but figured I should share the info.

if i have lets say 5 servers then i get 5x the traffic with this method

Similar Threads

Similar Threads

Similar Threads