if i, lets say want to build some kind of "low-latency" storage for small files, that is kinda guara

if i, lets say want to build some kind of "low-latency" storage for small files, that is kinda guaranteed to have sub <50-60 ms latency, i could make that with less (or no) js on the user side, while still being able to serve the data quickly.

Ddave It's really nice since you can do "advanced" ish load balancing too if want. e...

tobiOP•5/24/23, 5:40 PM

what do you mean with that?

Ttobi if i, lets say want to build some kind of "low-latency" storage for small files,...

dave•5/24/23, 5:40 PM

https://untrusted.files.ai.moda/api/v1.0.0-alpha/url/generate_one_time

https://untrusted.files.ai.moda/api/v1.0.0-alpha/url/generate_one_time

^ we have a similar ish use-case, and made this to return the endpoint with the estimated lowest latency for the user.

tobiOP•5/24/23, 5:41 PM

ahhh

tobiOP•5/24/23, 5:41 PM

basically just a request to where the client should then upload to. so same thing accomplished, just not intransparent to the user

dave•5/24/23, 5:41 PM

sure it can be

tobiOP•5/24/23, 5:42 PM

uh, nearest datacenter hit is london?

Ttobi uh, nearest datacenter hit is london?

dave•5/24/23, 5:43 PM

that sounds right, since that uses an S3 backend with buckets in Frankfurt, Ireland, London, Montreal, San Jose, and Virginia.

tobiOP•5/24/23, 5:43 PM

well, then from austria it should have hit frankfurt

Ttobi basically just a request to where the client should then upload to. so same thin...

dave•5/24/23, 5:44 PM

if it's <100MB, you can just accept the request data body directly in the Worker and POST it to an endpoint elsewhere without needing to make the client wait.

Ttobi well, then from austria it should have hit frankfurt 😛 😉

dave•5/24/23, 5:45 PM

ah so this crappy S3 provider has terrible uptime, and their Frankfurt server has crazy reliability issues (hours of downtime per month). That's why this Worker is really nice, because it actually checks if the S3 endpoint is working or not.

dave•5/24/23, 5:46 PM

(off-topic but are only using this unreliable S3 provider temporarily mainly because of the lack of per-bucket API tokens for R2.)

tobiOP•5/24/23, 5:47 PM

ahhh

thanks for the insight, always interresting to hear what others are building.

Ttobi if i, lets say want to build some kind of "low-latency" storage for small files,...

dave•5/24/23, 5:47 PM

Will your users be using a browser, or is this going to be used by another backend?

tobiOP•5/24/23, 5:49 PM

mainly browser.

Ttobi mainly browser.

dave•5/24/23, 5:51 PM

we do a lot of preloading on our website. e.g. we have this at the bottom of our head tags:

<link fetchpriority="low" rel="preload" href="/geolocation.json" as="fetch" crossorigin="anonymous" />

<link fetchpriority="low" rel="preload" href="/geolocation.json" as="fetch" crossorigin="anonymous" />

That way when we fetch

/geolocation.json

/geolocation.json

later, it's already loaded.

dave•5/24/23, 5:56 PM

if it's something that could change, then IMO I would do like:

setInterval(() => {
  const link = document.createElement("link");
  link.setAttribute("fetchpriority", "low");
  link.setAttribute("rel", "preload");
  link.setAttribute("href", `/geolocation.json`);
  link.setAttribute("as", "fetch");
  link.setAttribute("crossorigin", "anonymous");
  document.head.appendChild(link);
}, 60000)

setInterval(() => {
  const link = document.createElement("link");
  link.setAttribute("fetchpriority", "low");
  link.setAttribute("rel", "preload");
  link.setAttribute("href", `/geolocation.json`);
  link.setAttribute("as", "fetch");
  link.setAttribute("crossorigin", "anonymous");
  document.head.appendChild(link);
}, 60000)

You should probably remove the old link so you don't end up with hundreds of elements in your header over time

tobiOP•5/24/23, 5:57 PM

hmm, it could change, but you wouldn't expect the user to suddenly change country xD

kian•5/24/23, 6:19 PM

since Chrome 103 meowwink

kian•5/24/23, 6:19 PM

oh hey Edge does too - I guess by extension of being Chromium

kian•5/24/23, 6:19 PM

Safari is the one I'll be hoping for - Safari on iOS is a big chunk of our users

sakuta•5/24/23, 7:22 PM

what permissions does my api token need for

wrangler deploy

wrangler deploy

? i'm using only

Workers Scripts:Edit

Workers Scripts:Edit

permission at the moment and get an "unauthorized" error in my logs...

NNeiki i cant find anything to workers or crons in the notifications tab, only about pa...

Neiki•5/24/23, 7:26 PM

bump

Peter Belbin•5/24/23, 8:00 PM

hey there folks. question: https://github.com/cloudflare/wrangler-legacy/pull/2221 talks about being able to mark assets for future expiration of cached content to avoid users getting 404 errors when a new deployment happens for Wrangler workers. It also talks about providing something similar for 2.x (and now we're on 3.x). What's the story with this currently (particularly for 3.x). How does KV content get held over until the upgrade is done currently? Really want our users to not see 404 errors because they've been cuaght mid-update.

GitHub

fix: expire unused assets on site upload by caass · Pull Request #2...

backport of cloudflare/workers-sdk#587
currently, we just delete unused assets when uploading a new version
of a workers-sites project. this can cause problems when a user
accessing an old site get...

James•5/24/23, 8:05 PM

I believe in v2/v3 with Workers Sites, old assets are deleted as soon as new ones are published (ref: https://github.com/cloudflare/workers-sdk/blob/4082cfcbdf08740d4a608d3d87df22e51ad0ce4a/packages/wrangler/src/sites.ts#L319). The holistic solution these days would be to migrate to Cloudflare Pages (https://pages.cloudflare.com/) where development efforts are focused instead of Workers Sites

NNeiki bump

Cyb3r-Jak3•5/24/23, 8:10 PM

Workers doesn’t have error notifications. You have to put those in your script

CCyb3r-Jak3 Workers doesn’t have error notifications. You have to put those in your script

Neiki•5/24/23, 8:18 PM

but i cant catch the "Exceeded Resources" thing, it just happens sometimes i cant really say why its weird (only happend twice so far) but i thought maybe i can let me notify or something

kian•5/24/23, 8:25 PM

https://developers.cloudflare.com/workers/platform/tail-workers/ if you're on a paid plan

Tail Workers · Cloudflare Workers docs

A Tail Worker receives information about the execution of other Workers, such as HTTP statuses, data passed to console.log() or uncaught exceptions. …

Neiki•5/24/23, 8:26 PM

thanks and yes

NNeiki thanks and yes

Neiki•5/24/23, 8:48 PM

nvm sorry i found it, sorry for ping

Original message was deleted

dave•5/24/23, 9:11 PM

Correct. It's a bit annoying to have to have different preloading implementations for Chrome vs Safari though. For ease of use, we just stick with doing preloading client side for now.

Kkian Safari is the one I'll be hoping for - Safari on iOS is a big chunk of our users

dave•5/24/23, 9:12 PM

and those are the ones with the slow interwebz who you care about usually

dave•5/24/23, 9:17 PM

@sathoro btw Supabase staff will turn off rate limiting for your project if you ask them and say it's because you're using a Worker. Don't think you were running into issues yet, but figured I should share the info.

Nick Harris•5/24/23, 9:55 PM

Is it possible to use an HTTP proxy server when making outbound fetch calls from a Worker? I have some external services that are IP-restricted, and since CF Worker IPs aren't static I need to proxy it through another server (via tunneling) to make the call.

Nick Harris•5/24/23, 10:03 PM

Ah, gotcha, so you just made requests directly to the nginx reverse proxy?

setInterval(() => { const link = document.createElement("link"); link.setAttribute("fetchpriority", "low"); link.setAttribute("rel", "preload"); link.setAttribute("href", `/geolocation.json`); link.setAttribute("as", "fetch"); link.setAttribute("crossorigin", "anonymous"); document.head.appendChild(link); }, 60000)

if i, lets say want to build some kind of "low-latency" storage for small files, that is kinda guara

Similar Threads

Similar Threads

Similar Threads