Also native fetch cache will support tiered cache so bonus hit rate

EErisa Also native fetch cache will support tiered cache so bonus hit rate

D

dave•12/29/23, 1:56 AM

ah yes, that brings up another question! For my use-case, caching S3 subrequests to a single Backblaze, do these tier cache settings make sense?

Ddave ah yes, that brings up another question! For my use-case, caching S3 subrequests...

D

dave•12/29/23, 1:56 AM

and how can I confirm if tiered caching is actually doing anything?

Ddave ah yes, that brings up another question! For my use-case, caching S3 subrequests...

E

ErisaOP•12/29/23, 1:57 AM

If your goal is to minimise requests to Backblaze then yes thats perfect. If your goal was to minimise global response times and you had a ton of traffic from all around the world then Generic might be slightly better.

EErisa If your goal is to minimise requests to Backblaze then yes thats perfect. If you...

D

dave•12/29/23, 1:58 AM

it's the former, ty!

Ddave and how can I confirm if tiered caching is actually doing anything?

E

ErisaOP•12/29/23, 1:58 AM

CacheTieredFill

CacheTieredFill

CacheTieredFill

CacheTieredFill in logs will tell you whether it was used for a given request, you may need to inspect logs over an extended period before you find one because if you keep hitting the same colo its just going to serve from local colo cache

E

ErisaOP•12/29/23, 1:59 AM

some ent customers will use logpush to send to data visualisation software and then graph the tiered cache fills over time

D

dave•12/29/23, 2:00 AM

ahhh neat, I see it now!

EErisa `CacheTieredFill` in logs will tell you whether it was used for a given request,...

D

dave•12/29/23, 2:02 AM

so right now I am running a script locally that's hitting my Worker (that does requests to B2's S3 API).

The Worker is running in MIA because that's the closest to me; when another service (i.e. Google Cloud Run) in IAD requests the same URL that was cached by my requests, will it:

Pull the MIA cache to IAD
Not use the MIA cache at all, instead going to origin and saving that in IAD
Something else?

E

ErisaOP•12/29/23, 2:03 AM

It depends on what toplogy its using (which for smart is dynamically determined), I couldnt speak to it with any certainty

E

ErisaOP•12/29/23, 2:04 AM

But if MIA was one of the colos in the toplogy and it was working optimally then yes it would pull it

E

ErisaOP•12/29/23, 2:05 AM

You may also see in logs a

UpperTierColoID

UpperTierColoID

UpperTierColoID

UpperTierColoID which will tell you which other colo the cache was fetched from, if not the local one

D

dave•12/29/23, 2:07 AM

This is B2's IP. So does that mean my requests are already being cached in IAD?

E

ErisaOP•12/29/23, 2:09 AM

Yeah. IAD will store cache and other colos will request copies from IAD if they dont have any locally

E

ErisaOP•12/29/23, 2:10 AM

A generic topology would store the cache in more places but would need more origin requests in order to fill them

E

ErisaOP•12/29/23, 2:11 AM

Also worth noting since you have regional tier enabled its a lot more nuanced than that, theres various tiers in the middle before it gets to IAD

E

ErisaOP•12/29/23, 2:12 AM

as a simplified example if you hit a colo on the west coast of the US and it didnt have cache but a regional tier on the west coast did, it can avoid a request to IAD all the way over in the east by taking it from that nearby regional tier

E

ErisaOP•12/29/23, 2:13 AM

which keeps latency down

E

ErisaOP•12/29/23, 2:14 AM

Smart + Regional is a very powerful combination if your goal is to maximise cache hit rate

EErisa Smart + Regional is a very powerful combination if your goal is to maximise cach...

D

dave•12/29/23, 2:19 AM

nice! from what you described, sounds like I'm worrying bout nothing

EErisa cf-worker will never work in rules

D

dave•12/29/23, 2:21 AM

Is there really no way to detect if something is or isn't a Worker subrequest?

E

ErisaOP•12/29/23, 2:21 AM

I mean I would have expected

cf.worker.upstream_zone

cf.worker.upstream_zone

cf.worker.upstream_zone

cf.worker.upstream_zone to be supported but if it isnt then no

E

ErisaOP•12/29/23, 2:21 AM

That property exists to be used in lieu of cf-worker

EErisa Did a quick test to confirm and it impacts serving as well, if you dont match th...

D

dave•12/29/23, 2:45 AM

Another question! If I set

"Cache-Control": "no-cache, no-store, no-transform"

"Cache-Control": "no-cache, no-store, no-transform"

"Cache-Control": "no-cache, no-store, no-transform"

"Cache-Control": "no-cache, no-store, no-transform" on my fetch request in a Worker, will that bypass the Cache Rules?

D

dave•12/29/23, 2:51 AM

✘ [ERROR]   Error: Promise will never complete.

✘ [ERROR]   Error: Promise will never complete.

✘ [ERROR]   Error: Promise will never complete.

✘ [ERROR]   Error: Promise will never complete.

What logic triggers this? I have a setTimeout used for polling that increases on every attempt.

K

kian•12/29/23, 2:56 AM

Most of the errors around scripts never returning a response or promises never completing means there’s stuff being used in the global scope

K

kian•12/29/23, 2:57 AM

The runtime can only see what was created in that request - so if Request B is depending on a global Promise from Request A then it sees that nothing is left in Request B yet it hasn’t returned a response.

K

kian•12/29/23, 2:57 AM

That’s usually a different error though, “script will never return a response” or similar.

Kkian Most of the errors around scripts never returning a response or promises never c...

D

dave•12/29/23, 3:13 AM

odd... I added some more logging, maybe I missed a throw or something on my promise.

D

dave•12/29/23, 3:13 AM

didn't think I had anything in my globals that would cause it

Kkian Nope

D

dave•12/29/23, 3:22 AM

thank you, it's working great for all my JSON files in this bucket!

Pretty big data savings, since this is/was all uncompressed text before.

K

kian•12/29/23, 3:23 AM

I guess a theoretical downside that isn’t specific to S3 or R2 is that you might lose Content-Length if on-the-fly decompression is needed.

K

kian•12/29/23, 3:23 AM

If you don’t need that header then it’s fine.

Kkian Most of the errors around scripts never returning a response or promises never c...

D

dave•12/29/23, 3:24 AM

Could it be a OOM error somehow?

D

dave•12/29/23, 3:25 AM

actually I can test this, gonna try the same request on wrangler locally

D

dave•12/29/23, 3:26 AM

✘ [ERROR] *** Received signal #11: Segmentation fault: 11

  stack:


✘ [ERROR] Error in ProxyController: Error inside ProxyWorker

   {
    name: 'Error',
    message: 'Network connection lost.',
    stack: 'Error: Network connection lost.'
  }

✘ [ERROR] *** Received signal #11: Segmentation fault: 11

  stack:


✘ [ERROR] Error in ProxyController: Error inside ProxyWorker

   {
    name: 'Error',
    message: 'Network connection lost.',
    stack: 'Error: Network connection lost.'
  }

✘ [ERROR] *** Received signal #11: Segmentation fault: 11

  stack:


✘ [ERROR] Error in ProxyController: Error inside ProxyWorker

   {
    name: 'Error',
    message: 'Network connection lost.',
    stack: 'Error: Network connection lost.'
  }

✘ [ERROR] *** Received signal #11: Segmentation fault: 11

  stack:


✘ [ERROR] Error in ProxyController: Error inside ProxyWorker

   {
    name: 'Error',
    message: 'Network connection lost.',
    stack: 'Error: Network connection lost.'
  }

D

dave•12/29/23, 3:26 AM

wat

D

dave•12/29/23, 3:27 AM

how... how did a network connection lost cause a segfault?

Kkian Most of the errors around scripts never returning a response or promises never c...

D

dave•12/29/23, 3:31 AM

I think it's an OOM. It's trying to download a 70MB JSON file, parse it, compress it, and stringify it again lol

K

kian•12/29/23, 3:37 AM

D

dave•12/29/23, 3:40 AM

yeeeeeeeaaaah.

D

dave•12/29/23, 3:47 AM

aw fudge

D

dave•12/29/23, 3:47 AM

How am I gonna tee the thing..

D

dave•12/29/23, 3:53 AM

ah, looks like it's okay? https://github.com/cloudflare/workerd/blob/main/src/workerd/api/streams/README.md#what-about-tees

GitHub

workerd/src/workerd/api/streams/README.md at main · cloudflare/work...

The JavaScript / Wasm runtime that powers Cloudflare Workers - cloudflare/workerd

K

kian•12/29/23, 4:09 AM

Depends what you’re trying to do - it generally only avoids memory usage when you’re consuming streams that are from outside of your Worker

K

kian•12/29/23, 4:09 AM

like sending an R2 object to the client and to cache

K

kian•12/29/23, 4:15 AM

like, when you parse it, how are you compressing it?

Kkian like, when you parse it, how are you compressing it?

D

dave•12/29/23, 4:16 AM

uh it's pretty bad code tbh..

const operation_json = await operation_result.json() as any;

if(operation_json.result === "win") {
  const filename = cacheKey.pathname.split("/").pop() ?? "undefined.pdf";
  const filename_json = filename.replace(/\.[^/.]+$/, "") + ".json";

  const operation_json_str = JSON.stringify(operation_json, null, 2);
  const custom_res = new Response(operation_json_str);

  const compressedReadableStream = custom_res.body!.pipeThrough(
    new CompressionStream('gzip')
  );

  // ...
}

const operation_json = await operation_result.json() as any;

if(operation_json.result === "win") {
  const filename = cacheKey.pathname.split("/").pop() ?? "undefined.pdf";
  const filename_json = filename.replace(/\.[^/.]+$/, "") + ".json";

  const operation_json_str = JSON.stringify(operation_json, null, 2);
  const custom_res = new Response(operation_json_str);

  const compressedReadableStream = custom_res.body!.pipeThrough(
    new CompressionStream('gzip')
  );

  // ...
}

const operation_json = await operation_result.json() as any;

if(operation_json.result === "win") {
  const filename = cacheKey.pathname.split("/").pop() ?? "undefined.pdf";
  const filename_json = filename.replace(/\.[^/.]+$/, "") + ".json";

  const operation_json_str = JSON.stringify(operation_json, null, 2);
  const custom_res = new Response(operation_json_str);

  const compressedReadableStream = custom_res.body!.pipeThrough(
    new CompressionStream('gzip')
  );

  // ...
}

const operation_json = await operation_result.json() as any;

if(operation_json.result === "win") {
  const filename = cacheKey.pathname.split("/").pop() ?? "undefined.pdf";
  const filename_json = filename.replace(/\.[^/.]+$/, "") + ".json";

  const operation_json_str = JSON.stringify(operation_json, null, 2);
  const custom_res = new Response(operation_json_str);

  const compressedReadableStream = custom_res.body!.pipeThrough(
    new CompressionStream('gzip')
  );

  // ...
}

K

kian•12/29/23, 4:18 AM

gotcha - I didn’t know how it’d work without stringifying before compression but that’s what you’re doing

D

dave•12/29/23, 4:18 AM

the thing is I'd need to know the size of the compressed gzip object to be able to upload it via S3 anyway

D

dave•12/29/23, 4:18 AM

so gzip for huge objects is just never gonna work for me here

Also native fetch cache will support tiered cache so bonus hit rate

Similar Threads

Similar Threads

Similar Threads