Is there a way to check from which specific datacenter a cache response comes back from?

Llukas Is there a way to check from which specific datacenter a cache response comes ba...

C

Cyb3r-Jak3•12/12/23, 10:18 PM

For cache API it would be whatever datacenter your request hit, which you can get from the ray-id response header

L

lukasOP•12/12/23, 10:55 PM

Other than enabling tiered cache, is there anything you need to do within the worker? i.e setting up zones or something, etc

K

kian•12/12/23, 10:57 PM

The Cache API isn't compatible with Tiered Cache - only the cache used by

fetch()

fetch()

subrequests.

C

Constantin•12/12/23, 11:13 PM

i'm using wrangler.json, and i'm having some questions about how can github secrets can be referenced during the pipeline. is this a correct way of doing this?

K

kian•12/12/23, 11:14 PM

Wrangler.toml/.json don't do any text replacement of their own - it's just passed to TOML.parse/JSON.parse

K

kian•12/12/23, 11:14 PM

You'd be better off passing stuff like that to the CLI, i.e

wrangler deploy --var FOO:$BAR

wrangler deploy --var FOO:$BAR

wrangler deploy --var FOO:$BAR

wrangler deploy --var FOO:$BAR

L

lukasOP•12/12/23, 11:48 PM

Man, the cache hit latency fluctuates so much, two cache hits to the same data center from the same location resulted in the first being 7 latency, and the other 55 latency. huh

I

Isaac McFadyen•12/12/23, 11:51 PM

It's possible that you're not actually hitting the same datacenter, given that there can be multiple datacenters for a single airport code. For example, Chicago (ORD) has at least 4 (and probably more).

L

lukasOP•12/12/23, 11:51 PM

ah gotcha

C

Constantin•12/12/23, 11:57 PM

@kian so, having them like this seems not to make them available to wrangler

name: Build
on:
  push:
    branches:
      - main
      - develop
env:
  API_URL: ${{ vars.API_URL }}
  CF_Access_Client_Id: ${{ vars.CF_Access_Client_Id }}
  CF_Access_Client_Secret: ${{secrets.CF_Access_Client_Secret}}  

jobs:
  deploy:
    runs-on: ubuntu-latest
    name: Deploy
    steps:
      - uses: actions/checkout@v4
      - name: Deploy
        uses: cloudflare/wrangler-action@v3
        with:
          apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
          command: deploy --env=${{ github.ref_name }} --experimental-json-config

name: Build
on:
  push:
    branches:
      - main
      - develop
env:
  API_URL: ${{ vars.API_URL }}
  CF_Access_Client_Id: ${{ vars.CF_Access_Client_Id }}
  CF_Access_Client_Secret: ${{secrets.CF_Access_Client_Secret}}  

jobs:
  deploy:
    runs-on: ubuntu-latest
    name: Deploy
    steps:
      - uses: actions/checkout@v4
      - name: Deploy
        uses: cloudflare/wrangler-action@v3
        with:
          apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
          command: deploy --env=${{ github.ref_name }} --experimental-json-config

name: Build
on:
  push:
    branches:
      - main
      - develop
env:
  API_URL: ${{ vars.API_URL }}
  CF_Access_Client_Id: ${{ vars.CF_Access_Client_Id }}
  CF_Access_Client_Secret: ${{secrets.CF_Access_Client_Secret}}  

jobs:
  deploy:
    runs-on: ubuntu-latest
    name: Deploy
    steps:
      - uses: actions/checkout@v4
      - name: Deploy
        uses: cloudflare/wrangler-action@v3
        with:
          apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
          command: deploy --env=${{ github.ref_name }} --experimental-json-config

name: Build
on:
  push:
    branches:
      - main
      - develop
env:
  API_URL: ${{ vars.API_URL }}
  CF_Access_Client_Id: ${{ vars.CF_Access_Client_Id }}
  CF_Access_Client_Secret: ${{secrets.CF_Access_Client_Secret}}  

jobs:
  deploy:
    runs-on: ubuntu-latest
    name: Deploy
    steps:
      - uses: actions/checkout@v4
      - name: Deploy
        uses: cloudflare/wrangler-action@v3
        with:
          apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
          command: deploy --env=${{ github.ref_name }} --experimental-json-config

they must be passed via var flag?

Kkian The Cache API isn't compatible with Tiered Cache - only the cache used by `fetch...

L

lukasOP•12/13/23, 12:04 AM

How would that look like within a cloudflare worker? I have tiered cache enabled, and at the current level, i have a

cache

cache

cache

cache class that uses

caches.default.put

caches.default.put

caches.default.put

caches.default.put to set the cached

request / response

request / response

request / response

request / response.

so if its not in

caches.default

caches.default

then it fetches the origin, where does the higher tier come in here? is it done automatically?

K

kian•12/13/23, 12:10 AM

There isn't any -

caches.default

caches.default

(or

caches.open("...")

caches.open("...")

caches.open("...")

caches.open("...")) doesn't use Tiered Cache.

K

kian•12/13/23, 12:10 AM

It just talks to the cache of the colo the Worker is being run in.

Kkian It just talks to the cache of the colo the Worker is being run in.

L

lukasOP•12/13/23, 12:12 AM

so, CF determines which datacenter to run the worker in thats closest to the end user, and if the "colo" does not have the needed cache, then it will request from the origin? Is that how it works?

Llukas so, CF determines which datacenter to run the worker in thats closest to the end...

I

Isaac McFadyen•12/13/23, 12:13 AM

cache.match

cache.match

cache.match

cache.match checks for a cached value. If it's in the cache, it returns the value, otherwise it returns

null

null

. Your Worker does all of the fetching and inserting (

cache.put

cache.put

cache.put

cache.put) into the cache itself.

IIsaac McFadyen `cache.match` checks for a cached value. If it's in the cache, it returns the va...

L

lukasOP•12/13/23, 12:15 AM

is it looking at the

colo

colo

colo

colo the worker is being run in, or is it checking the global cache?

i'm just trying to figure out how to lower my latency, cuz im getting 100ms+ for requests outside of the origin, even though they're resulting in a 'cache hit'

Llukas is it looking at the `colo` the worker is being run in, or is it checking the gl...

I

Isaac McFadyen•12/13/23, 12:15 AM

There is no global cache. It's checking the cache of the colo the Worker is run in.

I

Isaac McFadyen•12/13/23, 12:15 AM

Even tiered caching isn't "global" per-say. But in your case Tiered Cache isn't applicable anyway.

I

Isaac McFadyen•12/13/23, 12:16 AM

Are you doing any other processing in your Worker that might cause high response times?

I

Isaac McFadyen•12/13/23, 12:16 AM

And can you confirm that the response from

cache.match

cache.match

cache.match

cache.match is actually being returned and the origin isn't being hit anyway?

IIsaac McFadyen There is no global cache. It's checking the cache of the colo the Worker is run ...

L

lukasOP•12/13/23, 12:16 AM

ok so thats what i was wondering where its requesting that cache

L

lukasOP•12/13/23, 12:17 AM

ill respond to those in a few, gotta go outside before the sun goes down lol

I

Isaac McFadyen•12/13/23, 12:17 AM

Yeah. The colo that the Worker is run in. There may be multiple different colos in a city, and they don't share cache currently.

IIsaac McFadyen Are you doing any other processing in your Worker that might cause high response...

L

lukasOP•12/13/23, 1:07 AM

This the workflow of whats going on in my app

endpointCreate -> D1, take the created record, turn it into a response, and cache it

endpointCheck -> check `caches.default.match`, if response exists for cacheKey return endpoint response based off that cache, sync both D1 and cache via executionContext.waitUntil(update D1, update cache)

endpointCreate -> D1, take the created record, turn it into a response, and cache it

endpointCheck -> check `caches.default.match`, if response exists for cacheKey return endpoint response based off that cache, sync both D1 and cache via executionContext.waitUntil(update D1, update cache)

endpointCreate -> D1, take the created record, turn it into a response, and cache it

endpointCheck -> check `caches.default.match`, if response exists for cacheKey return endpoint response based off that cache, sync both D1 and cache via executionContext.waitUntil(update D1, update cache)

endpointCreate -> D1, take the created record, turn it into a response, and cache it

endpointCheck -> check `caches.default.match`, if response exists for cacheKey return endpoint response based off that cache, sync both D1 and cache via executionContext.waitUntil(update D1, update cache)

So majority of the time, i am working off of the cached result, and then updating d1 and cache with fresh data

So, based on what I understand now, CF argo routing will determine which

colo

colo

colo

colo to invoke the worker, in which the worker checks the cache of that

colo

colo

colo

colo and if its not there it routes back to the origin to retrieve the data

L

lukasOP•12/13/23, 1:08 AM

Oh and computation is very little, its mainly my caching layer that I'm trying to optimize here

L

lukasOP•12/13/23, 1:10 AM

With tiered caching, If end user lives on the east, and the

colo

colo

colo

colo on the east coast does not have the cache, it would retrieve it from another

colo

colo

colo

colo nearby that might have the cache, and if not, it would go to the origin?

Llukas Oh and computation is very little, its mainly my caching layer that I'm trying t...

H

Hello, I’m Allie!•12/13/23, 7:34 AM

User to Worker Routing is handled by BGP, not Argo

H

Hello, I’m Allie!•12/13/23, 7:34 AM

Basically, the nearest colo to the user(according to their ISP) is chosen

H

Hello, I’m Allie!•12/13/23, 7:34 AM

And according to your plan

B

bun•12/13/23, 11:42 AM

getting

crypto.pbkdf2Sync is not a function

crypto.pbkdf2Sync is not a function

crypto.pbkdf2Sync is not a function

crypto.pbkdf2Sync is not a function when using https://www.npmjs.com/package/cryptr on workers

npm

cryptr

a simple encrypt and decrypt module for node.js. Latest version: 6.3.0, last published: 3 months ago. Start using cryptr in your project by running

npm i cryptr

npm i cryptr

npm i cryptr

npm i cryptr. There are 309 other projects in the npm registry using cryptr.

B

bun•12/13/23, 11:43 AM

not sure how to fix it

Bbun getting `crypto.pbkdf2Sync is not a function` when using https://www.npmjs.com/p...

E

Erisa•12/13/23, 11:50 AM

Based on the code and that error I suspect its trying to use Node.js crypto, which in Workers requires the

nodejs_compat

nodejs_compat

compatability flag https://developers.cloudflare.com/workers/runtime-apis/nodejs/crypto/ but also probably still wont work without a modification to the library because it

require

require

require

requires

crypto

crypto

crypto

crypto instead of

node:crypto

node:crypto

node:crypto

node:crypto

But once those two hurdles are surpassed,

pbkdf2Sync

pbkdf2Sync

pbkdf2Sync

pbkdf2Syncis listed as supported in our implementattion of Node.js crypto

B

bun•12/13/23, 11:51 AM

B

bun•12/13/23, 11:51 AM

so change

const crypto = require('crypto');

const crypto = require('crypto');

const crypto = require('crypto');

const crypto = require('crypto'); to

import crypto from "node:crypto"

import crypto from "node:crypto"

import crypto from "node:crypto"

import crypto from "node:crypto"?

E

Erisa•12/13/23, 11:52 AM

Something like that yeah, sounds about right

E

Erisa•12/13/23, 11:52 AM

It also uses Buffer so you might need to do something for that if its mandatory https://developers.cloudflare.com/workers/runtime-apis/nodejs/buffer/

B

bun•12/13/23, 11:52 AM

noted

E

Erisa•12/13/23, 11:53 AM

And the compatibility flag to unlock the

node:

node:

node:

node: stuff is just

compatibility_flags = [ "nodejs_compat" ]

compatibility_flags = [ "nodejs_compat" ]

compatibility_flags = [ "nodejs_compat" ]

compatibility_flags = [ "nodejs_compat" ]

in your wrangler.toml

B

bun•12/13/23, 11:53 AM

oh i didthis

B

bun•12/13/23, 11:53 AM

thats what the error message said to use

E

Erisa•12/13/23, 11:54 AM

The error message lies

B

bun•12/13/23, 11:54 AM

E

Erisa•12/13/23, 11:54 AM

node_compat = true

node_compat = true

node_compat = true

node_compat = true is the old implementation

E

Erisa•12/13/23, 11:55 AM

methinks the error message needs to be updated, I should check if theres an existing issue on workers-sdk

B

bun•12/13/23, 11:55 AM

rip i dont think

createDecipheriv

createDecipheriv

createDecipheriv

createDecipheriv works

E

Erisa•12/13/23, 11:55 AM

B

Bogdan•12/13/23, 3:45 PM

i think i might have posted in the wrong channel: wranglerWebsocket using correctly

B

Bogdan•12/13/23, 3:45 PM

some help would be appreciated

Is there a way to check from which specific datacenter a cache response comes back from?

Similar Threads

Similar Threads

Similar Threads