Workers cache overhead?
Hi, I am trying to figure out the best way to optimize response times for cached resources served from a worker. With worker + cache API, I am getting response times of about 200ms pretty consistently for an image. The same image, when served from cloudfront, is about 50-70ms. Both responses appear to be coming from IAD. Is the extra 150ms a result of worker overhead? Is there any way to improve this? (My application is an interactive medical imaging viewer, so load times are really important)
22 Replies
Sounds more like routing or missing cache. Workers are a few ms extra at best, they run on the same machine processing your request. Cache is ~5-15ms. If you're directly returning the result of cache match, it should have a cf-cache-status: hit response header as part of it
Yes, the requests do have cf-cache-status: HIT - that's why i'm confused
What do you see in the cf-ray? IAD, and you're on the east coast near Ashburn?
Cf-Ray is 96fb56bb29b03b8c-IAD, I'm in harrisonburg VA so yes about 2 hrs from DC by car
The cloudfront responses are also being served from IAD
If it helps to have something to compare to, simple cache test: https://new-quick-cache-test.workers.chaika.me/, send a request or two and look for cf-cache-status: HIT to see pure cache latency, without cache, raw worker response w/o cache (should be faster): https://new-quick-cache-test.workers.chaika.me/?no-cache
I'm guessing you can't share a repro url. If that's any better then your worker, I'd ask what your worker is doing/fetching from on cache misses. If you're doing logic before cache checks or have a really big bundle/lots of deps, worth checking cpu time & wall time on your metrics overview page of the worker
I just checked a log for a request, total request time was 220ms, worker wall time 27, cpu time 4
seems like the worker is fast, not sure where the rest of the time is getting spent
Your cache test page gives more like 50-100ms for both
I am decrypting a jwt prior to the cache lookup, but given the reported wall time i don't think that's the issue
You can try this url to try and reproduce:
I mean it's not like super slow, I'm getting 150-250ms for cache hits, but i feel like it should be faster
Cache hits should be basically your latency + 20-30ms. My latency to Cloudflare is ~25ms or so, I get some hits on the cache test link between ~47ms-60ms or so
That link doesn'twork, just says invalid. Worth noting though that if you're serving big enough of a response your internet speed is going to start playing a role as well
Whoops, wrong link format:
Actually, try this one, this just returns 3KB of json:
I just noticed that even with the same payload, compared to cloudfront it was a pretty sizeable difference.
~40-50ms for me when cache hit, ~80ms or so on first without cf-cache-status header
If you just ping
litevna.app, what's your avg latency?
Super fast:
round-trip min/avg/max/stddev = 10.738/14.979/20.725/2.494 ms
wth
What are your response headers on a slower (150ms+) response? Do you have anything like cf-placement?
No:
sorry that formatting is horrendous
I'm getting some responses in the 50ms range, but most 150.
The connection close bit you have on there is curious, what are you testing from?
That particular one i think was from HTTPie macos client
but most testing in chrome/arc
I see the same header in chrome as well
ah ok, yea browsers or curl are usually best to test in, rest clients/etc aren't quite meant to be performance benchmarks, that one looked like it was closing the connection each time
It looks like your cache ttl is really low, like 30s or so, if possible I would bump it up a fair bit more so there's more chance for hits and less surprise cold hits
I guess if you’re getting 50 ms that’s good news, as long as my users get fast loads
With a warmed up connection and hitting cache, it's possible to get low enough latency on that url even from IAD
IAD (CF IAD) 38.29ms 200 Cf-Cache-Status -> HIT
IAD (CF IAD) 41.55ms 200 Cf-Cache-Status -> HIT
That particular path is a query path where data is more likely to change. That larger payload link (the first one) is longer with unchanging data
What are you getting on that first link (the one that returns binary data)? Still fast?
From my device on a residental connection ~45ms
From a server in IAD ~1ms away from Cloudflare
IAD (CF IAD) 24.02ms 200 Cf-Cache-Status -> HIT
IAD (CF IAD) 22.1ms 200 Cf-Cache-Status -> HIT
Not bad
Dang not bad at all. Must be something weird with my setup/connection
Maybe my ISP? Idk. Thanks for your help!
Whenever perf testing always do what a browser would, so many people spin up new connections for every request, use http1.1, etc.
There's a lot that browsers/servers do these days to ensure things are super fast
That makes sense. I also directly bound two workers in the request chain instead of using fetch, and reworked the request pattern to avoid a server hit (which was used to get around a cors preflight) and now I’m seeing lower latency as well. Very much appreciate the help!