I care somewhat. Right now, to my best measure, routing everything through a Cloudflare Worker adds 80-250ms latency (as compared to routing directly to origin). This is acceptable, but 500ms wouldn't be. Our specific use-case is an SPA with API calls, so it's mostly plaintext and relatively little bandwidth.