I guess it kind of sucks that pricing ends up warping the implementation like this
I guess it kind of sucks that pricing ends up warping the implementation like this
KIt's true we need to improve the spread of objects across isolates. But at the same time, any in-memory cache implementation that caches a large data set really needs to track memory usage and prune the cache to stay within limits -- and it needs to account for the possibility of multiple objects in the same isolate, because eventually if you have enough objects, even if we evenly distribute them across the colo, some will land in the same isolate.
Tracking memory pressure is admittedly kind of hard to do in JavaScript, but not impossible -- you could count memory usage in a global variable, so that it tracks across all objects in the isolate.
Or if you rely on the built-in cache, it'll be taken care of for you. But yeah, that might incur additional billing... (I don't yet know if cache hits will count for billing, but it's possible.)
Tracking memory pressure is admittedly kind of hard to do in JavaScript, but not impossible -- you could count memory usage in a global variable, so that it tracks across all objects in the isolate.
Or if you rely on the built-in cache, it'll be taken care of for you. But yeah, that might incur additional billing... (I don't yet know if cache hits will count for billing, but it's possible.)
JIsn't the unpredictable sharing of objects within a single isolate a larger problem than just storage caching scenarios tho? You have the same problem without storage in the picture. Any data structures allocated by the app (ws-connection-level stuff, encryption artifacts, things coming in from input or external fetch) are going to use memory, and it won't be good if your scenario is architected to work in dev on Monday, runs fine in production on Tuesday, but then fails on Wednesday because more object instances are now put into the same isolate. Say what you want about lambda, I've used it from the very beginning, and one nice property is that it's predictable about what you get in the container you specify, and you can architect within the advertised limits.
KSo, an important design goal of durable objects is that it should scale to extremely large numbers of fine-grained objects. We strongly encourage developers to design for finer granularity, and we intentionally don't bill on the number of objects because we'd rather see people building apps around more, smaller objects.
If we said that each object gets its own isolate, that would unfortunately blow up this goal, since even though isolates are much cheaper than containers, they are still much more expensive than what we'd like to see for fine-grained objects.
What we really want to be able to do is adaptively adjust the isolate count to match what the app needs, so apps don't need to worry about it. We're still working on that, though, and it's a lot harder for large / coarse-grained objects compared to fine-grained.
If we said that each object gets its own isolate, that would unfortunately blow up this goal, since even though isolates are much cheaper than containers, they are still much more expensive than what we'd like to see for fine-grained objects.
What we really want to be able to do is adaptively adjust the isolate count to match what the app needs, so apps don't need to worry about it. We're still working on that, though, and it's a lot harder for large / coarse-grained objects compared to fine-grained.
KSame code should work in durable objects.
KYour stateless worker can directly forward a request it received to a DO, like where
requestjust
Kthat's fine, streaming will work
KKThe ReadableStream interface of the Streams API represents a readable stream of byte data. The Fetch API offers a concrete instance of a ReadableStream through the body property of a Response object.
KYes, the CPU limit only get refreshed each time a new request arrives. Stream chunks don't count and new requests. Currently the limit is 500ms, but will be extended to 30s when we start billing -- maybe that's enough?
WI did a similar math on my side and im keeping the memory cache for the same reason. I read 100-1000x more from memory than from storage accounting everytime the object is (for any reason) reinitialized. If read from cache can have a lower price tier i will probably remove my memory map.... apps are constrained by cost all the time, thats not a surprise.
WThats explain few erros that i got... Nice to know!