Cloudflare Developers•3mo ago

Output gate and async operations

Why are you concerned about the output gate I'm confused. Can you show a more accurate example? I don't understand the purpose of the psuedo-code you posted, it doesn't do anything

45 Replies

João Castro•3mo ago

Its just an example that showcases a general scenario of wanting to executing something asynchronously while ignoring the output gate Today that is not very practical You can work around it, but its going to be a pain to manage

MarakOP•3mo ago

Yes, but it fires off a bunch of async requests without waiting for them using a fire and forget. If you don't need the results of these requests why does it matter about the storage API calls? I feel like you may be inventing a problem where there is none; I am not sure. I'm curious what the actual issue is. Happy to help if I can.

João Castro•3mo ago

And I think something like this can be weird with the current API

async something(a) {
  await fetch("https://foo.example/" + a);
  await fetch("https://bar.example/" + a);
  await fetch("https://foo.example/bar/" + a);
}

await this.ctx.storage.put('a', 'b', {
  allowUnconfirmed: true
});

something('a');
await this.ctx.storage.put('a', 'b');

async something(a) {
  await fetch("https://foo.example/" + a);
  await fetch("https://bar.example/" + a);
  await fetch("https://foo.example/bar/" + a);
}

await this.ctx.storage.put('a', 'b', {
  allowUnconfirmed: true
});

something('a');
await this.ctx.storage.put('a', 'b');

I could await everything except something, if that helps Same problem

MarakOP•3mo ago

what is the point of storing the same data a and b twice?

João Castro•3mo ago

It doesn't matter Just random values to call something that can close the output gate Its just an example to show some specific behavior It doesn't need to be an actual production code, as the point is to exemplify a more general scenario that people can get into

MarakOP•3mo ago

Why are you not awaiting something? If the results of something() aren't relevant to the what is being stored why put it inside that code path at all?

João Castro•3mo ago

Because I don't want them to block execution

MarakOP•3mo ago

If you are making requests to outside world and you don't need the results, it might be better to move that code outside of the durable object method invocation? Especially if you aren't needing to wait for them?

João Castro•3mo ago

Regardless if you will need the results inside the DO at that moment, you could want to send something somewhere That can be an operation that takes a couple hundred ms And I don't need to wait for the response

MarakOP•3mo ago

I would suggest it may be better design to let your durable object RPC methods only deal with data and let them return to a worker which may make outside requests.

João Castro•3mo ago

You can't think of any use cases where you might talk to the external world with data being unconfirmed? Then why bother having allowUnconfirmed at all? Its an API to do the same thing Just that it is more limited Needing to use the result of the operation is irrelevant here

MarakOP•3mo ago

We've mostly designed our durable object methods to only talk data and not make outside requests if possible. Anywhere there is an outside request we await

João Castro•3mo ago

And there are scenarios that awaiting is unnecessary

MarakOP•3mo ago

If that's the case it shouldn't need to be in the same code path, I am just not sure. Maybe someone else can help or have better insight. A working example showing the issue may help

João Castro•3mo ago

If awaiting and blocking execution every time was the "right" way to do it, we wouldn't need async functions In a lot of cases blocking is the best approach, in some others it is not And in the cases where it isn't the right approach, the current { allowUnconfirmed: true } API might cause some headaches Because you will have some steps ignoring the output gate, and some others might be blocked again. In this scenario there isn't much you can do

MarakOP•3mo ago

If I understand correctly ( which I don't because I don't really understand the contrived example ), you can just place the fire and forget calls outside the scope of the function you are managing state in. If there was a real-world example I might understand better. Have you hit an issue in production? Or in testing real code?

João Castro•3mo ago

Its just an issue of mutability really, same thing about why you want pure functions You pass the parameters and things happens in a specific way Here you are relying on the state of the output gate It can change over time, and it will affect what is happening inside that function execution I want a way to run a function knowing that it will not be affected by this external state Currently I need to make sure that nothing else will mutate this state while its executing I would need to place them outside of the durable object It would be a workaround, but it would require me to place code in a different place, make an rpc call to it when it could be just a small function inside the DO

MarakOP•3mo ago

I think it's generally better to keep code for communicating with the outside world inside the worker, keep the DO's working purely with data when possible.

João Castro•3mo ago

Yes, currently that would be the case It doesn't need to be this way

MarakOP•3mo ago

It would probably cost more money since the durable objects charge for duration and workers only charge for compute?

João Castro•3mo ago

Not necessarily because the async operations could be happening while some other stuff would be keeping the DO busy anyways But yeah for very long running IO it would probably be better to have it somewhere else

MarakOP•3mo ago

That's what I'm saying, workers won't charge for inactive time, only compute

João Castro•3mo ago

Sure, but again, a lot of the time this could be a non-issue In some cases it would definitely be better to have it somewhere else I'm not saying that there isn't cases where the current API is enough I'm proposing something that could be just as good or better for the current cases where it works, and also handle more scenarios with ease

MarakOP•3mo ago

I would stick to keeping the durable object methods working purely with data when possible and move as much I/O to the worker as you can. I think this is the preferred design method

João Castro•3mo ago

Sure. I wouldn't want to be required to do this if a simpler solution was possible

MarakOP•3mo ago

I think it's a matter of what the DOs are designed to do. The first few iterations of our chat system had a bunch of I/O and HTTP inside the DO. It wasn't very good. When we migrated the I/O and HTTP outside of the DO into the worker; everything started flowing a lot more smoothy. It reduced total time of requests by a significant amount. Let's see what someone else has to say. I'm no expert on the subject matter.

João Castro•3mo ago

Yeah I'm not arguing that this can't be good a lot of the times But thats the thing, allowUnconfirmed is an escape hatch for more specific scenarios If you ignore the specific scenarios where it can be useful, then it really doesn't make sense for it to exist at all Regardless of the design of the api

MarakOP•3mo ago

I think allowUnconfirmed is meant for situations where you need to immediately broadcast the data before it's confirmed to be saved, low-latency situations where you can sync after the fact. Like saving a message for a chat room using allowUnconfirmed, immediately broadcasting the message to the clients, then running sync.

async handleMessage(message, clientId) {
    // Store message with allowUnconfirmed to reduce latency
    await this.state.storage.put(`message:${Date.now()}`, message, { allowUnconfirmed: true });

    // Broadcast to all connected WebSocket clients
    for (const [id, ws] of this.clients) {
      if (id !== clientId) {
        ws.send(JSON.stringify({ type: 'message', content: message }));
      }
    }

    // Optionally, ensure persistence later
    await this.state.storage.sync(); // Ensure all writes are persisted
  }

async handleMessage(message, clientId) {
    // Store message with allowUnconfirmed to reduce latency
    await this.state.storage.put(`message:${Date.now()}`, message, { allowUnconfirmed: true });

    // Broadcast to all connected WebSocket clients
    for (const [id, ws] of this.clients) {
      if (id !== clientId) {
        ws.send(JSON.stringify({ type: 'message', content: message }));
      }
    }

    // Optionally, ensure persistence later
    await this.state.storage.sync(); // Ensure all writes are persisted
  }

João Castro•3mo ago

If you run handleMessage without awaiting you can block some of the ws messages with another write that can happen right after put finishes executing

handleMessage('something', 'something');
await this.state.storage.put('a', 'b');

handleMessage('something', 'something');
await this.state.storage.put('a', 'b');

MarakOP•3mo ago

Why would I not await handleMessage? clearly it's waiting for the sync at the bottom?

João Castro•3mo ago

Its a function that can be awaited or not If you could place the ws.send calls inside a allowUnconfirmed() function, this would never be a problem Regardless of how you call the function Its just better

MarakOP•3mo ago

allowUnconfirmed is a boolean argument, not a function

João Castro•3mo ago

I know, I'm proposing a method that allows you to pass a callback that ignores the output gate. this.allowUnconfirmed(() => ws.send()); With this api the scenario that i described wouldn't give you unexpected behavior

MarakOP•3mo ago

I would suggest building some code and seeing what works. Our first iteration tried to make assumptions outside of the durable object design and it didn't work out super well. It took a few weeks to figure out the paradigm.

João Castro•3mo ago

I know how it works right now, I'm proposing a change to how it works Sometimes there isn't really much to do because of the tradeoffs that were chosen, but other times things can still be improved I think this is something that can be improved

lambrospetrou•3mo ago

I might be missing something but if what you want is "fire and forget" actions, you can just store the promise of your something() in an in-memory variable inside the DO and it will keep running in the background. Note that if the DO doesn't receive any incoming request for 10s, it will hibernate though. If that action remotely needs more time, you can use setTimeout/setInterval to keep it alive for more. But, even then, if there is no incoming request/alarm handler within 1-2 minutes the DO will be evicted. We use the fire and forget approach as well for analytics or other background tasks, so it depends how long those operations take for you. Storing the promise is for being able to get the result later if you need and not to garbage collect it.

João Castro•3mo ago

I'm talking about closing the output gate after calling an async function with "fire and forget" If it was open before, either because storage was synced or used { allowUnconfirmed: true }, it can close while that function is still running So I can't guarantee that what is inside that function will ignore the output gates even if I would like that behavior Its one of the reasons why I asked you had considered having some way to ignore output gates for a function call What I though of was having a method that you could call passing a callback function, something like this.allowUnconfirmed(() => something()); Then it would guarantee that whatever happens here will ignore the output gate I also think this could be less error prone as it communicates intent better

lambrospetrou•3mo ago

And you want this in order to avoid the delay of waiting for the confirmation of the writes? If I understand correctly what you want, I guess you can build this in userland already, by adding a helper withUnconfirmed(fn: async () => void) and that function will override the storage methods to always provide the allowUnconfirmed: true option to the underlying methods (i.e. using a proxy to the storage object). Not 100% what you asked, but it should solve that usecase.

João Castro•3mo ago

I don't have a concrete use case for this right now, just a hypothetical scenario that I imagined when thinking about it This would allow everything to pass the output gates The argument is that it seems to make more sense to have a function that allows you to execute things that will ignore the output gate, rather than not closing them on specific write operations Since this is something you want specifically for the operation that you want to talk to the outside world Having to close it again and remember to add allowUnconfirmed to new writes is a lot more work and room for error When you could have an explicit this.allowUnconfirmed(() => fetch());

await this.ctx.storage.put('a', 'b', {
  allowUnconfirmed: true
});

await fetch();
await this.ctx.storage.sync();
await fetch();

//vs.

await this.ctx.storage.put('a', 'b');
this.allowUnconfirmed(() => fetch());
await fetch();

await this.ctx.storage.put('a', 'b', {
  allowUnconfirmed: true
});

await fetch();
await this.ctx.storage.sync();
await fetch();

//vs.

await this.ctx.storage.put('a', 'b');
this.allowUnconfirmed(() => fetch());
await fetch();

It also seems to be better for libraries like Drizzle, since they wouldn't need to change their apis to support passing allowUnconfirmed

lambrospetrou•3mo ago

Unless you have a use-case, my personal suggestion to everyone is NEVER use allowUnconfirmed: true. It makes reasoning for your code much harder and it is inevitable that it will lead to consistency bugs. Now, if you are doing thousands of reqs per second and you want to squeeze out every single millisecond from the DO and you are very careful, at that point I am sure you are probably in a position to add a few lines of code to use the option. Anyway seems like we are talking about imaginary cases that might even have other solutions when it comes down to it. If you get a concrete case at some point, please bring it up and we can see what we can improve.

João Castro•3mo ago

Yeah I agree, it makes you rely on external state and it becomes a mess. Thats exactly why something like this.allowUnconfirmed(() => fetch()); would be much better. You would be able to talk to the outside world when you want without needing to worry about managing the state of the output gate by hand Its easier to understand what is happenning, you won't have issues with the gate closing unexpectedly, and its better for libraries that wrap the storage api like Drizzle and other ORMs Even for the DO team it seems like it would be better. We would already have the solution for the SQL api for example, that we don't have today Because it would be tied to the output gate and not to every single write operation that we can have in the DO

MarakOP•3mo ago

Who exactly is "you" and "we" in this scenario? I don't understand where the strong opinion is coming from and who it's directed at? Have you been tasked with building a solution on CF @João Castro? If you aren't in active development or managing an existing application that needs such functionality, who is it that you are making this argument for?

João Castro•3mo ago

I'm managing an existing application that uses the current api, and I think there could be a better one Thats just it Don't see a clear downside of what I was thinking, so I brought it here to see if I was missing something But until now it just seems that it would handle the current use cases with less cognitive overhead and would be less error prone It seems like a simpler and more concise solution overall

MarakOP•3mo ago

I don't appreciate the nasty message you sent and then deleted. Happy to have helped you today. Perhaps you might want to summarize your thoughts in a blog post and circulate it for review. Everyone here is happy to learn and improve. Have a great day.

João Castro•3mo ago

I misread your message, then when I read it again I deleted mine, sorry for that Anyways, I understand that a lot of things can be handled with the current way that things are done But I also think this is a much more complicated way to deal with this than it could be with a slightly different api The opinion is about the api, not about someone. Doesn't need to be directed at anyone.

Gaming

Programming

Output gate and async operations

Did you find this page helpful?