How to persist partial step output to memory for single step agent invocations
Hi Folks,
I was wondering if its possible to stop agent stream midway while also preserving the tokens that were streamed till the time user stopped the stream.
So essentially:
User sends a message.
Lets say agent streams about 200 tokens and user clicks on stop.
Agent stops generating and we save these 200 tokens to the memory.
Currently, if I stop the stream from frontend and abort using abort controller on the backend, it does not persist the message to the backend.
I had asked this before and someone recommended me to use
But my agents are only making single LLM calls and I want to save the response to memory as it streams.
I was wondering if its possible to stop agent stream midway while also preserving the tokens that were streamed till the time user stopped the stream.
So essentially:
User sends a message.
Lets say agent streams about 200 tokens and user clicks on stop.
Agent stops generating and we save these 200 tokens to the memory.
Currently, if I stop the stream from frontend and abort using abort controller on the backend, it does not persist the message to the backend.
I had asked this before and someone recommended me to use
savePerStepBut my agents are only making single LLM calls and I want to save the response to memory as it streams.
