Saving context on verbose tool calls
I had an idea to save some tokens on tool calling. I created a workflow that maps the output of verbose tool calls to a more compact and friendly text format. Apparently, the model receives the entire workflow execution log with all intermediate values passed.
Issues identified:
1. Lack of documentation clarity: It is not clear from the documentation that the model receives the full execution log.
2. Suboptimal format with duplication: The current execution JSON format is inefficient due to duplication. The output of step 1 is then repeated as the input of step 2.
3. Incomplete despite verbosity: Given that the output is too verbose, it is still not complete as it doesn't contain the actual path that the execution has taken.
All of the above make workflows a very expensive tool in terms of context usage.
8 Replies
Luckily I was able to solve it with model middleware, still think we need discuss it.
For example we can introduce configuration property for createWorkflow whether to expose full path and intermediate values. And still we want to remove duplications anyway
Oh boy. I just explored output in mcp server tool call exposing agent run which uses workflows to execute tool.
It has tool calls and workflow steps 9 times. And all of them has this input/output duplication. It was just one workflow call by an agent.
result - root object
1.
result.toolResults[0] - "type": "tool-result" object, with payload.result.result.steps - workflow steps with input/output for each step
2. result.response.messages - array of message objects including "role": "tool" with content "type": "tool-result" object that contains workflow steps with input/output for each step
3. result.response.uiMessages - array of ui message objects including tool using object that contains workflow steps with input/output for each step
result.steps[0].stepType: "initial"
4. result.steps[0].toolResults[0].payload.result.result.steps - workflow steps with input/output for each step
5. result.steps[0].content - array of content objects including "type": "tool-result" that contains workflow steps with input/output for each step
6. result.steps[0].response.messages - array of message objects including "role": "tool" with content "type": "tool-result" object that contains workflow steps with input/output for each step
7 .result.steps[0].response.uiMessages - array of ui message objects including tool using object that contains workflow steps with input/output for each step
result.steps[1].stepType: "tool-result"
8. results.steps[1].response.messages - array of message objects including "role": "tool" with content "type": "tool-result" object that contains workflow steps with input/output for each step
9. result.steps[1].response.uiMessages - array of ui message objects including tool using object that contains workflow steps with input/output for each step
Hey @Guria ! Yeah, we have been discussing this case, and also other cases where it would be great to have the ability to do some context summarization/compaction/cleanup. We don't have anything available yet out of the box, but it's planned, just no ETA yet 😉
I'll let our triager bot create a GH issue to keep track of this feature request 😉
I think you didnt trigger the Github issue bot
to do some context summarization/compaction/cleanupI think it worth to revisit formats before trying LLM summarization approach 🙂
📝 Created GitHub issue: https://github.com/mastra-ai/mastra/issues/8951
GitHub
[DISCORD:1428421607776583751] Saving context on verbose tool calls ...
This issue was created from Discord post: https://discord.com/channels/1309558646228779139/1428421607776583751 I had an idea to save some tokens on tool calling. I created a workflow that maps the ...
@! .kinderjaje it runs daily automatically 🙂