Structured output sometimes not getting followed in agent generate()
I have an agent taht uses generate() to output a structured response and I notice that this sometimes results in an erro that gets logged as this in my generate() function onError handler:
It looks like the output is undefined based on the error we catch in our try-cacth around this workflow step:
Type validation failed: Value: undefined. Error message: Validation failed
For context, we're using gpt-4.1 and here is the agent code that lives in one of our workflow steps:
This happens pretty sparsely but is still a real issue since this is a critical step in the workflow. Some ideas I have that may help (though still unsure) are:
- using .strict() in my schema definition
- defining structuredOutput.instructions to give custom formatting instructions
- using a different model provider in structuredOutput.model.provider for formatting
Please let me know what the fix here is, thanks!14 Replies
Hi @seb7wake! We actually just shipped some big improvements to structuredOutputs in our latest release yesterday @mastra/core@0.21.0 including:
- Using native response format by default (if your model does not support that you can opt into jsonPromptInjection: true which will inject the schema and other information into the prompt to coerce the model to try to return structuredOutput)
- Validating the output properly using zods safeParse and returning a list of detailed zod errors. This allows you to use all Zod's methods like transform and all native zod methods. When something goes wrong you now have detailed error messaging so you can get the model to retry in cases when the LLM trips up
- Only use the structuring sub agent if a model is passed in, otherwise do everything in the main agentic loop
- Better type safety with the response object
- Passing context that was missing before to the structuring agent so it can provide better results
- Merge the subagent stream into the parent stream to get object-result chunks in the main stream.
Try it out and let us know what you think! Let us know if it fixes your issues or if there is still some more work to be done!
thanks!! a few things:
- what is the structuring subagent you're referring to?
- anything we can do to improve this on our current version (@mastra/core@0.20.0)? We have a launch coming up this week and can't take on the risk of upgrading right now.
If you pass a model to the options, it will create an agent with that model and that agent takes the output of your main agent and generates structured output with it.
I see, good to know. I assume this would add some latency since we're using another agent?
As for improving this on your current version, have you tried to set maxSteps to 1 and not pass a model to it
isn't maxSteps = 1 by default?
The latency added will be pretty minimal as every step is an extra LLM call anyway
I don't believe so
Just checked, maxSteps default is 5
Awesome, thanks for checking—I'll definitely test that out then
and would you recommend also experimenting with the structuring subagent in our current version? Or is this only available in your most recent release?
In that version, it will automatically use the structuring subagent unless maxSteps is set to 1, patterns like this weren't great and led to bad performance which is why we switched it up. We wanted less "magic" to happen that the user has to just know how things work behind the scenes
Love that pattern, nice work! Also just generally curious why setting maxSteps=1 would help?
maxSteps: 1 in your version might help because it will bypass the structuring agent
huh, and the structuring agent in my version isn't super reliable?
I would think it the structuring agent would help here, no?
it's possible, it's hard to say what exactly the issue is
Good to know, appreciate the quick replies!
We tried setting maxSteps to 1 but it didn't work - we still see the
OUTPUT_SCHEMA_VALIDATION_FAILED error.