M
Mastra•2mo ago
chriscx

agent.generate() with maxTokens set is ignored and defaults to 4096

It seems that maxTokens is totally ignored and defaulted to 4096 when I tried to increase the maxTokens. The code below doesn't respect the maxTokens: 
const result = await agent.generate('Write a short poem about coding', {
  maxTokens: 50,
});
const result = await agent.generate('Write a short poem about coding', {
  maxTokens: 50,
});
The code below will respect the max tokens but setting modelSettings.maxOutputTokens is undocumented on https://mastra.ai/en/reference/agents/generate#options: 
const result = await agent.generate('Write a short poem about coding', {
  modelSettings: {
    maxOutputTokens: 50,
  },
});
const result = await agent.generate('Write a short poem about coding', {
  modelSettings: {
    maxOutputTokens: 50,
  },
});
I can reproduce this with Vertex AI Anthropic and OpenAI.
Reference: Agent.generate() | Agents | Mastra Docs
Documentation for the Agent.generate() method in Mastra agents, which enables non-streaming generation of responses with enhanced capabilities.
2 Replies
Mastra Triager
Mastra Triager•2mo ago
📝 Created GitHub issue: https://github.com/mastra-ai/mastra/issues/9499
GitHub
[DISCORD:1433094638071185591] agent.generate() with maxTokens set i...
This issue was created from Discord post: https://discord.com/channels/1309558646228779139/1433094638071185591 It seems that maxTokens is totally ignored and defaulted to 4096 when I tried to incre...
Abhi Aiyer
Abhi Aiyer•3w ago
Thanks for reporting this! Definitely a documentation mishap here! Fix here: https://github.com/mastra-ai/mastra/pull/10286
GitHub
Fix docs on maxTokens by abhiaiyer91 · Pull Request #10286 · mast...
Description Related Issue(s) Type of Change Bug fix (non-breaking change that fixes an issue) New feature (non-breaking change that adds functionality) Breaking change (fix or feature that wo...

Did you find this page helpful?