Scorer doesn't use custom gateway.
We have to use a custom gateway internally, so I created a https://mastra.ai/models/v1/gateways/custom-gateways
eg. when creating my own Agent it works perfectly, but I found that using prebuilt Scorer doesn't resolve the model from the gateway - I traced it to here, I think it's because it doesn't receive the mastra instance maybe: https://github.com/mastra-ai/mastra/blob/b0f3c9861c05ef07f282d5e70bfa2111b9f109c1/packages/core/src/evals/base.ts#L547
I'm using;
"@mastra/core": "1.0.0-beta.5",
"@mastra/evals": "1.0.0-beta.1"
Custom Gateways | Models | Mastra | Mastra Docs v1 Beta
Mastra v1 Beta: Create custom model gateways for private or specialized LLM deployments
GitHub
mastra/packages/core/src/evals/base.ts at b0f3c9861c05ef07f282d5e70...
The TypeScript AI agent framework. ⚡ Assistants, RAG, observability. Supports any LLM: GPT-4, Claude, Gemini, Llama. - mastra-ai/mastra
2 Replies
📝 Created GitHub issue: https://github.com/mastra-ai/mastra/issues/10745
🔍 If you're experiencing an error, please provide a minimal reproducible example whenever possible to help us resolve it quickly.
🙏 Thank you for helping us improve Mastra!
Hi! We’ll take a look!!