Scorer doesn't use custom gateway.
AgentEval
We have to use a custom gateway internally, so I created a https://mastra.ai/models/v1/gateways/custom-gateways
eg. when creating my own Agent it works perfectly, but I found that using prebuilt Scorer doesn't resolve the model from the gateway - I traced it to here, I think it's because it doesn't receive the mastra instance maybe: https://github.com/mastra-ai/mastra/blob/b0f3c9861c05ef07f282d5e70bfa2111b9f109c1/packages/core/src/evals/base.ts#L547
I'm using;
"@mastra/core": "1.0.0-beta.5",
"@mastra/evals": "1.0.0-beta.1"
eg. when creating my own Agent it works perfectly, but I found that using prebuilt Scorer doesn't resolve the model from the gateway - I traced it to here, I think it's because it doesn't receive the mastra instance maybe: https://github.com/mastra-ai/mastra/blob/b0f3c9861c05ef07f282d5e70bfa2111b9f109c1/packages/core/src/evals/base.ts#L547
I'm using;
"@mastra/core": "1.0.0-beta.5",
"@mastra/evals": "1.0.0-beta.1"
Mastra v1 Beta: Create custom model gateways for private or specialized LLM deployments
GitHub
The TypeScript AI agent framework. 
Assistants, RAG, observability. Supports any LLM: GPT-4, Claude, Gemini, Llama. - mastra-ai/mastra
Assistants, RAG, observability. Supports any LLM: GPT-4, Claude, Gemini, Llama. - mastra-ai/mastra