M
MastraAI•5w ago
emekdahl

Put Agent Memory on a Different Model than Response?

Is there a way to put agent memory on a different model than the response? For example, I may want my agent to respond to a customer using a high quality / expensive / longer running model but I'd ideally like it to write to memory with a fast cheap model.
2 Replies
Mastra Triager
Mastra Triager•5w ago
📝 Created GitHub issue: https://github.com/mastra-ai/mastra/issues/8074
GitHub
[DISCORD:1419423115280388137] Put Agent Memory on a Different Model...
This issue was created from Discord post: https://discord.com/channels/1309558646228779139/1419423115280388137 Is there a way to put agent memory on a different model than the response? For example...
_roamin_
_roamin_•5w ago
Hi, @emekdahl! I think you migh be able to do this using the prepareStep function that's available on streamVNext and generateVNext. I haven't tried it myself, but in prepareStep you have access to the messages and you can return a model. Let me check if I can find an example You can try this, I'm not sure it'll work though because the model has already done the tool selection. To be tested: 
const response = await agent.streamVNext(
      [
        {
          role: "user",
          content: prompt,
        },
      ],
      {
        prepareStep: async ({ messages, model }) => {
          // check if calling
          const lastMessage = messages[messages.length - 1];

          // if last message is a tool-call and the tool is updateWorkingMemory, we want to use a different model
          if (
            lastMessage.role === "assistant" &&
            Array.isArray(lastMessage.content)
          ) {
            const toolCall = lastMessage.content.find(
              (c) => c.type === "tool-call"
            );
            if (toolCall && toolCall.toolName === "updateWorkingMemory") {
              model = openai("gpt-4o-mini");
            }
          }

          return {
            model,
          };
        },
      }
    );
const response = await agent.streamVNext(
      [
        {
          role: "user",
          content: prompt,
        },
      ],
      {
        prepareStep: async ({ messages, model }) => {
          // check if calling
          const lastMessage = messages[messages.length - 1];

          // if last message is a tool-call and the tool is updateWorkingMemory, we want to use a different model
          if (
            lastMessage.role === "assistant" &&
            Array.isArray(lastMessage.content)
          ) {
            const toolCall = lastMessage.content.find(
              (c) => c.type === "tool-call"
            );
            if (toolCall && toolCall.toolName === "updateWorkingMemory") {
              model = openai("gpt-4o-mini");
            }
          }

          return {
            model,
          };
        },
      }
    );

Did you find this page helpful?