MastraAI•2w ago

@mastra/ai-sdk chatRoute with middleware context

Hey all, I'm trying to use the chatRoute from the mastra ai-sdk and I have middleware configured that is grabbing some additional metadata my backend sends and adding it to the runtime context. When the agent is called up with a users chat my middleware sees and sets the runtimeContext successfully but the agent runtimeContext is empty. Is this a known issue or perhaps something I'm missing with this setup? This is from the docs here https://mastra.ai/en/docs/frameworks/agentic-uis/ai-sdk

export const mastra = new Mastra({
  workflows: { fileUploadProcessorWorkflow },
  agents: { fileProcessorAgent, investmentChatAgent },
  storage: getStorage(),

  // Server configuration to bind to all interfaces
  server: {
    port: 4111,
    host: "0.0.0.0",

    middleware: [
      async (c, next) => {
        const runtimeContext = c.get("runtimeContext");
        if (c.req.method === "POST" && c.req.url.includes("/chat")) {
          try {
            const clonedReq = c.req.raw.clone();
            const body = await clonedReq.json();
            if (body?.data) {
              for (const [key, value] of Object.entries(body.data)) {
                runtimeContext.set(key, value);
              }
            }
          } catch {}
        }
        await next();
      },
    ],

    apiRoutes: [
      chatRoute({
        path: "/chat",
        agent: "investmentChatAgent",
      }),
    ],
  }
})

export const mastra = new Mastra({
  workflows: { fileUploadProcessorWorkflow },
  agents: { fileProcessorAgent, investmentChatAgent },
  storage: getStorage(),

  // Server configuration to bind to all interfaces
  server: {
    port: 4111,
    host: "0.0.0.0",

    middleware: [
      async (c, next) => {
        const runtimeContext = c.get("runtimeContext");
        if (c.req.method === "POST" && c.req.url.includes("/chat")) {
          try {
            const clonedReq = c.req.raw.clone();
            const body = await clonedReq.json();
            if (body?.data) {
              for (const [key, value] of Object.entries(body.data)) {
                runtimeContext.set(key, value);
              }
            }
          } catch {}
        }
        await next();
      },
    ],

    apiRoutes: [
      chatRoute({
        path: "/chat",
        agent: "investmentChatAgent",
      }),
    ],
  }
})

14 Replies

Incisiv3OP•2w ago

Or is there another way when ineracting with this new /chat endpoint to send along data i'd like to be put in runtimeContext automatically? my setup is basically react frontend -> NestJS backend server -> mastra API

Mastra Triager•2w ago

📝 Created GitHub issue: https://github.com/mastra-ai/mastra/issues/8609

GitHub

[DISCORD:1425287716114792499] @mastra/ai-sdk chatRoute with middlew...

This issue was created from Discord post: https://discord.com/channels/1309558646228779139/1425287716114792499 Hey all, I'm trying to use the chatRoute from the mastra ai-sdk and I have middlew...

_roamin_•2w ago

Hi @Incisiv3 ! I think that's because you're not setting the runtimeContext back into the hono context, can you try adding c.set("runtimeContext", runtimeContext); after you're do setting up your runtimeContext?

Incisiv3OP•2w ago

Oh fair point yes let me give it a try shortly and I’ll get back to you @Romain I adjusted it to re-set the runtimeContext and my agent still is getting an empty object when it's invoked

    middleware: [
      async (c, next) => {
        if (c.req.method === "POST" && c.req.url.includes("/chat")) {
          try {
            const clonedReq = c.req.raw.clone();
            const body = await clonedReq.json();
            if (body?.data) {
              let runtimeContext = c.get("runtimeContext");
              // Initialize if it doesn't exist
              if (!runtimeContext) {
                runtimeContext = new Map();
              }
              for (const [key, value] of Object.entries(body.data)) {
                runtimeContext.set(key, value);
              }
              c.set("runtimeContext", runtimeContext);
            }
          } catch (error) {
            console.error("Error setting runtime context:", error);
          }
        }
        await next();
      },
    ],

    middleware: [
      async (c, next) => {
        if (c.req.method === "POST" && c.req.url.includes("/chat")) {
          try {
            const clonedReq = c.req.raw.clone();
            const body = await clonedReq.json();
            if (body?.data) {
              let runtimeContext = c.get("runtimeContext");
              // Initialize if it doesn't exist
              if (!runtimeContext) {
                runtimeContext = new Map();
              }
              for (const [key, value] of Object.entries(body.data)) {
                runtimeContext.set(key, value);
              }
              c.set("runtimeContext", runtimeContext);
            }
          } catch (error) {
            console.error("Error setting runtime context:", error);
          }
        }
        await next();
      },
    ],

_roamin_•2w ago

Hmm, I think that's a bug in chatRoute... I think that the first code you shared should have been working. Let me check something.

Incisiv3OP•2w ago

Ok thanks

_roamin_•2w ago

Could you try using the tag fix-runtimeContext-passing-chatRoute as your Mastra packages version and let me know if it's correctly passing the runtime context?

Incisiv3OP•2w ago

@Romain yep that seems to have fixed it! seems to work without me explicitly setting the runtime context back too with c.set("runtimeContext", runtimeContext);

Unknown User•2w ago

Message Not Public

_roamin_•2w ago

Thanks @Omicrxn & @Incisiv3 for testing out the fix, I'll have the PR reviewed and merged asap.

Unknown User•2w ago

Message Not Public

_roamin_•2w ago

You can use the snapshort version above for now btw, I'll try to have an alpha released today, but not sure we'll have another stable release before next tuesday

Unknown User•2w ago

Message Not Public

Incisiv3OP•2w ago

Thanks all!

Gaming

Programming

@mastra/ai-sdk chatRoute with middleware context

Did you find this page helpful?