MastraAI•2mo ago

streamVNext converting image URLs to embedded content - need URLs as strings for tools

Problem: When I pass image URLs as file parts, the agent receives only the embedded image content for visual analysis (it treats it as “embedded directly” ?) instead of also retaining the original URL string. While embedding the content is fine for visual analysis, losing the URL is problematic. The agent should still have access to the original URL so it can pass it to other tools, rather than only seeing the embedded content and losing the source URL. What I need: - Agent should also see: "https://cdn.example.co/chat-images/image.png" as a string - Agent should NOT see: embedded image content/data without the source url

sendMessage({
  text,
  files: images.map((img) => ({
    type: "file" as const,
    filename: img.fileName,
    mediaType: img.fileType,
    url: img.url,  // URL here - expecting it to stay as a string
  })),
});

sendMessage({
  text,
  files: images.map((img) => ({
    type: "file" as const,
    filename: img.fileName,
    mediaType: img.fileType,
    url: img.url,  // URL here - expecting it to stay as a string
  })),
});

My chat history shows me these parts as the input:

{
  "parts": [
    {
      "type": "file",
      "url": "https://cdn.example
.co/chat-images/image.png",
      "mediaType": "image/png"
    },
    {
      "type": "text",
      "text": "Can you see the image URL?"
    }
  ]
}

{
  "parts": [
    {
      "type": "file",
      "url": "https://cdn.example
.co/chat-images/image.png",
      "mediaType": "image/png"
    },
    {
      "type": "text",
      "text": "Can you see the image URL?"
    }
  ]
}

Agent responds: "No, there's no image URL provided—it's embedded directly" This tells me that the URL is being transformed into something else.. Question: Is streamVNext fetching the URL and converting it to embedded image data (base64/ASCII art/...)? How do I prevent this transformation and keep the URL as a plain string that my agent can read and pass to tools? I just need the URL string preserved as-is for my tools to use.

const stream = await agent.streamVNext(messages, {
  format: "aisdk",
  // ...
});

const stream = await agent.streamVNext(messages, {
  format: "aisdk",
  // ...
});

4 Replies

Mastra Triager•2mo ago

📝 Created GitHub issue: https://github.com/mastra-ai/mastra/issues/8059

GitHub

[DISCORD:1419071223958212679] streamVNext converting image URLs to ...

This issue was created from Discord post: https://discord.com/channels/1309558646228779139/1419071223958212679 Problem: When I pass image URLs using file parts, the agent is seeing the IMAGE CONTEN...

dotkoffOP•5w ago

Is there a way to handle this guys?🙏

_roamin_•6d ago

Hi @dotkoff ! Are you still running into this issue?

dotkoffOP•5d ago

I found a workaround. Instead of using file parts, append the url to the text content in the backend and filter the urls on

.getMessages

.getMessages

so they dont appear on the frontend

Gaming

Programming

streamVNext converting image URLs to embedded content - need URLs as strings for tools

Did you find this page helpful?