Mastra•2mo ago

Cant use OCR of mistral

I have this error

{
  error: UnsupportedFunctionalityError [AI_UnsupportedFunctionalityError]: 'file part media type application/pdf' functionality not supported.
      at <anonymous> (/Users/robinmarquet/Documents/hirify/main/node_modules/@ai-sdk/openai-compatible-v5/src/chat/convert-to-openai-compatible-chat-messages.ts:63:25)
      at Array.map (<anonymous>)
      at convertToOpenAICompatibleChatMessages (/Users/robinmarquet/Documents/hirify/main/node_modules/@ai-sdk/openai-compatible-v5/src/chat/convert-to-openai-compatible-chat-messages.ts:39:28)
      at OpenAICompatibleChatLanguageModel.getArgs (/Users/robinmarquet/Documents/hirify/main/node_modules/@ai-sdk/openai-compatible-v5/src/chat/openai-compatible-chat-language-model.ts:201:19)
      at async OpenAICompatibleChatLanguageModel.doStream (/Users/robinmarquet/Documents/hirify/main/node_modules/@ai-sdk/openai-compatible-v5/src/chat/openai-compatible-chat-language-model.ts:313:32)
      at async pRetry.default.retries (/Users/robinmarquet/Documents/hirify/main/node_modules/@mastra/core/src/agent/trip-wire.ts:72:14)
      at async Module.pRetry (file:///Users/robinmarquet/Documents/hirify/main/node_modules/p-retry/index.js:195:19)
      at async createStream (/Users/robinmarquet/Documents/hirify/main/node_modules/@mastra/core/src/agent/trip-wire.ts:72:14)
      at async Object.start (/Users/robinmarquet/Documents/hirify/main/node_modules/@mastra/core/src/agent/trip-wire.ts:53:36) {
    cause: undefined,
    functionality: 'file part media type application/pdf',
    [Symbol(vercel.ai.error)]: true,
    [Symbol(vercel.ai.error.AI_UnsupportedFunctionalityError)]: true
  },
  runId: '89311788-dfd1-414f-9b6e-978bab4a62ca'
}

{
  error: UnsupportedFunctionalityError [AI_UnsupportedFunctionalityError]: 'file part media type application/pdf' functionality not supported.
      at <anonymous> (/Users/robinmarquet/Documents/hirify/main/node_modules/@ai-sdk/openai-compatible-v5/src/chat/convert-to-openai-compatible-chat-messages.ts:63:25)
      at Array.map (<anonymous>)
      at convertToOpenAICompatibleChatMessages (/Users/robinmarquet/Documents/hirify/main/node_modules/@ai-sdk/openai-compatible-v5/src/chat/convert-to-openai-compatible-chat-messages.ts:39:28)
      at OpenAICompatibleChatLanguageModel.getArgs (/Users/robinmarquet/Documents/hirify/main/node_modules/@ai-sdk/openai-compatible-v5/src/chat/openai-compatible-chat-language-model.ts:201:19)
      at async OpenAICompatibleChatLanguageModel.doStream (/Users/robinmarquet/Documents/hirify/main/node_modules/@ai-sdk/openai-compatible-v5/src/chat/openai-compatible-chat-language-model.ts:313:32)
      at async pRetry.default.retries (/Users/robinmarquet/Documents/hirify/main/node_modules/@mastra/core/src/agent/trip-wire.ts:72:14)
      at async Module.pRetry (file:///Users/robinmarquet/Documents/hirify/main/node_modules/p-retry/index.js:195:19)
      at async createStream (/Users/robinmarquet/Documents/hirify/main/node_modules/@mastra/core/src/agent/trip-wire.ts:72:14)
      at async Object.start (/Users/robinmarquet/Documents/hirify/main/node_modules/@mastra/core/src/agent/trip-wire.ts:53:36) {
    cause: undefined,
    functionality: 'file part media type application/pdf',
    [Symbol(vercel.ai.error)]: true,
    [Symbol(vercel.ai.error.AI_UnsupportedFunctionalityError)]: true
  },
  runId: '89311788-dfd1-414f-9b6e-978bab4a62ca'
}

6 Replies

RobinOP•2mo ago

I put this in input message

{
  type: 'file',
  data: new URL(fileUrl),
  mimeType: 'application/pdf',
}

{
  type: 'file',
  data: new URL(fileUrl),
  mimeType: 'application/pdf',
}

And I have this in agent (I use mistral-medium-latest)

const stream = await agent.stream(inputMessage, {
  structuredOutput: {
    schema: templateGeneratorOutputSchema,
    jsonPromptInjection: true,
  },
  modelSettings: {
    temperature: 0.3,
    maxOutputTokens: 8192,
  },
  providerOptions: {
    mistral: {
      documentImageLimit: 8,
      documentPageLimit: 64,
    },
  },
})

const stream = await agent.stream(inputMessage, {
  structuredOutput: {
    schema: templateGeneratorOutputSchema,
    jsonPromptInjection: true,
  },
  modelSettings: {
    temperature: 0.3,
    maxOutputTokens: 8192,
  },
  providerOptions: {
    mistral: {
      documentImageLimit: 8,
      documentPageLimit: 64,
    },
  },
})

Mastra Triager•2mo ago

📝 Created GitHub issue: https://github.com/mastra-ai/mastra/issues/9685

GitHub

Cant use OCR of mistral · Issue #9685 · mastra-ai/mastra

This issue was created from Discord post 1434935296176423002: I have this error { error: UnsupportedFunctionalityError [AI_UnsupportedFunctionalityError]: 'file part media type application/json...

_roamin_•2mo ago

Hi @Robin ! Unfortunately, that seems to be a limitation of the @ai-sdk/openai-compatible package... I'm not even sure you can send a json to llms as a message part 🤔

RobinOP•2mo ago

Sorry it's application/pdf

_roamin_•3w ago

I see, thanks for updating the error message. I'm able to reproduce but I think it's just a limitation of @ai-sdk/openai-compatible looking at their source code https://github.com/vercel/ai/blob/e649a60298c0ade133914a13d0cfe1ecbd2307f9/packages/openai-compatible/src/chat/convert-to-openai-compatible-chat-messages.ts#L45 We added support for mistral recently so you should actually be able to use it this way directly:

const agent = new Agent({
  // ...
  model: "mistral/mistral-medium-latest",
});

const agent = new Agent({
  // ...
  model: "mistral/mistral-medium-latest",
});

RobinOP•3w ago

Thanks !!!

Gaming

Programming

Cant use OCR of mistral

Did you find this page helpful?