Hey I‘m trying to use the vision capabilities for this mistral model @cf/mistralai/mistral-small-3.1

Aandifined.dev Hey I‘m trying to use the vision capabilities for this mistral model @cf/mistral...

U

uhsheesh•4/12/25, 6:08 PM

hello ! You need to either send

messages

messages

or

prompt

prompt

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: [
      { role: "system", content: "You're a helpful assistant." },
      { role: "assistant", content: prompt }
    ]
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: [
      { role: "system", content: "You're a helpful assistant." },
      { role: "assistant", content: prompt }
    ]
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: [
      { role: "system", content: "You're a helpful assistant." },
      { role: "assistant", content: prompt }
    ]
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: [
      { role: "system", content: "You're a helpful assistant." },
      { role: "assistant", content: prompt }
    ]
  }
);

or

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
     "prompt": prompt
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
     "prompt": prompt
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
     "prompt": prompt
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
     "prompt": prompt
  }
);

Uuhsheesh hello ! You need to either send `messages` or `prompt` ```typescript const aiRes...

A

andifined.devOP•4/12/25, 6:11 PM

Ah okay Thank you very much! But let‘s say for example, I want to do vision analysis and provide 6 images, where do I Place the User prompt? The First 6 items as objects utilazing image_url and than the user promt to search for within those images?

U

uhsheesh•4/12/25, 6:25 PM

You'd do something like this:

const messages = [
{
  role: "system",
  content: "You are a helpful assistant that can analyze images."
},
{
  role: "user",
  content: [
    {
      type: "text",
      text: "What's in this image?"
    },
    {
      type: "image_url",
      image_url: {
        url: datauri

      }
    },
    ... // extra
  ]
}
];

const messages = [
{
  role: "system",
  content: "You are a helpful assistant that can analyze images."
},
{
  role: "user",
  content: [
    {
      type: "text",
      text: "What's in this image?"
    },
    {
      type: "image_url",
      image_url: {
        url: datauri

      }
    },
    ... // extra
  ]
}
];

const messages = [
{
  role: "system",
  content: "You are a helpful assistant that can analyze images."
},
{
  role: "user",
  content: [
    {
      type: "text",
      text: "What's in this image?"
    },
    {
      type: "image_url",
      image_url: {
        url: datauri

      }
    },
    ... // extra
  ]
}
];

const messages = [
{
  role: "system",
  content: "You are a helpful assistant that can analyze images."
},
{
  role: "user",
  content: [
    {
      type: "text",
      text: "What's in this image?"
    },
    {
      type: "image_url",
      image_url: {
        url: datauri

      }
    },
    ... // extra
  ]
}
];

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: messages
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: messages
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: messages
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: messages
  }
);

Uuhsheesh You'd do something like this: ```typescript const messages = [ { role: "syste...

A

andifined.devOP•4/13/25, 8:14 AM

Already tried that but it seams that the mistral model is not capable of analyzing images even it‘s possible to provide one?

Using the following request:

const aiResponse = await env.AI.run(
"@cf/mistralai/mistral-small-3.1-24b-instruct",
{
messages: [
{
role: 'system',
content: "You are a helpful assistant that analyzes images."
},
{
role: "user",
content: [
{
type: "image_url",
image_url: {
url: images[0],
}
},
{
type: "text",
text: prompt
}
]
}
]
}
);

I'm sorry, but I can't directly analyze images or videos.

What am I doing wrong?

Sshivam I'm encountering an Error: 10000: Authentication error when trying to connect to...

F

flashblaze•4/13/25, 8:08 PM

Were you able to solve this?

L

LeftyLlama•4/13/25, 8:16 PM

Will the MeloTTS model ever have more voices/voice customization options? It seems like MeloTTS can do this but I don't see anything about it in Cloudflare's documentation for it

A

ajgeiss0702•4/13/25, 9:30 PM

has there been any progress on the issue of reasoning models not returning valid reasoning responses?

A

ajgeiss0702•4/13/25, 9:30 PM

the issue still appears to exist on the newly-added qwq model

A

ajgeiss0702•4/13/25, 9:32 PM

please ping me if you reply

Aandyt Hey is there a file size limit for toMarkdown? I have tried a few larger files t...

I

ItsWendell•4/14/25, 12:28 PM

Curious to hear what file sizes are failing for you. I've actually made a toMarkdown function myself for PDFs, which uses unpdf to stream the contents per page to the response, minimizing memory usage.

C

cosbgn•4/15/25, 1:59 PM

Are there any options to pass to

ai/tomarkdown

ai/tomarkdown

?

I currently get a whole paragraph before the actual content of the PDF which I definitely don't want

 ## Metadata - PDFFormatVersion=1.4 - Language=en-GB - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Creator=Writer - Producer=OpenOffice.org 2.4 - CreationDate=D:20090324113315-06'00'

 ## Metadata - PDFFormatVersion=1.4 - Language=en-GB - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Creator=Writer - Producer=OpenOffice.org 2.4 - CreationDate=D:20090324113315-06'00'

 ## Metadata - PDFFormatVersion=1.4 - Language=en-GB - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Creator=Writer - Producer=OpenOffice.org 2.4 - CreationDate=D:20090324113315-06'00'

 ## Metadata - PDFFormatVersion=1.4 - Language=en-GB - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Creator=Writer - Producer=OpenOffice.org 2.4 - CreationDate=D:20090324113315-06'00'

C

cosbgn•4/15/25, 2:02 PM

perhaps would be a good idea to have data as an object {metadata, content} ?

Ccosbgn Are there any options to pass to `ai/tomarkdown` ? I currently get a whole par...

I

Isaac McFadyen•4/15/25, 3:48 PM

It looks like this is one line in which case you could just remove any lines that start with "## Metadata"

M

michaelthatsit•4/16/25, 12:41 AM

I'm running into an issue where the response is being cut short. I've looked into the discord history here and can't seem to find an answer. any ideas?

Mmichaelthatsit I'm running into an issue where the response is being cut short. I've looked int...

I

Isaac McFadyen•4/16/25, 12:56 AM

max_tokens

max_tokens

defaults to 256 tokens, try setting that higher in your request to the model.

M

michaelthatsit•4/16/25, 1:06 AM

it's unclear where to set that.

M

michaelthatsit•4/16/25, 1:15 AM

nvm I'm dumb

IIsaac McFadyen It looks like this is one line in which case you could just remove any lines tha...

C

cosbgn•4/16/25, 12:11 PM

In this case yes, but files are user submitted, I don't have control on how the metadata line will be. Is it always a single line? not sure

M

Muhammadreza Haghiri•4/16/25, 9:15 PM

Hello everyone. Is there a chance of having Flux Dev as well?

B

Brinamite•4/17/25, 8:59 PM

I do believe that the warning is a bug. I am only 93.8 neurons in for today

Mmewfree hey team, seeing this scary banner even though I did not even have any requests ...

B

Brinamite•4/17/25, 9:00 PM

ah, seeing that even others are posting this

BBrinamite I do believe that the warning is a bug. I am only 93.8 neurons in for today 🧐

I

Isaac McFadyen•4/17/25, 9:27 PM

Do you know how much you're using? Could that mean 93.8k? Not sure of the unit there.

R

Rapid•4/18/25, 12:41 PM

Which way should I go if I want to receive text from a user and turn it into a predefined schema? like, parameter extracting of text using LLM or what's the best practice for that? Anyone got a clue

BBrinamite I do believe that the warning is a bug. I am only 93.8 neurons in for today 🧐

A

ale•4/18/25, 6:29 PM

hi @Brinamite, thanks for reporting this. The team is working on it

C

Clint•4/18/25, 7:07 PM

why might I see working tool calling locally but not when deployed to workers.dev?

A

Alexander•4/19/25, 1:20 PM

Hey

Is there any way to increase AutoRAG pipeline limit during beta? I am building a RAG app that requires separation of data between users and since there is no way to add metadata before / during retrieval the only way I can enforce the above is 1 user == 1 rag pipeline

I'd be happy to explore other options - maybe I am missing something?

1) ideally - increase rag pipeline limit
2) or allow query / search endpoint to use metadata filters so that I can exclude data belonging to different users

M

michaelthatsit•4/20/25, 12:59 AM

I'm getting this error when trying to use streamObject. Is there plans to support JSON Schema mode with stream mode?

InferenceUpstreamError [AiError]: 5023: JSON Schema mode is not supported with stream mode.

InferenceUpstreamError [AiError]: 5023: JSON Schema mode is not supported with stream mode.

A

Alexander•4/20/25, 4:03 AM

Hey, is it possible to trigger AutoRAG to re-index from the API? Couldn't find any way to do this?

AAlexander Hey, is it possible to trigger AutoRAG to re-index from the API? Couldn't find a...

V

Viktor•4/20/25, 5:22 PM

AutoRAG does indexing when data is uploaded to R2. So if you need to re-index, maybe you could override data in R2

RRapid Which way should I go if I want to receive text from a user and turn it into a p...

R

Rapid•4/20/25, 10:12 PM

Anyone?

M

mr.niko.la•4/21/25, 5:23 AM

What is the whsiper turbo file
Size limit ? 100mb

Also requesting the documentation or repo to show sample code to run it on workers ai.

Current documentation is not clear. Thx

M

mr.niko.la•4/21/25, 8:08 AM

wait this is crazy. my code has failover using whisper turbo. does it show up as regular whisper on this dashboard? im very confused as i never refernce

@cf/openai/whisper

@cf/openai/whisper

is this a bug on the dashboard. tbh i dont even know this was being run anywhere as i never really had the failover ever trigger....

TTait Hey all. If I wanted to extract audio from a video before hitting a workers AI ...

面

面条•4/21/25, 1:15 PM

I just solved this problem https://kai.bi/post/cloudflare-audio-concat

Mmr.niko.la wait this is crazy. my code has failover using whisper turbo. does it show up a...

I

Isaac McFadyen•4/21/25, 4:13 PM

Whisper and Whisper Turbo are different models, if you're using turbo it should appear as Turbo not base

Y

yinxingmaiming6409•4/22/25, 1:26 AM

Isn't the Rag size limit 4M? Why does my xlsx file 2.1M still prompt that it exceeds the size limit?

K

Kasjmier Leela•4/22/25, 4:18 AM

i have a possibly silly question. forgive me if it is.

i want to use an email worker to intercept incoming emails, extract email content, run some workers-ai logic on the content to gather context and send the context out as a json response.

is this possible with an email worker?

IIsaac McFadyen Whisper and Whisper Turbo are different models, if you're using turbo it should ...

M

mr.niko.la•4/22/25, 5:49 AM

this is strange i looked at my code and only

this.env.AI.run('@cf/openai/whisper-large-v3-turbo

this.env.AI.run('@cf/openai/whisper-large-v3-turbo

. i cant find any reference to the othe rone yet i see it in my dashboard. ...

is there any other way to debug it further?

P

Puru Dahal•4/22/25, 6:38 AM

@Cloudflare Not sure which one to tag specifically, but when can we expect Deepseek - V3?

S

SuperHelpflare•4/22/25, 7:06 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

M

mr.niko.la•4/22/25, 7:36 AM

where is this playground the one i got doesnt have grok in it....

M

mr.niko.la•4/22/25, 7:40 AM

https://playground.ai.cloudflare.com/ this doesnt have grok...

RRapid Anyone?

V

Viktor•4/22/25, 11:03 AM

What exactly are you trying to reach? Do you want something you can parse afterwards? You can always ask the llm to output data in different formats, I noticed json works well with llama

V

Viktor•4/22/25, 11:06 AM

like this, "Output as valid json, dont write additional text or markdown"

R

Rapid•4/22/25, 11:48 AM

I have a function that I wanna call using LLM. So basically imagine a user writes I want to buy 5 NVDA stock.

So I want it to interpt to a JSON and then send the user a message with the parameters so he clicks "confirm" and then execute it. So basically just parsing text into predefined schema.

VViktor like this, "Output as valid json, dont write additional text or markdown"

R

Rapid•4/22/25, 12:03 PM

I will try it, was wondering if there is something that is meant for that though

RRapid I have a function that I wanna call using LLM. So basically imagine a user write...

V

Viktor•4/22/25, 12:37 PM

You can use functions / "tools" for that

V

Viktor•4/22/25, 12:39 PM

See this: https://developers.cloudflare.com/workers-ai/features/function-calling/
Basicly you enter your prompt, and pass additional functions that the LLM can call. If the LLM wants to call it, it'll pass information about the function, you execute the function, and add the result as

assistant

assistant

message.
There is a

cloudflare/ai-utils

cloudflare/ai-utils

making this simpler, but I'm struggling with that one right now (see my thread from sunday)

Cloudflare Docs

Function calling · Cloudflare Workers AI docs

Function calling enables people to take Large Language Models (LLMs) and use the model response to execute functions or interact with external APIs. The developer usually defines a set of functions and the required input schema for each function, which we call tools. The model then intelligently understands when it needs to do a tool call, and i...

V

Viktor•4/22/25, 12:42 PM

Also there is the "human in the loop", which can be used to actually intercept an ai workflow. But I didn't work with those ones yet, so I can't give any futher information if that is just a function waiting for confirmation, or has some other additional things
https://developers.cloudflare.com/agents/concepts/human-in-the-loop/

Cloudflare Docs

Human in the Loop · Cloudflare Agents docs

Human-in-the-Loop (HITL) workflows integrate human judgment and oversight into automated processes. These workflows pause at critical points for human review, validation, or decision-making before proceeding. This approach combines the efficiency of automation with human expertise and oversight where it matters most.

Hey I‘m trying to use the vision capabilities for this mistral model @cf/mistralai/mistral-small-3.1

Similar Threads

Similar Threads

Similar Threads