i think this is what your looking for, https://developers.cloudflare.com/workers-ai/features/json-mo

M

mongj•4/9/25, 9:57 AM

hi! what's the difference between @cf/openai/whisper-large-v3-turbo and @cf/openai/whisper? which whisper model is the latter using?

Aandyt i think this is what your looking for, https://developers.cloudflare.com/workers...

A

a4amaan•4/9/25, 11:11 AM

Really helpfull thanks...

M

Muhammadreza Haghiri•4/9/25, 12:12 PM

it seems flux schnell is down.

M

Muhammadreza Haghiri•4/9/25, 12:24 PM

It takes so long and nothing happens.

Aandyt Hey is there a file size limit for toMarkdown? I have tried a few larger files t...

A

andytOP•4/9/25, 1:49 PM

found this, assuming these limist for AutoRag https://developers.cloudflare.com/autorag/configuration/data-source/

Cloudflare Docs

Data source · AutoRAG

AutoRAG currently supports Cloudflare R2 as the data source for storing your knowledge base. To get started, configure an R2 bucket containing your data.

M

mongj•4/9/25, 7:23 PM

will the whisper model on workers ai eventually support auto language detection? we would like to move from openai to workers ai but the lack of language detection is the biggest blocker right now

M

mongj•4/9/25, 7:30 PM

it's certainly possible to put a language detection model in front, but that defeats the cost and latency benefit from using the serverless models on workers ai, especially since other whisper providers (openai, fireworks, groq) already provides it

S

sjg•4/9/25, 9:50 PM

Has anyone used https://developers.cloudflare.com/workers-ai/markdown-conversion/ via REST API? I've not been able to create a token that works.

Cloudflare Docs

Markdown Conversion · Cloudflare Workers AI docs

Markdown is essential for text generation and large language models (LLMs) in training and inference because it can provide structured, semantic, human, and machine-readable input. Likewise, Markdown facilitates chunking and structuring input data for better retrieval and synthesis in the context of RAGs, and its simplicity and ease of parsing a...

R

Rapid•4/9/25, 10:09 PM

Hiya, I'm trying to understand waht should I use, my case is as follow:

I have an API that receives X amount of parameters, I want the user to be able to write "buy me X with Y value and then do Z actions", I'm basically looking for a way for an AI to interept that and return a json like {param1: x, param2: y, param3:z}.

What should I use for that?

R

Rapid•4/9/25, 10:13 PM

Is JSON mode the best solution for that?

Ssjg Has anyone used https://developers.cloudflare.com/workers-ai/markdown-conversion...

S

sjg•4/9/25, 10:15 PM

I created an API token with AI Workers and AutoRAG permissions, no longer get

{"code":10000,"message":"Authentication error"}

{"code":10000,"message":"Authentication error"}

{"code":10000,"message":"Authentication error"}

{"code":10000,"message":"Authentication error"} in the REST response, but do get

{"message":"Server Error","code":6001}

{"message":"Server Error","code":6001}

{"message":"Server Error","code":6001}

{"message":"Server Error","code":6001}, anyone seen this error before?

C

can•4/9/25, 10:53 PM

hello y'all, is there anybody who is using AI SDK with Cloudflare Workers AI provider or anybody who is working on the provider atm?

I was surprised when I saw that image-part is not implemented on the provider, so I was wondering if I should implement it and make a PR or is there anything to be discussed first?

A

a4amaan•4/9/25, 11:52 PM

Trying to return stream from worker

const answer = await ctx.env.AI.autorag("ats-rag").aiSearch({
    query: 'Who is Amaan Khan',
    // model: "@cf/meta/llama-3-8b-instruct",
    model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
    rewrite_query: true,
    max_num_results: 10,
    ranking_options: {
        score_threshold: 0.4,
    },
    stream: true
});
return new Response(answer, {
    headers: {
        'Content-Type': 'text/event-stream'
    }
});

const answer = await ctx.env.AI.autorag("ats-rag").aiSearch({
    query: 'Who is Amaan Khan',
    // model: "@cf/meta/llama-3-8b-instruct",
    model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
    rewrite_query: true,
    max_num_results: 10,
    ranking_options: {
        score_threshold: 0.4,
    },
    stream: true
});
return new Response(answer, {
    headers: {
        'Content-Type': 'text/event-stream'
    }
});

const answer = await ctx.env.AI.autorag("ats-rag").aiSearch({
    query: 'Who is Amaan Khan',
    // model: "@cf/meta/llama-3-8b-instruct",
    model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
    rewrite_query: true,
    max_num_results: 10,
    ranking_options: {
        score_threshold: 0.4,
    },
    stream: true
});
return new Response(answer, {
    headers: {
        'Content-Type': 'text/event-stream'
    }
});

const answer = await ctx.env.AI.autorag("ats-rag").aiSearch({
    query: 'Who is Amaan Khan',
    // model: "@cf/meta/llama-3-8b-instruct",
    model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
    rewrite_query: true,
    max_num_results: 10,
    ranking_options: {
        score_threshold: 0.4,
    },
    stream: true
});
return new Response(answer, {
    headers: {
        'Content-Type': 'text/event-stream'
    }
});

getting error

"logs": [
    {
      "message": [
        "SyntaxError: Unexpected token 'd', \"data: {\"no\"... is not valid JSON"
      ],
      "level": "error",
      "timestamp": 1744242318653
    }
  ]

"logs": [
    {
      "message": [
        "SyntaxError: Unexpected token 'd', \"data: {\"no\"... is not valid JSON"
      ],
      "level": "error",
      "timestamp": 1744242318653
    }
  ]

"logs": [
    {
      "message": [
        "SyntaxError: Unexpected token 'd', \"data: {\"no\"... is not valid JSON"
      ],
      "level": "error",
      "timestamp": 1744242318653
    }
  ]

"logs": [
    {
      "message": [
        "SyntaxError: Unexpected token 'd', \"data: {\"no\"... is not valid JSON"
      ],
      "level": "error",
      "timestamp": 1744242318653
    }
  ]

not sure what is wrong....

A

a4amaan•4/9/25, 11:53 PM

missing in autorage documentation...

S

Samir•4/10/25, 3:52 AM

Is there any way to increase the max input tokens? I want to use

bge-base-en-v1.5

bge-base-en-v1.5

bge-base-en-v1.5

bge-base-en-v1.5 (768 dimension) model

S

Samir•4/10/25, 3:54 AM

I actually want to read content of every pages of pdf, each page might have 1350 tokens

S

Samir•4/10/25, 3:56 AM

I think I might to have split in chunks

T

Tait•4/10/25, 4:58 PM

Hey all. If I wanted to extract audio from a video before hitting a workers AI audio endpoint what would be the most idiomatic was to do that with cloudflare infrastructure?

Apologies if this is a bit off topic but I figured someone in here would know the answer. My current solution is just to use lambda on AWS with ffmpeg but if there's an easy way to do this on CF I'd love to hear about it.

TTait Hey all. If I wanted to extract audio from a video before hitting a workers AI ...

K

klouzo35•4/11/25, 6:07 AM

I'm also searching for a solution, if you find anything

P

PPT•4/11/25, 7:31 AM

Hello all, the hyperlink in the screenshot does not seem to work, just an FYI!

N

nclevenger•4/11/25, 1:17 PM

the link on the blog post this morning to QwQ is 404: https://developers.cloudflare.com/workers-ai/models/qwq-32b

Nnclevenger the link on the blog post this morning to QwQ is 404: https://developers.cloudfl...

C

Chaika•4/11/25, 1:33 PM

they fixed now, looks like ~3 mins ago

Kklouzo35 I'm also searching for a solution, if you find anything

T

Tait•4/11/25, 2:38 PM

Look what just dropped.

https://blog.cloudflare.com/cloudflare-containers-coming-2025/#stateless-and-global-ffmpeg-everywhere

The Cloudflare Blog

Simple, scalable, and global: Containers are coming to Cloudflare W...

Cloudflare Containers are coming this June. Run new types of workloads on our network with an experience that is simple, scalable, global and deeply integrated with Workers.

TTait Look what just dropped. https://blog.cloudflare.com/cloudflare-containers-comin...

I

Isaac McFadyen•4/11/25, 3:03 PM

Note that containers are not yet available and probably won't be until at least June (per the blog post oday)

IIsaac McFadyen Note that containers are not yet available and probably won't be until at least ...

T

Tait•4/11/25, 3:10 PM

Totally fair. The announcement dropped. The product did not.

T

tutankhamen•4/11/25, 6:03 PM

melotts – Why don't languages other than English work? I tried Spanish and French and got AiError: AiError: Invalid input, code 8002

M

mr.niko.la•4/12/25, 4:57 AM

Workers AI grok is dope all with one endpoint.

Too bad grok pricing is ridiculous

A

andifined.dev•4/12/25, 10:30 AM

Hey I‘m trying to use the vision capabilities for this mistral model @cf/mistralai/mistral-small-3.1-24b-instruct

It doesn‘t seems to work yet. Is that right? Can‘t provide conversation history with content. Always says Type mismatch of '/messages/0/content', 'array' not in 'string'

const aiResponse: any = await env.AI.run(
"@cf/mistralai/mistral-small-3.1-24b-instruct",
{
prompt: prompt,
messages: [
{
role: "system",
content: "You're a helpful assistant."
}
]
}
);

Aandifined.dev Hey I‘m trying to use the vision capabilities for this mistral model @cf/mistral...

U

uhsheesh•4/12/25, 6:08 PM

hello ! You need to either send

messages

messages

messages

messages or

prompt

prompt

prompt

prompt

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: [
      { role: "system", content: "You're a helpful assistant." },
      { role: "assistant", content: prompt }
    ]
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: [
      { role: "system", content: "You're a helpful assistant." },
      { role: "assistant", content: prompt }
    ]
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: [
      { role: "system", content: "You're a helpful assistant." },
      { role: "assistant", content: prompt }
    ]
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: [
      { role: "system", content: "You're a helpful assistant." },
      { role: "assistant", content: prompt }
    ]
  }
);

or

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
     "prompt": prompt
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
     "prompt": prompt
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
     "prompt": prompt
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
     "prompt": prompt
  }
);

Uuhsheesh hello ! You need to either send `messages` or `prompt` ```typescript const aiRes...

A

andifined.dev•4/12/25, 6:11 PM

Ah okay Thank you very much! But let‘s say for example, I want to do vision analysis and provide 6 images, where do I Place the User prompt? The First 6 items as objects utilazing image_url and than the user promt to search for within those images?

U

uhsheesh•4/12/25, 6:25 PM

You'd do something like this:

const messages = [
{
  role: "system",
  content: "You are a helpful assistant that can analyze images."
},
{
  role: "user",
  content: [
    {
      type: "text",
      text: "What's in this image?"
    },
    {
      type: "image_url",
      image_url: {
        url: datauri

      }
    },
    ... // extra
  ]
}
];

const messages = [
{
  role: "system",
  content: "You are a helpful assistant that can analyze images."
},
{
  role: "user",
  content: [
    {
      type: "text",
      text: "What's in this image?"
    },
    {
      type: "image_url",
      image_url: {
        url: datauri

      }
    },
    ... // extra
  ]
}
];

const messages = [
{
  role: "system",
  content: "You are a helpful assistant that can analyze images."
},
{
  role: "user",
  content: [
    {
      type: "text",
      text: "What's in this image?"
    },
    {
      type: "image_url",
      image_url: {
        url: datauri

      }
    },
    ... // extra
  ]
}
];

const messages = [
{
  role: "system",
  content: "You are a helpful assistant that can analyze images."
},
{
  role: "user",
  content: [
    {
      type: "text",
      text: "What's in this image?"
    },
    {
      type: "image_url",
      image_url: {
        url: datauri

      }
    },
    ... // extra
  ]
}
];

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: messages
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: messages
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: messages
  }
);

const aiResponse: any = await env.AI.run(
  "@cf/mistralai/mistral-small-3.1-24b-instruct",
  {
    messages: messages
  }
);

Uuhsheesh You'd do something like this: ```typescript const messages = [ { role: "syste...

A

andifined.dev•4/13/25, 8:14 AM

Already tried that but it seams that the mistral model is not capable of analyzing images even it‘s possible to provide one?

Using the following request:

const aiResponse = await env.AI.run(
"@cf/mistralai/mistral-small-3.1-24b-instruct",
{
messages: [
{
role: 'system',
content: "You are a helpful assistant that analyzes images."
},
{
role: "user",
content: [
{
type: "image_url",
image_url: {
url: images[0],
}
},
{
type: "text",
text: prompt
}
]
}
]
}
);

I'm sorry, but I can't directly analyze images or videos.

What am I doing wrong?

Sshivam I'm encountering an Error: 10000: Authentication error when trying to connect to...

F

flashblaze•4/13/25, 8:08 PM

Were you able to solve this?

L

LeftyLlama•4/13/25, 8:16 PM

Will the MeloTTS model ever have more voices/voice customization options? It seems like MeloTTS can do this but I don't see anything about it in Cloudflare's documentation for it

A

ajgeiss0702•4/13/25, 9:30 PM

has there been any progress on the issue of reasoning models not returning valid reasoning responses?

A

ajgeiss0702•4/13/25, 9:30 PM

the issue still appears to exist on the newly-added qwq model

A

ajgeiss0702•4/13/25, 9:32 PM

please ping me if you reply

Aandyt Hey is there a file size limit for toMarkdown? I have tried a few larger files t...

I

ItsWendell•4/14/25, 12:28 PM

Curious to hear what file sizes are failing for you. I've actually made a toMarkdown function myself for PDFs, which uses unpdf to stream the contents per page to the response, minimizing memory usage.

C

cosbgn•4/15/25, 1:59 PM

Are there any options to pass to

ai/tomarkdown

ai/tomarkdown

ai/tomarkdown

ai/tomarkdown ?

I currently get a whole paragraph before the actual content of the PDF which I definitely don't want

 ## Metadata - PDFFormatVersion=1.4 - Language=en-GB - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Creator=Writer - Producer=OpenOffice.org 2.4 - CreationDate=D:20090324113315-06'00'

 ## Metadata - PDFFormatVersion=1.4 - Language=en-GB - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Creator=Writer - Producer=OpenOffice.org 2.4 - CreationDate=D:20090324113315-06'00'

 ## Metadata - PDFFormatVersion=1.4 - Language=en-GB - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Creator=Writer - Producer=OpenOffice.org 2.4 - CreationDate=D:20090324113315-06'00'

 ## Metadata - PDFFormatVersion=1.4 - Language=en-GB - IsLinearized=false - IsAcroFormPresent=false - IsXFAPresent=false - IsCollectionPresent=false - IsSignaturesPresent=false - Creator=Writer - Producer=OpenOffice.org 2.4 - CreationDate=D:20090324113315-06'00'

C

cosbgn•4/15/25, 2:02 PM

perhaps would be a good idea to have data as an object {metadata, content} ?

Ccosbgn Are there any options to pass to `ai/tomarkdown` ? I currently get a whole par...

I

Isaac McFadyen•4/15/25, 3:48 PM

It looks like this is one line in which case you could just remove any lines that start with "## Metadata"

M

michaelthatsit•4/16/25, 12:41 AM

I'm running into an issue where the response is being cut short. I've looked into the discord history here and can't seem to find an answer. any ideas?

Mmichaelthatsit I'm running into an issue where the response is being cut short. I've looked int...

I

Isaac McFadyen•4/16/25, 12:56 AM

max_tokens

max_tokens

max_tokens

max_tokens defaults to 256 tokens, try setting that higher in your request to the model.

M

michaelthatsit•4/16/25, 1:06 AM

it's unclear where to set that.

M

michaelthatsit•4/16/25, 1:15 AM

nvm I'm dumb

IIsaac McFadyen It looks like this is one line in which case you could just remove any lines tha...

C

cosbgn•4/16/25, 12:11 PM

In this case yes, but files are user submitted, I don't have control on how the metadata line will be. Is it always a single line? not sure

M

Muhammadreza Haghiri•4/16/25, 9:15 PM

Hello everyone. Is there a chance of having Flux Dev as well?

i think this is what your looking for, https://developers.cloudflare.com/workers-ai/features/json-mo

Similar Threads