When using `@cf/meta/llama-3.1-8b-instruct-fp8-fast` or `@cf/meta/llama-3.2-3b-instruct` I'm gettin

B

blindChickenOP•7/4/25, 12:03 AM

Okay! Found the fix, type must

json_object

json_object

json_object

json_object and not

json_schema

json_schema

json_schema

json_schema like so:

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

So error in the docs?

B

blindChickenOP•7/4/25, 12:10 AM

While I'm ranting about the docs, it would be nice if the json format of the API Schema would be collapsible, as it is very long and hard to follow indentation down the page. I copy it to VS-code when I need it, but it would be great if would be included on the page as UI element.

S

Sinisha Djukic•7/4/25, 6:41 AM

I have an issue with trying out MeloTTS - it doesn't seem to be in the AiModels list as a constant, even though I'm using @Cloudflare/ai version 1.2.2, which seems to be the latest. Anyone has this issue?

K

kwang•7/4/25, 7:04 AM

think the authorization method is no longer supported. would be good to update the docs

S

scotto•7/4/25, 12:58 PM

can we ask for some of the great qwen3 models? for example https://huggingface.co/Qwen/Qwen3-Reranker-4B

Qwen/Qwen3-Reranker-4B · Hugging Face

S

scotto•7/6/25, 3:10 PM

How to obtain sparse embeddings separated in M3?

C

Cedrick•7/7/25, 1:21 PM

Hello everyone, does anyone here already called LLAMA4 (

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct) with some tools/ function calling. With or without the compatibility mode (https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) i got answer with no tool results, and no Ai message either. I am using an official Java Client library... would like to know if i need to keep pushing (tried with strict tools) and is it expected ?

Cloudflare Docs

OpenAI compatible API endpoints

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions) and text embedding models (/v1/embeddings). This allows you to use the same code as you would for your OpenAI commands, but swap in Workers AI easily.

BblindChicken When using `@cf/meta/llama-3.1-8b-instruct-fp8-fast` or `@cf/meta/llama-3.2-3b-...

Ꭱ

ᎡᴜᴛʜʟᴇssŦᴀᴄᴛɪᴏɴ•7/7/25, 3:42 PM

The docs are generated wrong. I reported and docs team said it was out of their hands. You can see the list of supported models at the bottom of this page however: https://developers.cloudflare.com/workers-ai/features/json-mode/

Interesting that you could coax a json object with

json_object

json_object

json_object

json_object however! I'll have to check that out

Cloudflare Docs

JSON Mode

When we want text-generation AI models to interact with databases, services, and external systems programmatically, typically when using tool calling or building AI agents, we must have structured response formats rather than natural language.

Ꭱ

ᎡᴜᴛʜʟᴇssŦᴀᴄᴛɪᴏɴ•7/7/25, 4:20 PM

On a unrealted note:

Anyone have insight as to why I consistently am getting the same output for text-generation for the same prompt? I explicitly have tried setting

{ temperature: 5, top_p: 1, seed: Math.floor(Math.random() * 9999999999) + 1 }

{ temperature: 5, top_p: 1, seed: Math.floor(Math.random() * 9999999999) + 1 }

{ temperature: 5, top_p: 1, seed: Math.floor(Math.random() * 9999999999) + 1 }

{ temperature: 5, top_p: 1, seed: Math.floor(Math.random() * 9999999999) + 1 } to attempt to encourage different results every run... but no luck

M

Marco Aleixo•7/10/25, 2:27 PM

BGE-M3 batch request failed after all retry attempts
    details: {
    "attempts": 4,
    "error": "10000: Authentication error",
    "batchSize": 1
  }

BGE-M3 batch request failed after all retry attempts
    details: {
    "attempts": 4,
    "error": "10000: Authentication error",
    "batchSize": 1
  }

BGE-M3 batch request failed after all retry attempts
    details: {
    "attempts": 4,
    "error": "10000: Authentication error",
    "batchSize": 1
  }

BGE-M3 batch request failed after all retry attempts
    details: {
    "attempts": 4,
    "error": "10000: Authentication error",
    "batchSize": 1
  }

this is getting very common locally, not only on my environment but my colleagues as well. are we miss using something?

G

GoldenCrow•7/10/25, 4:01 PM

hey guys,
new here, trying to workout a simple thing but i'm blocked.
Is there a way to have a python worker call workers ai env.ai.run()?

C

cqm21•7/13/25, 1:28 PM

hey guys,

When trying to use Workers AI I'm getting this error: 4006: you have used up your daily free allocation of 10,000 neurons, please upgrade to Cloudflare's Workers Paid plan if you would like to continue usage.

In dashboard it's currently showing 0/10k

"Daily usage (resets at 00:00 UTC)-Neurons used today: 0/10k"

Have anyone experienced something similar?

Y

yinxingmaiming6409•7/13/25, 3:24 PM

Can we use QWEN? Why is there no default model? autorag

Ggabrielstuff opened an isue : https://community.cloudflare.com/t/gemma-aierror-3040-capacity-...

B

brylan•7/14/25, 3:09 AM

did you fix this?

B

brylan•7/14/25, 3:11 AM

Cloudflare AI API error: {"errors":[{"message":"AiError: AiError: could not route request to AI model (1b738130-21eb-47d2-a741-41984e3e71e1)","code":4002}],"success":false,"result":{},"messages":[]}

Cloudflare AI API error: {"errors":[{"message":"AiError: AiError: could not route request to AI model (1b738130-21eb-47d2-a741-41984e3e71e1)","code":4002}],"success":false,"result":{},"messages":[]}

Cloudflare AI API error: {"errors":[{"message":"AiError: AiError: could not route request to AI model (1b738130-21eb-47d2-a741-41984e3e71e1)","code":4002}],"success":false,"result":{},"messages":[]}

Cloudflare AI API error: {"errors":[{"message":"AiError: AiError: could not route request to AI model (1b738130-21eb-47d2-a741-41984e3e71e1)","code":4002}],"success":false,"result":{},"messages":[]}

B

brylan•7/14/25, 3:11 AM

B

brylan•7/14/25, 3:11 AM

'

B

brylan•7/14/25, 3:14 AM

B

brylan•7/14/25, 3:14 AM

I dont understand why this is having issues? I used it a while ago and it worked fine. Now its full of errors and bugs

B

brylan•7/14/25, 3:19 AM

errorText: '{"errors":[{"message":"AiError: AiError: Capacity temporarily exceeded, please try again. (5f37591a-d0ff-4304-8625-6d0fa8035c96)","code":3040}],"success":false,"result":{},"messages":[]}',

??

B

brylan•7/14/25, 3:20 AM

Im well within my nueron count

B

brylan•7/14/25, 3:21 AM

B

brylan•7/14/25, 3:23 AM

If I go into workers paid will this issue resolve itself?

Bbrylan errorText: '{"errors":[{"message":"AiError: AiError: Capacity temporarily exceed...

C

Chaika•7/14/25, 2:12 PM

Nothing to do with Neuron count nor plan level. Sometimes capacity for these models just waxes and wanes, as far as I can see, as users scale up and the infra does as well. I saw a short(ish) period of this yesterday but seems fine now

C

Chaika•7/14/25, 2:16 PM

(ish) is doing some heavy lifting

C

Chaika•7/14/25, 2:17 PM

it wasn't 100% errors during that time but it was high, granted I'm only doing 4 requests a minute to the model (this one is in proper utc time)

CChaika Nothing to do with Neuron count nor plan level. Sometimes capacity for these mod...

B

brylan•7/14/25, 4:04 PM

I understand with scaling

B

brylan•7/14/25, 4:04 PM

But from a consumer perspective this is kinda unacceptable no?

B

brylan•7/14/25, 4:04 PM

4 requests per minute and your seeing errors? I mean with any other service this would just be crazy

B

brylan•7/14/25, 4:15 PM

Being said I love cloudflare ai, but this does seem like this should be talked about internally or somthing

Bbrylan I understand with scaling

C

Chaika•7/14/25, 4:23 PM

Just saying in case it's worth noting: Green/champs are just volunteers and the AI team has said things in the past about needing to scale models, espec after launches/big increases: workers-ai, which is where I'm saying that from, but they did not comment on this exact issue/rise in errors (nor was there a status page). I track those capacity errors on my own (unrelated to the team) and I do see them come and go for various models, and it does seem to be around just scaling and such

B

brylan•7/14/25, 4:40 PM

ok thanks

R

razor•7/14/25, 8:57 PM

Hey, Is the @cf/openai/whisper-large-v3-turbo model workers only ?? do we also have an API endpoint for this like a CURL way of doing it ??

L

lachlan•7/15/25, 3:15 AM

is there a place we can request new open source models? https://developers.cloudflare.com/workers-ai/models/ I am wanting to use the new Kimi K2?

Cloudflare Docs

Models

N

nclevenger•7/15/25, 2:45 PM

is there a problem with AI.toMarkdown from HTML? I'm just getting raw HTML whenever passing html in:

export default {
  async fetch(req, env) {
    const response = await fetch('https://example.com')
    const blob = await response.blob()
    const markdown = await env.ai.toMarkdown({ name: 'example.com', blob })
    return Response.json(markdown )
  }
}

export default {
  async fetch(req, env) {
    const response = await fetch('https://example.com')
    const blob = await response.blob()
    const markdown = await env.ai.toMarkdown({ name: 'example.com', blob })
    return Response.json(markdown )
  }
}

export default {
  async fetch(req, env) {
    const response = await fetch('https://example.com')
    const blob = await response.blob()
    const markdown = await env.ai.toMarkdown({ name: 'example.com', blob })
    return Response.json(markdown )
  }
}

export default {
  async fetch(req, env) {
    const response = await fetch('https://example.com')
    const blob = await response.blob()
    const markdown = await env.ai.toMarkdown({ name: 'example.com', blob })
    return Response.json(markdown )
  }
}

returns:

{
  "name": "example.com",
  "mimeType": "text/html",
  "format": "markdown",
  "tokens": 314,
  "data": "<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset=\"utf-8\" />\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n    <style type=\"text/css\">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n"
}

{
  "name": "example.com",
  "mimeType": "text/html",
  "format": "markdown",
  "tokens": 314,
  "data": "<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset=\"utf-8\" />\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n    <style type=\"text/css\">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n"
}

{
  "name": "example.com",
  "mimeType": "text/html",
  "format": "markdown",
  "tokens": 314,
  "data": "<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset=\"utf-8\" />\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n    <style type=\"text/css\">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n"
}

{
  "name": "example.com",
  "mimeType": "text/html",
  "format": "markdown",
  "tokens": 314,
  "data": "<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset=\"utf-8\" />\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n    <style type=\"text/css\">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n"
}

V

Vladimir_dev•7/16/25, 11:19 AM

Feature Request: Add MedGemma Models to Cloudflare Workers AI
Dear Cloudflare Workers AI Team,
I would like to request the addition of MedGemma models to the Workers AI platform. MedGemma is Google's specialized collection of medical AI models that would significantly enhance the platform's capabilities for healthcare applications.
Key Benefits:

Compact Size: MedGemma models are relatively small, making them well-suited for edge deployment
Medical Specialization: These models are specifically trained for medical use cases, filling a current gap in Workers AI's model offerings
Healthcare Applications: Would enable developers to build medical applications directly on Cloudflare's edge infrastructure
Cost-Effective: Smaller model sizes mean lower computational costs while maintaining medical domain expertise

Use Cases:

Medical chatbots and virtual assistants
Clinical decision support tools
Medical document analysis
Healthcare workflow automation

Reference:
MedGemma models are available in Google's model collection and have been designed with medical applications in mind.
Adding MedGemma would position Cloudflare Workers AI as a leading platform for healthcare AI applications, combining the benefits of edge computing with specialized medical AI capabilities.
Thank you for considering this request. I believe this addition would greatly benefit the developer community working on healthcare solutions.
Best regards, Vladimir

MedGemma Release - a google Collection

Original message was deleted

V

Vladimir_dev•7/16/25, 11:24 AM

Claude sonnet4)))
Simulated Pre-visit Intake Demo by google
https://huggingface.co/spaces/google/appoint-ready

Appoint Ready - MedGemma Demo - a Hugging Face Space by google

Llachlan is there a place we can request new open source models? https://developers.cloud...

V

vmg•7/16/25, 2:52 PM

seconded, kimi k2 would be awesome

T

trace•7/17/25, 10:24 AM

Hello cloudflare team, the links mentioned in this url for terms and license and more information is not working https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-img2img/

Cloudflare Docs

stable-diffusion-v1-5-img2img

Stable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images. Img2img generate a new image from an input image with Stable Diffusion.

T

trace•7/17/25, 12:13 PM

Tried dreamshaper-8-lcm model through workers AI and the quality is ~~absolutely terrible~~. Is there an issue happening at your end?

edit: need to lower the guidance value for better quality

N

NekoKenma•7/19/25, 8:27 PM

Hey guys! I'm trying to use workers AI with the openai compatible endpoint. While it's been great for text generation, I wanted to plug it into n8n as the model for the AI agent, since the endpoint is openai compatible, and the model I'm using (

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct) does support tool calls as per this page.

The problem is that it's not working in agents that have tools to call. Upon looking further into the problem, there was a thread in n8n's GitHub issues

#15862

#15862

#15862

#15862 that explored this exact issue. Apparently, the tool call input format in the response from the cf model was wrong, and hence langchain would pick it up as a normal response (more details in the github thread).

Would really appreciate some response on this. Thanks.

N

NekoKenma•7/20/25, 5:55 PM

Devs?

NNekoKenma Hey guys! I'm trying to use workers AI with the openai compatible endpoint. Whil...

C

creet•7/21/25, 6:46 AM

do you have the actual example? or do you have a specific question?

Ccreet do you have the actual example? or do you have a specific question?

N

NekoKenma•7/21/25, 11:35 AM

The last post of the gh thread documents exactly what I'm facing, basically the OpenAI compatible endpoint, when plugged as a model for the langchain agent with tools, does not work.

NNekoKenma The last post of the gh thread documents exactly what I'm facing, basically the ...

C

creet•7/22/25, 12:32 AM

i see, looking at the docs Workers AI doesnt natively support "Tool Calling" the way the openai SDK or others does. where tool calling in other SDKs parses the <tool_call> portion of the generated text for you. its pretty straightforward to do yourself tho, im not sure if/how you would do that in n8n but with code you would just have a method that wraps the llm generation, collects the text between the <tool_call> and </tool_call> tags, and then streams the remainder as the text response.

C

creet•7/22/25, 12:33 AM

oh nevermind it looks like the docs does say that they support openai tool calling

C

creet•7/22/25, 12:34 AM

okay yeah so it is broken but a workaround is like how i mentioned above

Ccreet okay yeah so it is broken but a workaround is like how i mentioned above

N

NekoKenma•7/22/25, 12:36 AM

Actually, I think it's not exactly broken - but can be a feature request maybe. You see, the openai compatible endpoint page clearly states its for text generation and embedding models only, so I suppose that doesn't include tool calling.

Thanks, ig I'll just have to do what you said, make a wrapper worker.

Cloudflare Docs

OpenAI compatible API endpoints

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions) and text embedding models (/v1/embeddings). This allows you to use the same code as you would for your OpenAI commands, but swap in Workers AI easily.

When using `@cf/meta/llama-3.1-8b-instruct-fp8-fast` or `@cf/meta/llama-3.2-3b-instruct` I'm gettin

Similar Threads

Similar Threads

Similar Threads