I'll try using it like this, not ideal but maybe it will work.

J

James•10/22/25, 12:49 PM

Types for workers AI feel mostly abandoned at this point sadly. I’ve raised it again and again, both publicly and privately, and there’s never any real conclusion:

- https://github.com/cloudflare/workerd/issues/5080
- https://github.com/cloudflare/workerd/issues/2418
- https://github.com/cloudflare/workerd/issues/2181

If you want good DX, with a maintained platform, sdk, and reliable inference, I’d personally recommend using vercel’s ai sdk and OpenAI directly at this point unfortunately.

JJames Types for workers AI feel mostly abandoned at this point sadly. I’ve raised it a...

L

LaurenceOP•10/22/25, 1:24 PM

Thanks, I'll check it them out. I've currently been using fal.ai but it doesn't have all the models that I'm looking for.

LLaurence I'll try using it like this, not ideal but maybe it will work.

L

LaurenceOP•10/22/25, 8:56 PM

So, the method that I used above works but it's kind of finicky. The response is kind of inconsistent at it throws a lot of internal errors. So, I'm currently using the direct WebSocket connection according to Deepgram's website https://developers.deepgram.com/docs/flux/quickstart (works well). I'll try Cloudflare's implementation later.

Getting Started with Flux | Deepgram's Docs

Flux is the first conversational speech recognition model built specifically for voice agents. Unlike traditional STT that just transcribes words, Flux understands conversational flow and automatically handles turn-taking.

JJames Types for workers AI feel mostly abandoned at this point sadly. I’ve raised it a...

C

carter•10/23/25, 3:19 AM

Wow, that's really unfortunate. No indication that they'll work to improve workers AI? I'm a bit new to CF's inference, but what do you mean by (not) "reliable inference"? Is the latency all over?

S

samjs•10/23/25, 3:10 PM

Hey @James , @carter and others. Apologies for the slow progress here, and I understand and share the frustration.

We are absolutely working on fixing these issues, and we hope to have an update soon. We've got a little backed up on some work here, but should be getting around to it soon. To give you an idea of the kinds of things we've been working on:

- Improved platform stability and performance: we've recently rolled out a lot of "invisible" changes to make the platform more stable to things like high load and more consistently return successful responses. While there will still be occasions that people will see 3040 errors due to limited capacity, these should be much less often than before
- For the gpt-oss models, we've been waiting for some vLLM changes to land before pushing out more updates. In hindsight we shouldn't have let ourselves be blocked on this, and we're working to address this by seeing if we can build this ourselves internally. That's going to bring streaming, chat/completions support, and support for async requests. Along with a number of bug fixes.
- Support for realtime audio models like deepgram/flux, and websocket support for those.

On the DX front -- all the feedback that gets shared here/in GitHub is absolutely heard internally. e.g. you have @Kevin and I both core member of the Workers AI team checking this regularly.

Ssamjs Hey @James , @carter and others. Apologies for the slow progress here, and I un...

J

James•10/23/25, 3:33 PM

Thanks Sam, excited for updates. I'd love to see DX around types with model releases a clear focus, so that models aren't added to the platform with an immediately negative DX. Other platforms have types available with great DX on day one or two, especially for platforms they advertise as a "partner", but some models on Cloudflare end up weeks, or even months out of date, if not missing entirely.

S

samjs•10/23/25, 3:37 PM

Agreed. We have a process and tooling problem we need to fix here. On the former, as you say we've been way to slow to update types along with model releases. And on the latter, currently updating types is a somewhat manual process for us (which obviously feeds into the former).

S

samjs•10/23/25, 3:38 PM

Kevin and I have been discussing a bunch how we want to improve it. We'll get better

J

James•10/23/25, 3:40 PM

Great to hear. Let me know if there's anything I can do to help here!

C

carter•10/23/25, 3:41 PM

Thanks for the updates Sam! Appreciate the communication.

JJames Great to hear. Let me know if there's anything I can do to help here!

S

samjs•10/23/25, 3:46 PM

Thanks James! Will share with you updated types for feedback when we have them

!

! Tᐯ-GᗩᗰE.ᗪEᐯ•10/24/25, 10:46 AM

How can I send to a model with mcp server?

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-4-scout-17b-16e-instruct \
  -X POST \
  -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \
  -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}'

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-4-scout-17b-16e-instruct \
  -X POST \
  -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \
  -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}'

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-4-scout-17b-16e-instruct \
  -X POST \
  -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \
  -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}'

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-4-scout-17b-16e-instruct \
  -X POST \
  -H "Authorization: Bearer $CLOUDFLARE_AUTH_TOKEN" \
  -d '{ "messages": [{ "role": "system", "content": "You are a friendly assistant" }, { "role": "user", "content": "Why is pizza so good" }]}'

!! Tᐯ-GᗩᗰE.ᗪEᐯ How can I send to a model with mcp server? ```curl https://api.cloudflare.com/cl...

!

! Tᐯ-GᗩᗰE.ᗪEᐯ•10/24/25, 10:47 AM

my server: https://mcp.exa.ai/mcp

R

Razmjoo•10/25/25, 12:34 PM

Hi guys, any active incidents here? I am getting 408 error on @cf/openai/gpt-oss-120b, status 408;

{
  "error": {
    "code": "invalid_prompt",
    "message": "AiError: AiError: Request timeout (a613c7db-647c-4c07-ba94-a5702bc99d52)"
  }
}

{
  "error": {
    "code": "invalid_prompt",
    "message": "AiError: AiError: Request timeout (a613c7db-647c-4c07-ba94-a5702bc99d52)"
  }
}

{
  "error": {
    "code": "invalid_prompt",
    "message": "AiError: AiError: Request timeout (a613c7db-647c-4c07-ba94-a5702bc99d52)"
  }
}

{
  "error": {
    "code": "invalid_prompt",
    "message": "AiError: AiError: Request timeout (a613c7db-647c-4c07-ba94-a5702bc99d52)"
  }
}

R

Razmjoo•10/25/25, 12:42 PM

@sunbuhui do I click on general support or bug report? unsure which team is it?

R

Razmjoo•10/25/25, 12:46 PM

@sunbuhui could you please point it out;

R

Razmjoo•10/25/25, 12:48 PM

done

R

Razmjoo•10/25/25, 1:00 PM

Seriously, dude? @sunbuhui You're trying to scam people for their wallets in the Cloudflare channel full of security experts during debugging an error?

R

Razmjoo•10/25/25, 1:01 PM

@samjs would you please advise, and also ban the scammer user?

RRazmjoo Seriously, dude? @sunbuhui You're trying to scam people for their wallets in th...

Z

zegevlier•10/25/25, 1:03 PM

Thank you for the report, I have banned the user. For the next time, you can ping the green role (community champions) for moderation issues like this

Z

zegevlier•10/25/25, 1:04 PM

And just to confirm, yes, this is a scam. Cloudflare will never ask you to join another discord server for support

R

Razmjoo•10/25/25, 1:04 PM

he did zero efford; asked directly for 'wallet info' what an impatent scammer

S

samjs•10/25/25, 2:45 PM

Hey @Razmjoo , thanks for reporting! And thank you Zegevlier for jumping on it.

With respect to the timeouts: I'm not seeing any indications of an outage. I can see the spike in inference times that would have caused the timeouts, but there's no correlation with location/server or anything like that to suggest there's a bug. And no other accounts were experiencing any spikes. Were the inputs particularly long? E.g. using reasoning high as well?

R

Razmjoo•10/25/25, 6:32 PM

@samjs input were a bit long (maybe around 50,60 kb) but the reasoning was default;

R

Razmjoo•10/25/25, 6:33 PM

i mean total message (system + user + json schema) were in total around 50-60kb

R

Razmjoo•10/25/25, 7:10 PM

I have reduced them to 19kb but still the same issue.

I

Iñaki Fuentes•10/27/25, 8:31 AM

Hey team, anyone know what’s going on here? Looks like the counters reset and show 10k neurons available

but I’m still getting an API error.

Is this expected or is something borked? Lmk if you’ve seen this too.

  "error": {
        "message": "4006: you have used up your daily free allocation of 10,000 neurons, please upgrade to Cloudflare's Workers Paid plan if you would like to continue usage.",
        "stage": "pipeline",
        "code": "internal_error"
    },

    "timestamp": "2025-10-27T08:26:51.928Z"

  "error": {
        "message": "4006: you have used up your daily free allocation of 10,000 neurons, please upgrade to Cloudflare's Workers Paid plan if you would like to continue usage.",
        "stage": "pipeline",
        "code": "internal_error"
    },

    "timestamp": "2025-10-27T08:26:51.928Z"

  "error": {
        "message": "4006: you have used up your daily free allocation of 10,000 neurons, please upgrade to Cloudflare's Workers Paid plan if you would like to continue usage.",
        "stage": "pipeline",
        "code": "internal_error"
    },

    "timestamp": "2025-10-27T08:26:51.928Z"

  "error": {
        "message": "4006: you have used up your daily free allocation of 10,000 neurons, please upgrade to Cloudflare's Workers Paid plan if you would like to continue usage.",
        "stage": "pipeline",
        "code": "internal_error"
    },

    "timestamp": "2025-10-27T08:26:51.928Z"

IIñaki Fuentes Hey team, anyone know what’s going on here? Looks like the counters reset and sh...

I

Iñaki Fuentes•10/27/25, 9:04 AM

To pay is always a solution, solved...

M

MichaelC•10/27/25, 12:26 PM

Are there changes ongoing with the llama 4 model? The latency has gone through the roof for about a week now and it is very unstable. Can see some data here as well https://delay.chaika.me/ai/@cf/meta/llama-4-scout-17b-16e-instruct.

I

ItsAsheer•10/28/25, 7:19 AM

Hi, so I've been trying to connect LiteLLM (LLM proxy) to Cloudflare and when i try to send a message i get an error (check below), doing this with other services doesnt give me this error so i believe its Cloudflare.

Error code: 400 - {'success': False, 'errors': [{'code': 7000, 'message': 'No route for that URI'}], 'messages': [], 'result': None}

Error code: 400 - {'success': False, 'errors': [{'code': 7000, 'message': 'No route for that URI'}], 'messages': [], 'result': None}

Base Url: https ://api.cloudflare.com/client/v4/accounts/{my account id}/ai/v1

(the space between https and :// is on purpose to not format it like a link)

S

si.kiskre.•10/28/25, 9:12 AM

worker paid vs. free
isn't 300000 per month equals to 10000 per day ??

Z

zegevlier•10/28/25, 12:12 PM

Pretty much, yea. The difference is that with the 300k/month, you can (I think) use all of them up in a single day, then nothing for the rest of the month. With 10k/day, you cannot use any more than those 10k that day

S

si.kiskre.•10/28/25, 12:40 PM

yup

S

si.kiskre.•10/28/25, 12:40 PM

idk

D

David Raphi•10/28/25, 11:07 PM

Looking for input from folks testing Cloudflare Workers AI!
I’m using llama-3.1-8b-instruct-fast (free tier) — works fine until prompts go past ~9K tokens, then it starts ignoring system instructions and hallucinating (even though it’s supposed to support 128K context).
Anyone found free-tier models on Cloudflare that handle large contexts more reliably, or just work best for chat systems?
I’m testing a bunch and trying to build a list of the top free-tier models — any pointers would be awesome!

DDavid Raphi Looking for input from folks testing Cloudflare Workers AI! I’m using llama-3.1-...

R

Razmjoo•10/28/25, 11:19 PM

I have the same issue with gpt-oss-120b: it doesn't follow the JSON format from time to time. I am unsure whether these models are quantized. or something else is going on

DDavid Raphi Looking for input from folks testing Cloudflare Workers AI! I’m using llama-3.1-...

D

David Raphi•10/28/25, 11:28 PM

@Ahmad Awais @John Spurlock @nora @samjs

DDavid Raphi @Ahmad Awais @John Spurlock @nora @samjs

I

Isaac McFadyen•10/29/25, 12:52 AM

?pings

S

SuperHelpflare•10/29/25, 12:52 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

M

muhtasim•10/29/25, 9:03 AM

Hello
I am getting this error from my Worker AI suddenly, the system was working relatively good few days ago

{
  "httpCode": 408,
  "internalCode": 3046,
  "message": "AiError: AiError: Request timeout (a22d7817-d6c4-42a8-b5fb-1693703a1845)",
  "name": "AiError",
  "skipSentry": true,
  "description": "Request timeout"
}

{
  "httpCode": 408,
  "internalCode": 3046,
  "message": "AiError: AiError: Request timeout (a22d7817-d6c4-42a8-b5fb-1693703a1845)",
  "name": "AiError",
  "skipSentry": true,
  "description": "Request timeout"
}

{
  "httpCode": 408,
  "internalCode": 3046,
  "message": "AiError: AiError: Request timeout (a22d7817-d6c4-42a8-b5fb-1693703a1845)",
  "name": "AiError",
  "skipSentry": true,
  "description": "Request timeout"
}

{
  "httpCode": 408,
  "internalCode": 3046,
  "message": "AiError: AiError: Request timeout (a22d7817-d6c4-42a8-b5fb-1693703a1845)",
  "name": "AiError",
  "skipSentry": true,
  "description": "Request timeout"
}

what could be the reason? I don't find the internal code

3046

3046 can it happen that I am using max tokens? Because I am using base64 images

I am using this model

@cf/google/gemma-3-12b-it

@cf/google/gemma-3-12b-it

@cf/google/gemma-3-12b-it

@cf/google/gemma-3-12b-it

Can anyone help me please? any suggestions are welcome

D

DragoDiKomo•10/29/25, 3:58 PM

Sometimes I'm getting this error: Error: error code: 1031

While trying to use AI via a opennext worker

Z

zsueo•10/29/25, 7:14 PM

Hey Ive noticed a fairly large uptick in the rate I have been getting "AiError: Capacity temporarily exceeded, please try again. " is cloudflare workers ai stable enough to be relied on or will this issue persist into the future?

Z

zsueo•10/29/25, 7:16 PM

I mean right now I am getting 500 errors from workers ai through the cloudflare rest api

C

Celestial Rose•10/29/25, 10:29 PM

hey there !

trying to use MeloTTS in French but I get this error, the documentation does mention French (and MeloTTS actually should support way more languages btw)
any guesses ?

Capture_decran_2025-10-30_a_00.28.14.png

Capture_decran_2025-10-30_a_00.28.40.png

CCelestial Rose hey there ! 🙂 trying to use MeloTTS in French but I get this error, the documen...

C

Chaika•10/30/25, 12:20 AM

last mentioned they said they only support en workers-ai

CChaika last mentioned they said they only support en https://discord.com/channels/59531...

C

Celestial Rose•10/30/25, 6:52 AM

thanks! usually have the reflex on looking up first but forgot !
alrighty - but still its either proposing other languages or fixing the doc right ? how can i contribute ?

IIsaac McFadyen ?pings

D

David Raphi•10/30/25, 7:07 AM

I was surprised to see that this discord channel has an overall message of below 20 for a day. Great news, might be Cloudflare is perfect and has no issues, or might be people don't have any hope finding any solutions here. I handle a discord channel where overall daily message is above 50, and I find it ok to handle. If one ping was disturbing you, I am really sorry for that, and thank you for the help. It was really helpful getting nothing.

DDavid Raphi Looking for input from folks testing Cloudflare Workers AI! I’m using llama-3.1-...

A

Ashkan•10/30/25, 8:51 AM

Maybe this is your answer: "Batch processing is useful for large workloads such as summarization or embeddings when there is no human interaction. Using the batch API will guarantee that your requests are fulfilled eventually, rather than erroring out if Cloudflare does have enough capacity at a given time" https://developers.cloudflare.com/workers-ai/features/batch-api/

Cloudflare Docs

Asynchronous Batch API

Asynchronous batch processing lets you send a collection (batch) of inference requests in a single call. Instead of expecting immediate responses for every request, the system queues them for processing and returns the results later.

K

KZ•10/30/25, 3:24 PM

Which cloudflare group has the ball on web-bot-auth specifically the http message signature (9421) is that this group or agents?

F

Fra3957•10/31/25, 8:59 AM

Hi everyone — I have a question, and I’m not sure if this is the right place to ask. I’m trying to understand which Cloudflare product would be the best fit for my use case.

I’m building an application that, when a user clicks a button, needs to trigger an external process that runs independently from the app itself.
In my view, this process should follow three main steps:
1. Fetch data from the database
2. Call an LLM many times in parallel, maximizing throughput within a limited execution time
3. Write the results back to the database

This entire operation should happen outside the main application, which will later display the updated data once the process is completed. The goal is to run this background workflow efficiently and reliably on Cloudflare’s infrastructure.

I’m considering different ways to orchestrate these jobs:

(A) The application inserts items into a queue on Supabase and then triggers a process that reads and processes the queue.

(B) The queue itself automatically triggers the processing when new messages arrive, though I’m concerned this might rely on polling, which doesn’t seem the most elegant or efficient approach.

(C) The application sends the data directly for processing at the time of activation, without using any intermediate queue.

Each user can initiate this process multiple times in parallel, so concurrency, execution duration, and timeout management are important considerations.

Question: Based on these requirements, which Cloudflare products or combination of products would be best suited to implement this architecture?

If this isn’t the right place to ask — or if Workers / Workers AI aren’t the right tools for this kind of problem — could you please point me to the best channel or community to ask this question?
Thanks a lot!

I'll try using it like this, not ideal but maybe it will work.

Similar Threads