Workers AI released so many models, which is exciting! Some models support WebSocket, which is aweso

J

pricing for the deepgram nova models should be based on audio input, right? Here is a quick PR to tweak the pricing page wording https://github.com/cloudflare/cloudflare-docs/pull/24763

GitHub

Prices should be per input for audio input models by johnspurlock-s...

Aakazwz Workers AI released so many models, which is exciting! Some models support WebSo...

S

samjs•8/28/25, 2:55 PM

I think we're still working on the documentation! But in short it should look something like:

export default {
  async fetch(request, env, ctx): Promise<Response> {
    const resp = await env.AI.run("@cf/deepgram/aura-1", {}, {
      websocket: true
    });
    return resp;
  },
} satisfies ExportedHandler<Env>;

export default {
  async fetch(request, env, ctx): Promise<Response> {
    const resp = await env.AI.run("@cf/deepgram/aura-1", {}, {
      websocket: true
    });
    return resp;
  },
} satisfies ExportedHandler<Env>;

export default {
  async fetch(request, env, ctx): Promise<Response> {
    const resp = await env.AI.run("@cf/deepgram/aura-1", {}, {
      websocket: true
    });
    return resp;
  },
} satisfies ExportedHandler<Env>;

export default {
  async fetch(request, env, ctx): Promise<Response> {
    const resp = await env.AI.run("@cf/deepgram/aura-1", {}, {
      websocket: true
    });
    return resp;
  },
} satisfies ExportedHandler<Env>;

JJohn Spurlock pricing for the deepgram nova models should be based on audio *input*, right? H...

S

samjs•8/28/25, 3:01 PM

Thanks for flagging! Checking with the team, but yes that also sounds right to me given that it's an audio -> text model

Ssamjs I think we're still working on the documentation! But in short it should look so...

J

John Spurlock•8/29/25, 4:17 PM

thanks this is helpful for bindings. What is the equivalent way of using websockets over the CF REST api? https://developers.cloudflare.com/api/resources/ai/methods/run/

J

John Spurlock•8/29/25, 4:27 PM

how does the cf partnership thing work with deepgram in terms of the "model improvement plan" - does setting

mip_opt_out=true

mip_opt_out=true

mip_opt_out=true

mip_opt_out=true affect the unit pricing stated on the cf api page? (when using deepgram directly that significantly increases the price)

https://developers.cloudflare.com/workers-ai/models/nova-3/

Ssamjs FYI we have a fix going out for this issue shortly

H

Himanshu•8/30/25, 5:48 PM

Is nova-3 operational now? I'm still getting the same error as before in the API.

A

Ana•8/31/25, 4:45 PM

Hello. I’m experimenting with Traditional function calling and it seems to only work with one of the 4 models that should support it:

@hf/nousresearch/hermes-2-pro-mistral-7b

@hf/nousresearch/hermes-2-pro-mistral-7b

@hf/nousresearch/hermes-2-pro-mistral-7b

@hf/nousresearch/hermes-2-pro-mistral-7b

For the other models I get the response:

 {
  "errors": [
    {
      "message": "AiError: AiError: Invalid input",
      "code": 8001
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

 {
  "errors": [
    {
      "message": "AiError: AiError: Invalid input",
      "code": 8001
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

 {
  "errors": [
    {
      "message": "AiError: AiError: Invalid input",
      "code": 8001
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

 {
  "errors": [
    {
      "message": "AiError: AiError: Invalid input",
      "code": 8001
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

To stop receiving this message I need to give a type: “function” to my tools and wrap them in a function:

tools: [
    {
      type: “function”,
      function: { 
        name: "getWeather",
        description: "Return the weather for a latitude and longitude",
        parameters: {
          // ....
        },
      },
    }],

tools: [
    {
      type: “function”,
      function: { 
        name: "getWeather",
        description: "Return the weather for a latitude and longitude",
        parameters: {
          // ....
        },
      },
    }],

tools: [
    {
      type: “function”,
      function: { 
        name: "getWeather",
        description: "Return the weather for a latitude and longitude",
        parameters: {
          // ....
        },
      },
    }],

tools: [
    {
      type: “function”,
      function: { 
        name: "getWeather",
        description: "Return the weather for a latitude and longitude",
        parameters: {
          // ....
        },
      },
    }],

But this way none of the other 3 models (

@cf/meta/llama-3.3-70b-instruct-fp8-fast

@cf/meta/llama-3.3-70b-instruct-fp8-fast

@cf/meta/llama-3.3-70b-instruct-fp8-fast

@cf/meta/llama-3.3-70b-instruct-fp8-fast,

@cf/mistralai/mistral-small-3.1-24b-instruct

@cf/mistralai/mistral-small-3.1-24b-instruct

@cf/mistralai/mistral-small-3.1-24b-instruct

@cf/mistralai/mistral-small-3.1-24b-instruct,

@cf/meta/llama-4-scout-17b-16e-instruct)

@cf/meta/llama-4-scout-17b-16e-instruct)

@cf/meta/llama-4-scout-17b-16e-instruct)

@cf/meta/llama-4-scout-17b-16e-instruct) can use function calling at all (the reply is always text).

Anyone else with the same issue?

J

joe•8/31/25, 8:11 PM

are there any plans to expand the options that can be passed into runWithTools? not being able to set max output tokens is a real pain

AAna Hello. I’m experimenting with Traditional function calling and it seems to only ...

J

joe•8/31/25, 8:27 PM

I can also reproduce this

AAna Hello. I’m experimenting with Traditional function calling and it seems to only ...

P

Proman4713 🇵🇸•9/1/25, 9:50 AM

i have the same issue

B

bpence•9/1/25, 5:11 PM

Is there anyone here that could help with a whisper ai issue? I've got the full flow working (r2/whisper/kv) but the transcript text is really bad with lots of repetition of sentences and cut-off words. I use whisper on my local GPU and it's nearly perfect, but whisper on CF is pretty bad no matter which model I use.

PProman4713 🇵🇸i have the same issue

S

samjs•9/3/25, 12:43 PM

Sorry for the delay folks. I'm out of office this week but will look into this next week if none of my colleagues get to it first

Ssamjs Sorry for the delay folks. I'm out of office this week but will look into this n...

P

Proman4713 🇵🇸•9/3/25, 2:20 PM

Alright, I can work on other things until then

AAna Hello. I’m experimenting with Traditional function calling and it seems to only ...

Z

zsueo•9/3/25, 11:58 PM

huh i can get function calling to work but for llama instruct it sometimes just hangs without giving a response

Z

zsueo•9/3/25, 11:58 PM

id say it works around 30% of the time

Z

zsueo•9/3/25, 11:59 PM

it also likes to leak the function call into plain text

Z

zsueo•9/4/25, 12:11 AM

is the api docs for the traditional tool calling out of date?

J

jd•9/4/25, 5:54 AM

im super excited about workers ai https://blog.cloudflare.com/workers-ai/

and curious what's the latest status on it? im primarily looking into deploying my custom python script + model with compilation graph cache and see if cloudflare is able to run it at global scale serverless. with great cold start time.

The Cloudflare Blog

Workers AI: serverless GPU-powered inference on Cloudflare’s glob...

We are excited to launch Workers AI - an AI inference as a service platform, empowering developers to run AI models with just a few lines of code, all powered by our global network of GPUs

J

jd•9/4/25, 5:55 AM

for models im mean tasks like

1) upscaling
2) voice gen
3) ad-hoc AI processing
4) or even large diffusion transformer models that could take minimum of H100 to run if that's on the roadmap

Zzsueo is the api docs for the traditional tool calling out of date?

Z

zsueo•9/4/25, 10:04 AM

actually is there documentation on how the ai is supposed to return tool calls?

Y

yureckey•9/4/25, 10:12 AM

I'm testing aura-1 text2speech. There is no mentioning of Text input limit, but it seems to have 2k symbols limit (as I understand Deepgram have this limit). When the text is bigger - there is no proper error message, please fix (and add to doc)

Zzsueo is the api docs for the traditional tool calling out of date?

A

Ana•9/4/25, 11:56 AM

Probably, though it works for the hermes model. https://developers.cloudflare.com/workers-ai/features/function-calling/

AAna Probably, though it works for the hermes model. https://developers.cloudflare.co...

Z

zsueo•9/4/25, 12:03 PM

thanks! though i think the problem im facing is that A) my code was buggy and B) the input schema changed

Z

zsueo•9/4/25, 12:03 PM

took me a while to figure that out tho

SSunil Pai folks, we just published v2.0.0 of workers-ai-provider, the main feature of whic...

P

Proman4713 🇵🇸•9/4/25, 5:34 PM

does this fix the issue we were talking about?

SSunil Pai sorry, not sure what issue you're referring to

P

Proman4713 🇵🇸•9/4/25, 5:34 PM

AiError: Invalid Input 8081 when trying to use traditional function calling

P

Proman4713 🇵🇸•9/4/25, 5:35 PM

the response from the AI request (decoded from base64 after setting log-level to debug and turning off sanitisation):

{"httpCode":400,"internalCode":8001,"message":"AiError: AiError: Invalid input (95e650b5-30fd-461e-a800-95d18c5295ef)","name":"AiError","skipSentry":true,"description":"Invalid input"}

{"httpCode":400,"internalCode":8001,"message":"AiError: AiError: Invalid input (95e650b5-30fd-461e-a800-95d18c5295ef)","name":"AiError","skipSentry":true,"description":"Invalid input"}

{"httpCode":400,"internalCode":8001,"message":"AiError: AiError: Invalid input (95e650b5-30fd-461e-a800-95d18c5295ef)","name":"AiError","skipSentry":true,"description":"Invalid input"}

{"httpCode":400,"internalCode":8001,"message":"AiError: AiError: Invalid input (95e650b5-30fd-461e-a800-95d18c5295ef)","name":"AiError","skipSentry":true,"description":"Invalid input"}

SSunil Pai just saw it. this wasn't related to that, this is the library on top of workers-...

P

Proman4713 🇵🇸•9/4/25, 5:36 PM

oh okay, but still this issue works whenever i add

tools

tools

tools

tools to my AI.run function

P

Proman4713 🇵🇸•9/4/25, 5:37 PM

i was in the wrong channel, but these are the details: coding-help

SSunil Pai just saw it. this wasn't related to that, this is the library on top of workers-...

P

Proman4713 🇵🇸•9/4/25, 6:32 PM

uhhh sorry i meant "this issue happens" not works

P

Proman4713 🇵🇸•9/4/25, 6:32 PM

it doesnt work

P

Proman4713 🇵🇸•9/4/25, 6:32 PM

it shows this error whenever i try to use traditional function calling

S

starcarlos.•9/5/25, 3:00 AM

When I use worker ai, I get an error.
429 , reason: Too Many Requests,text:{"errors":[{"message":"AiError: AiError: Capacity temporarily exceeded, please try again.

M

maj•9/5/25, 5:52 AM

I'm trying to use AutoRAG to spell check names. I have a list of names in a R2 bucket (split in ~3MB chunks). AutoRAG claims it finished processing them. I then use

c.env.AI.autorag('my-rag-name').aiSearch()

c.env.AI.autorag('my-rag-name').aiSearch()

c.env.AI.autorag('my-rag-name').aiSearch()

c.env.AI.autorag('my-rag-name').aiSearch() to ask it basically "give me the correct version of the given name". Hoping it would fix the spelling of "Suma Covjek" to "Šuma Čovjek", as listed in one of the files. But results are underwhelming. When I looked at the AI Gateway and what the

aiSearch()

aiSearch()

aiSearch()

aiSearch() used for the default system message, I see "...no relevant documents were retrieved for the user's query...". For each and all of them. If I do a search in the playground it would sometimes return a file, but often not the right or relevant one, even if the exact correct spelling is given. What am I missing? How come

aiSearch()

aiSearch()

aiSearch()

aiSearch() would not find my content?

D

Daniel•9/5/25, 3:09 PM

are the types for the new deepgram models not up-to-date?

I am getting:

Argument of type '"@cf/deepgram/nova-3"' is not assignable to parameter of type 'keyof AiModels'.ts(2345)

this.env.AI.run(
        "@cf/deepgram/nova-3"

Argument of type '"@cf/deepgram/nova-3"' is not assignable to parameter of type 'keyof AiModels'.ts(2345)

this.env.AI.run(
        "@cf/deepgram/nova-3"

Argument of type '"@cf/deepgram/nova-3"' is not assignable to parameter of type 'keyof AiModels'.ts(2345)

this.env.AI.run(
        "@cf/deepgram/nova-3"

Argument of type '"@cf/deepgram/nova-3"' is not assignable to parameter of type 'keyof AiModels'.ts(2345)

this.env.AI.run(
        "@cf/deepgram/nova-3"

P

Pingo•9/8/25, 1:59 PM

Hi friends, has someone been able to add workers-ai to vs code extension cline or similar? I've been trying for months with no success. I tried the openai-compat providor route but kept getting errors. I've already connected in cline groq, mistral, cerebus, openrouter, gemini, and it was quite straight forward. Workers-ai in cline would be awesome. Btw I got curl test to work easily with and without ai gateway.

Ssamjs Sorry for the delay folks. I'm out of office this week but will look into this n...

P

Proman4713 🇵🇸•9/8/25, 5:11 PM

any news?

Y

yureckey•9/9/25, 5:05 AM

https://developers.cloudflare.com/workers-ai/models/lucid-origin/
there are two identical parameters "steps" and "num_steps" - so, which is correct? or I can use either one?

Cloudflare Docs

lucid-origin

Lucid Origin from Leonardo.AI is their most adaptable and prompt-responsive model to date. Whether you're generating images with sharp graphic design, stunning full-HD renders, or highly specific creative direction, it adheres closely to your prompts, renders text with accuracy, and supports a wide array of visual styles and aesthetics – from ...

Yyureckey https://developers.cloudflare.com/workers-ai/models/lucid-origin/ there are two ...

S

samjs•9/9/25, 3:11 PM

Both are fine!

H

Himanshu•9/9/25, 8:05 PM

Wish there were an easy way to try Cloudflare Workers AI models without any setup? I got you…

Well, I just shipped OpenGPT

Live demo: https://opengpt.weblinq.dev/
GitHub: https://github.com/devhims/opengpt

Built with OpenNext, AI SDK, and Workers AI. You can access 50+ models without signup.

If you find it useful, consider giving the repo a star. Thanks!

H

Himanshu•9/10/25, 7:54 PM

I just tried to use deepgram/nova-3 and it still doesn't seem to work. Has anyone used it successfully so far?

HHimanshu I just tried to use deepgram/nova-3 and it still doesn't seem to work. Has anyon...

H

Himanshu•9/10/25, 8:15 PM

Update: I've been able to get it working with the workers binding api. The requests in Postman are failing with "Error: Network connection lost."

T

toadilywasted•9/11/25, 1:42 AM

Any chance we'll get a new llamaguard model anytime soon?

AAna Hello. I’m experimenting with Traditional function calling and it seems to only ...

A

Ana•9/11/25, 10:14 AM

Hi @samjs any updates on this? Sorry to mention you directly, but saw that you created threads for other questions.

T

Tim•9/11/25, 10:51 AM

Hi, are websockets available for the new Nova 3 ASR model?

Workers AI released so many models, which is exciting! Some models support WebSocket, which is aweso

Similar Threads

Similar Threads

Similar Threads