Hola! I can't get the caching to work. Any hints here? The doc page says skipCache:false should

Hola!

I can't get the caching to work. Any hints here?
The doc page says skipCache:false should do it, but I'm seeing slow response times even with it set to 24hours. Does skipCache:false really work?

https://developers.cloudflare.com/ai-gateway/providers/workersai/

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

Cloudflare Docs

Workers AI

Use AI Gateway for analytics, caching, and security on requests to Workers AI. Workers AI integrates seamlessly with AI Gateway, allowing you to execute AI inference via API requests or through an environment binding for Workers scripts. The binding simplifies the process by routing requests through your AI Gateway with minimal setup.

AAdam Hola! I can't get the caching to work. Any hints here? The doc page says ski...

AdamOP•6/17/25, 4:37 PM

Ok, I get it. The caching only works for gateways. ChatGPT explained it to me.

Some kind of warning or error would have been nice though.

Kate6•6/17/25, 10:26 PM

Is there an example somewhere showing the format for the tools array that needs to be passed into runWithTools when using the embedded function calling? I'm trying to call something that doesn't provide an "openAPI" descriptor...

Kate6•6/17/25, 10:31 PM

Keep getting "✘ [ERROR] ERROR: Tool undefined not found, maybe AI hallucinated" in my Wrangler output

xingguif1111•6/18/25, 3:58 PM

Can cloudflare ai provide some advanced models instead of some outdated models

Xxingguif1111 Can cloudflare ai provide some advanced models instead of some outdated models

deleted_user_29bc0398aM3•6/20/25, 1:50 AM

agree

fandy_1•6/20/25, 5:49 AM

Hi team, I'm using traditional function call method, but it seems doesn't work for me . No function/tool could be picked.
Sample :
Request payload:
{"model":"@hf/nousresearch/hermes-2-pro-mistral-7b","messages":[{"role":"user","content":"what is the weather in london"}],"stream":false,"tools":[{"name":"getWeather","description":"Return the weather for a latitude and longitude","parameters":{"type":"object","properties":{"latitude":{"type":"string","description":"The latitude for the given location"},"longitude":{"type":"string","description":"The longitude for the given location"}},"required":["latitude","longitude"]}}]}
Response payload:
{"id":"id-1750398103502","object":"chat.completion","created":1750398103,"model":"@hf/nousresearch/hermes-2-pro-mistral-7b","choices":[{"index":0,"message":{"role":"assistant","content":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

zido•6/21/25, 3:22 AM

Can we add allenai/Molmo-7B-D-0924

Ffandy_1 Hi team, I'm using traditional function call method, but it seems doesn't work f...

Kate6•6/23/25, 3:08 PM

I'm having the same problem as you. And I've tried both the "traditional function calling" and the embedded. Neither one seems to work.

xCobraax•6/24/25, 9:55 AM

has anyone tried the tts model on spanish? i cant get it to work on spanish

cipherfunk•6/24/25, 6:08 PM

According to the livestream today, containers are now available but do not support GPUs and custom ai models. Can we please get a video generation model added to Workers AI? Would it help to request a specific open weights model here?

acusti•6/27/25, 4:06 AM

quick follow up on BFL’s Kontext model: they’ve now released the
open weights (“FLUX.1 Kontext [dev]”), which seems like a good candidate for workers AI: https://bfl.ai/models/flux-kontext

Black Forest Labs - Frontier AI Lab

Amazing AI models from the Black Forest.

Aacusti quick follow up on BFL’s Kontext model: they’ve now released the open weights (“...

Isaac McFadyen•6/27/25, 1:24 PM

Think the main blocker is going to be the fact that the non-commercial license is quite strict: https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md

Isaac McFadyen•6/27/25, 1:24 PM

Non-Commercial Use Only. You may only access, use, Distribute, or create Derivatives of the FLUX.1 [dev] Model or Derivatives for Non-Commercial Purposes. If you want to use a FLUX.1 [dev] Model or a Derivative for any purpose that is not expressly authorized under this License, such as for a commercial activity, you must request a license from Company, which Company may grant to you in Company’s sole discretion and which additional use may be subject to a fee, royalty or other revenue share. Please see www.bfl.ai if you would like a commercial license.

zido•6/27/25, 3:34 PM

Can we add allenai/Molmo-7B-D-0924. It has 150k monthly downloads and Apache 2 license.

IIsaac McFadyen Think the main blocker is going to be the fact that the non-commercial license i...

acusti•6/27/25, 8:22 PM

yeah that does look tricky. i wonder how hard it would be for cloudflare to have some kind of checkbox associated with the cloudflare account that we would need to click to confirm that we are using the model for non-commercial purposes, or at least that we understand that the license is only for non-commercial usage, before you could access the model from Workers AI. though i’m also not actually sure that is legally necessary, as long as cloudflare calls out the license and has a statement along the lines of “by using this model, you agree that you will use it only for non-commercial purposes and otherwise abide by BFL’s license” in the docs.

or is the idea that cloudflare itself is using the model for non-commercial purposes by providing it with workers, a product that they can charge for? but if that’s the case, how does a provider like FAL provide paid access to the model?

Aacusti yeah that does look tricky. i wonder how hard it would be for cloudflare to have...

Isaac McFadyen•6/27/25, 8:26 PM

Yeah, the issue is that by providing it to customers, Cloudflare is under the commercial terms.

Isaac McFadyen•6/27/25, 8:26 PM

I expect FAL has reached out to BFL and actually has a (paid) license.

acusti•6/30/25, 5:14 PM

excited to try out the ERNIE 4.5 family of models that were open-sourced today: https://ernie.baidu.com/blog/posts/ernie4.5/

ERNIE Blog

Announcing the Open Source Release of the ERNIE 4.5 Model Family

We introduce ERNIE 4.5, a new family of large-scale multimodal models comprising 10 distinct variants. The model family consist of Mixture-of-Experts (MoE) models with 47B and 3B active parameters, with the largest model having 424B total parameters, as well as a 0.3B dense model.

dan•7/2/25, 4:06 PM

i'm getting Capacity temporarily exceeded, please try again.Capacity temporarily exceeded, please try again. whenever i try to use whisper, it works briefly then pretty quickly gives me errors

blindChicken•7/3/25, 11:47 PM

When using
@cf/meta/llama-3.1-8b-instruct-fp8-fast@cf/meta/llama-3.1-8b-instruct-fp8-fast or @cf/meta/llama-3.2-3b-instruct@cf/meta/llama-3.2-3b-instruct
I'm getting the following error:
message: "AiError: 5025: This model doesn't suuport JSON Schema."message: "AiError: 5025: This model doesn't suuport JSON Schema."
JSON_Schema is shown in the CF's API docs for both models. So what's the issue, are the docs wrong? Seems to be possible, there is a typo in the error message

. If the docs are wrong how can I determine which model support JSON Schema? Otherwise, what could be the problem?

blindChicken•7/4/25, 12:03 AM

Okay! Found the fix, type must json_objectjson_object and not json_schemajson_schema like so:

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

So error in the docs?

blindChicken•7/4/25, 12:10 AM

While I'm ranting about the docs, it would be nice if the json format of the API Schema would be collapsible, as it is very long and hard to follow indentation down the page. I copy it to VS-code when I need it, but it would be great if would be included on the page as UI element.

Sinisha Djukic•7/4/25, 6:41 AM

I have an issue with trying out MeloTTS - it doesn't seem to be in the AiModels list as a constant, even though I'm using @Cloudflare/ai version 1.2.2, which seems to be the latest. Anyone has this issue?

kwang•7/4/25, 7:04 AM

think the authorization method is no longer supported. would be good to update the docs

scotto•7/4/25, 12:58 PM

can we ask for some of the great qwen3 models? for example https://huggingface.co/Qwen/Qwen3-Reranker-4B

Qwen/Qwen3-Reranker-4B · Hugging Face

scotto•7/6/25, 3:10 PM

How to obtain sparse embeddings separated in M3?

Cedrick•7/7/25, 1:21 PM

Hello everyone, does anyone here already called LLAMA4 (@cf/meta/llama-4-scout-17b-16e-instruct@cf/meta/llama-4-scout-17b-16e-instruct) with some tools/ function calling. With or without the compatibility mode (https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) i got answer with no tool results, and no Ai message either. I am using an official Java Client library... would like to know if i need to keep pushing (tried with strict tools) and is it expected ?

Cloudflare Docs

OpenAI compatible API endpoints

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions) and text embedding models (/v1/embeddings). This allows you to use the same code as you would for your OpenAI commands, but swap in Workers AI easily.

BblindChicken When using `@cf/meta/llama-3.1-8b-instruct-fp8-fast` or `@cf/meta/llama-3.2-3b-...

Ꭱ

ᎡᴜᴛʜʟᴇssŦᴀᴄᴛɪᴏɴ•7/7/25, 3:42 PM

The docs are generated wrong. I reported and docs team said it was out of their hands. You can see the list of supported models at the bottom of this page however: https://developers.cloudflare.com/workers-ai/features/json-mode/

Interesting that you could coax a json object with json_objectjson_object however! I'll have to check that out

Cloudflare Docs

JSON Mode

When we want text-generation AI models to interact with databases, services, and external systems programmatically, typically when using tool calling or building AI agents, we must have structured response formats rather than natural language.

Ꭱ

ᎡᴜᴛʜʟᴇssŦᴀᴄᴛɪᴏɴ•7/7/25, 4:20 PM

On a unrealted note:

Anyone have insight as to why I consistently am getting the same output for text-generation for the same prompt? I explicitly have tried setting { temperature: 5, top_p: 1, seed: Math.floor(Math.random() * 9999999999) + 1 }{ temperature: 5, top_p: 1, seed: Math.floor(Math.random() * 9999999999) + 1 } to attempt to encourage different results every run... but no luck

Marco Aleixo•7/10/25, 2:27 PM

BGE-M3 batch request failed after all retry attempts
    details: {
    "attempts": 4,
    "error": "10000: Authentication error",
    "batchSize": 1
  }

BGE-M3 batch request failed after all retry attempts
    details: {
    "attempts": 4,
    "error": "10000: Authentication error",
    "batchSize": 1
  }

this is getting very common locally, not only on my environment but my colleagues as well. are we miss using something?

GoldenCrow•7/10/25, 4:01 PM

hey guys,
new here, trying to workout a simple thing but i'm blocked.
Is there a way to have a python worker call workers ai env.ai.run()?

cqm21•7/13/25, 1:28 PM

hey guys,

When trying to use Workers AI I'm getting this error: 4006: you have used up your daily free allocation of 10,000 neurons, please upgrade to Cloudflare's Workers Paid plan if you would like to continue usage.

In dashboard it's currently showing 0/10k

"Daily usage (resets at 00:00 UTC)-Neurons used today: 0/10k"

Have anyone experienced something similar?

yinxingmaiming6409•7/13/25, 3:24 PM

Can we use QWEN? Why is there no default model? autorag

Ggabrielstuff opened an isue : https://community.cloudflare.com/t/gemma-aierror-3040-capacity-...

brylan•7/14/25, 3:09 AM

did you fix this?

brylan•7/14/25, 3:11 AM

Cloudflare AI API error: {"errors":[{"message":"AiError: AiError: could not route request to AI model (1b738130-21eb-47d2-a741-41984e3e71e1)","code":4002}],"success":false,"result":{},"messages":[]}

Cloudflare AI API error: {"errors":[{"message":"AiError: AiError: could not route request to AI model (1b738130-21eb-47d2-a741-41984e3e71e1)","code":4002}],"success":false,"result":{},"messages":[]}

brylan•7/14/25, 3:11 AM

brylan•7/14/25, 3:14 AM

brylan•7/14/25, 3:14 AM

I dont understand why this is having issues? I used it a while ago and it worked fine. Now its full of errors and bugs

brylan•7/14/25, 3:19 AM

errorText: '{"errors":[{"message":"AiError: AiError: Capacity temporarily exceeded, please try again. (5f37591a-d0ff-4304-8625-6d0fa8035c96)","code":3040}],"success":false,"result":{},"messages":[]}',

??

brylan•7/14/25, 3:20 AM

Im well within my nueron count

brylan•7/14/25, 3:21 AM

brylan•7/14/25, 3:23 AM

If I go into workers paid will this issue resolve itself?

Bbrylan errorText: '{"errors":[{"message":"AiError: AiError: Capacity temporarily exceed...

Chaika•7/14/25, 2:12 PM

Nothing to do with Neuron count nor plan level. Sometimes capacity for these models just waxes and wanes, as far as I can see, as users scale up and the infra does as well. I saw a short(ish) period of this yesterday but seems fine now