Thanks sorry I thought it was

I

I believe BFL has announced that they plan to make it open-weight (or, well, a lighter version of it the same as they've done with Flux Dev/Schnell vs Pro) but haven't actually done that yet.

IIsaac McFadyen ?workers-ai-models

G

gabrielstuff•6/7/25, 11:14 AM

Yes

A

Advany•6/7/25, 11:31 AM

For agents sdk, using the starter kit, when doing npm update. It started to fail. I had to add

import { defineWorkersConfig } from "@cloudflare/vitest-pool-workers/config";

export default defineWorkersConfig({
environments: {
ssr: {
keepProcessEnv: true,
},
},
test: {
poolOptions: {
workers: {
wrangler: { configPath: "./wrangler.jsonc" },
},
},
},
resolve: {
alias: {
ajv: 'ajv/dist/ajv.min.js',
},
},
optimizeDeps: {
include: ['ajv'],
},
});

To vitest to make it work again!

AAdvany For agents sdk, using the starter kit, when doing npm update. It started to fail...

A

Advany•6/7/25, 11:32 AM

so the way to replicate it is create agents sdk using the starter, then npm update. It will show something like

 FAIL  tests/index.test.ts [ tests/index.test.ts ]
SyntaxError: Unexpected token ':'
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/definition_schema.js?mf_vitest_no_cjs_esm_shim:3:18
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/keyword.js?mf_vitest_no_cjs_esm_shim:5:24
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/ajv.js?mf_vitest_no_cjs_esm_shim:29:21

 FAIL  tests/index.test.ts [ tests/index.test.ts ]
SyntaxError: Unexpected token ':'
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/definition_schema.js?mf_vitest_no_cjs_esm_shim:3:18
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/keyword.js?mf_vitest_no_cjs_esm_shim:5:24
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/ajv.js?mf_vitest_no_cjs_esm_shim:29:21

 FAIL  tests/index.test.ts [ tests/index.test.ts ]
SyntaxError: Unexpected token ':'
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/definition_schema.js?mf_vitest_no_cjs_esm_shim:3:18
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/keyword.js?mf_vitest_no_cjs_esm_shim:5:24
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/ajv.js?mf_vitest_no_cjs_esm_shim:29:21

 FAIL  tests/index.test.ts [ tests/index.test.ts ]
SyntaxError: Unexpected token ':'
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/definition_schema.js?mf_vitest_no_cjs_esm_shim:3:18
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/keyword.js?mf_vitest_no_cjs_esm_shim:5:24
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/ajv.js?mf_vitest_no_cjs_esm_shim:29:21

I fixed it but adding

 resolve: {
    alias: {
      ajv: 'ajv/dist/ajv.min.js',
    },
  },
  optimizeDeps: {
    include: ['ajv'],
  },

 resolve: {
    alias: {
      ajv: 'ajv/dist/ajv.min.js',
    },
  },
  optimizeDeps: {
    include: ['ajv'],
  },

 resolve: {
    alias: {
      ajv: 'ajv/dist/ajv.min.js',
    },
  },
  optimizeDeps: {
    include: ['ajv'],
  },

 resolve: {
    alias: {
      ajv: 'ajv/dist/ajv.min.js',
    },
  },
  optimizeDeps: {
    include: ['ajv'],
  },

LLiam HUGE loss of about 99% of AI requests failing currently...[Last 3 hours]

G

gabrielstuff•6/7/25, 1:58 PM

I can’t manage to run ai workers properly on my side

Ggabrielstuff Yes

I

Isaac McFadyen•6/7/25, 3:02 PM

See the bot message right below mine.

S

Stupefystar•6/8/25, 12:04 PM

CURL to

/ai/run/@cf/meta/llama-3.2-11b-vision-instruct

/ai/run/@cf/meta/llama-3.2-11b-vision-instruct

with LORA >

"errors": [{
        "message": "AiError: AiError: Lora not compatible with model. (c8afa8e1-48d6-42d4-871d-b04bce9b4c67)",
        "code": 3030
    }]

"errors": [{
        "message": "AiError: AiError: Lora not compatible with model. (c8afa8e1-48d6-42d4-871d-b04bce9b4c67)",
        "code": 3030
    }]

"errors": [{
        "message": "AiError: AiError: Lora not compatible with model. (c8afa8e1-48d6-42d4-871d-b04bce9b4c67)",
        "code": 3030
    }]

"errors": [{
        "message": "AiError: AiError: Lora not compatible with model. (c8afa8e1-48d6-42d4-871d-b04bce9b4c67)",
        "code": 3030
    }]

S

Stupefystar•6/8/25, 12:04 PM

S

Stupefystar•6/8/25, 12:05 PM

Creating new finetune "my-finetune-name" for model "@cf/meta/llama-3.2-11b-vision-instruct".
...

Assets uploaded, finetune "my-finetune-name" is ready to use.

IIsaac McFadyen See the bot message right below mine.

G

gabrielstuff•6/9/25, 7:20 AM

Thanks Isaac, per my question, I was asking about a roadmap. Is that something planned, considered or not. The bot successfully answers on the fact that it is not supported.

Thank you !

1

1984 Ford Laser•6/10/25, 10:52 PM

Wondering if there's plans to allow custom/uploaded models in future? Would love some more image detection and action recognition models for some video pipelines Im working on

K

Kraus•6/11/25, 3:53 PM

I keep getting network connection lost errors every few worker ai calls. What would be the reason for this?

S

s•6/12/25, 5:09 AM

Has LORAs support been updated for the models in this announcement? https://blog.cloudflare.com/workers-ai-improvements/

The docs are a bit confusing on this: Llama 3.1 and Mistral Small 3.1 are listed as supported, but the model finder and API docs don't reflect LORA support, and the blog post lists them as soon

The Cloudflare Blog

Workers AI gets a speed boost, batch workload support, more LoRAs, ...

We just made Workers AI inference faster with speculative decoding & prefix caching. Use our new batch inference for handling large request volumes seamlessly. Build tailored AI apps with more LoRA options. Lastly, new models and a refreshed dashboard round out this Developer Week update for Workers AI.

B

bomboclat•6/12/25, 8:10 AM

Yes, it sort of contradicts https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/ also wondering what soon means
edit: PR submitted

Cloudflare Docs

Using LoRA adapters

Upload and use LoRA adapters to get fine-tuned inference on Workers AI.

C

cipherfunk•6/14/25, 1:05 AM

When will containers be released? Still June? Will that allow us to run custom ai models? Specifically video gen?

B

bomboclat•6/17/25, 5:19 AM

I'm also looking out for it too, the press release mentioned end of this month

A

Adam•6/17/25, 2:58 PM

Hola!

I can't get the caching to work. Any hints here?
The doc page says skipCache:false should do it, but I'm seeing slow response times even with it set to 24hours. Does skipCache:false really work?

https://developers.cloudflare.com/ai-gateway/providers/workersai/

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

Cloudflare Docs

Workers AI

Use AI Gateway for analytics, caching, and security on requests to Workers AI. Workers AI integrates seamlessly with AI Gateway, allowing you to execute AI inference via API requests or through an environment binding for Workers scripts. The binding simplifies the process by routing requests through your AI Gateway with minimal setup.

AAdam Hola! I can't get the caching to work. Any hints here? The doc page says ski...

A

Adam•6/17/25, 4:37 PM

Ok, I get it. The caching only works for gateways. ChatGPT explained it to me.

Some kind of warning or error would have been nice though.

K

Kate6•6/17/25, 10:26 PM

Is there an example somewhere showing the format for the tools array that needs to be passed into runWithTools when using the embedded function calling? I'm trying to call something that doesn't provide an "openAPI" descriptor...

K

Kate6•6/17/25, 10:31 PM

Keep getting "✘ [ERROR] ERROR: Tool undefined not found, maybe AI hallucinated" in my Wrangler output

X

xingguif1111•6/18/25, 3:58 PM

Can cloudflare ai provide some advanced models instead of some outdated models

Xxingguif1111 Can cloudflare ai provide some advanced models instead of some outdated models

D

deleted_user_29bc0398aM3•6/20/25, 1:50 AM

agree

F

fandy_1•6/20/25, 5:49 AM

Hi team, I'm using traditional function call method, but it seems doesn't work for me . No function/tool could be picked.
Sample :
Request payload:
{"model":"@hf/nousresearch/hermes-2-pro-mistral-7b","messages":[{"role":"user","content":"what is the weather in london"}],"stream":false,"tools":[{"name":"getWeather","description":"Return the weather for a latitude and longitude","parameters":{"type":"object","properties":{"latitude":{"type":"string","description":"The latitude for the given location"},"longitude":{"type":"string","description":"The longitude for the given location"}},"required":["latitude","longitude"]}}]}
Response payload:
{"id":"id-1750398103502","object":"chat.completion","created":1750398103,"model":"@hf/nousresearch/hermes-2-pro-mistral-7b","choices":[{"index":0,"message":{"role":"assistant","content":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

Z

zido•6/21/25, 3:22 AM

Can we add allenai/Molmo-7B-D-0924

Ffandy_1 Hi team, I'm using traditional function call method, but it seems doesn't work f...

K

Kate6•6/23/25, 3:08 PM

I'm having the same problem as you. And I've tried both the "traditional function calling" and the embedded. Neither one seems to work.

X

xCobraax•6/24/25, 9:55 AM

has anyone tried the tts model on spanish? i cant get it to work on spanish

C

cipherfunk•6/24/25, 6:08 PM

According to the livestream today, containers are now available but do not support GPUs and custom ai models. Can we please get a video generation model added to Workers AI? Would it help to request a specific open weights model here?

A

acusti•6/27/25, 4:06 AM

quick follow up on BFL’s Kontext model: they’ve now released the
open weights (“FLUX.1 Kontext [dev]”), which seems like a good candidate for workers AI: https://bfl.ai/models/flux-kontext

Black Forest Labs - Frontier AI Lab

Amazing AI models from the Black Forest.

Aacusti quick follow up on BFL’s Kontext model: they’ve now released the open weights (“...

I

Isaac McFadyen•6/27/25, 1:24 PM

Think the main blocker is going to be the fact that the non-commercial license is quite strict: https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md

I

Isaac McFadyen•6/27/25, 1:24 PM

Non-Commercial Use Only. You may only access, use, Distribute, or create Derivatives of the FLUX.1 [dev] Model or Derivatives for Non-Commercial Purposes. If you want to use a FLUX.1 [dev] Model or a Derivative for any purpose that is not expressly authorized under this License, such as for a commercial activity, you must request a license from Company, which Company may grant to you in Company’s sole discretion and which additional use may be subject to a fee, royalty or other revenue share. Please see www.bfl.ai if you would like a commercial license.

Z

zido•6/27/25, 3:34 PM

Can we add allenai/Molmo-7B-D-0924. It has 150k monthly downloads and Apache 2 license.

IIsaac McFadyen Think the main blocker is going to be the fact that the non-commercial license i...

A

acusti•6/27/25, 8:22 PM

yeah that does look tricky. i wonder how hard it would be for cloudflare to have some kind of checkbox associated with the cloudflare account that we would need to click to confirm that we are using the model for non-commercial purposes, or at least that we understand that the license is only for non-commercial usage, before you could access the model from Workers AI. though i’m also not actually sure that is legally necessary, as long as cloudflare calls out the license and has a statement along the lines of “by using this model, you agree that you will use it only for non-commercial purposes and otherwise abide by BFL’s license” in the docs.

or is the idea that cloudflare itself is using the model for non-commercial purposes by providing it with workers, a product that they can charge for? but if that’s the case, how does a provider like FAL provide paid access to the model?

Aacusti yeah that does look tricky. i wonder how hard it would be for cloudflare to have...

I

Isaac McFadyen•6/27/25, 8:26 PM

Yeah, the issue is that by providing it to customers, Cloudflare is under the commercial terms.

I

Isaac McFadyen•6/27/25, 8:26 PM

I expect FAL has reached out to BFL and actually has a (paid) license.

A

acusti•6/30/25, 5:14 PM

excited to try out the ERNIE 4.5 family of models that were open-sourced today: https://ernie.baidu.com/blog/posts/ernie4.5/

ERNIE Blog

Announcing the Open Source Release of the ERNIE 4.5 Model Family

We introduce ERNIE 4.5, a new family of large-scale multimodal models comprising 10 distinct variants. The model family consist of Mixture-of-Experts (MoE) models with 47B and 3B active parameters, with the largest model having 424B total parameters, as well as a 0.3B dense model.

D

dan•7/2/25, 4:06 PM

i'm getting

Capacity temporarily exceeded, please try again.

Capacity temporarily exceeded, please try again.

whenever i try to use whisper, it works briefly then pretty quickly gives me errors

B

blindChicken•7/3/25, 11:47 PM

When using

@cf/meta/llama-3.1-8b-instruct-fp8-fast

@cf/meta/llama-3.1-8b-instruct-fp8-fast

or

@cf/meta/llama-3.2-3b-instruct

@cf/meta/llama-3.2-3b-instruct

I'm getting the following error:

message: "AiError: 5025: This model doesn't suuport JSON Schema."

message: "AiError: 5025: This model doesn't suuport JSON Schema."

JSON_Schema is shown in the CF's API docs for both models. So what's the issue, are the docs wrong? Seems to be possible, there is a typo in the error message

. If the docs are wrong how can I determine which model support JSON Schema? Otherwise, what could be the problem?

B

blindChicken•7/4/25, 12:03 AM

Okay! Found the fix, type must

json_object

json_object

and not

json_schema

json_schema

like so:

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

So error in the docs?

B

blindChicken•7/4/25, 12:10 AM

While I'm ranting about the docs, it would be nice if the json format of the API Schema would be collapsible, as it is very long and hard to follow indentation down the page. I copy it to VS-code when I need it, but it would be great if would be included on the page as UI element.

S

Sinisha Djukic•7/4/25, 6:41 AM

I have an issue with trying out MeloTTS - it doesn't seem to be in the AiModels list as a constant, even though I'm using @Cloudflare/ai version 1.2.2, which seems to be the latest. Anyone has this issue?

K

kwang•7/4/25, 7:04 AM

think the authorization method is no longer supported. would be good to update the docs

S

scotto•7/4/25, 12:58 PM

can we ask for some of the great qwen3 models? for example https://huggingface.co/Qwen/Qwen3-Reranker-4B

Qwen/Qwen3-Reranker-4B · Hugging Face

S

scotto•7/6/25, 3:10 PM

How to obtain sparse embeddings separated in M3?

C

Cedrick•7/7/25, 1:21 PM

Hello everyone, does anyone here already called LLAMA4 (

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct) with some tools/ function calling. With or without the compatibility mode (https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/) i got answer with no tool results, and no Ai message either. I am using an official Java Client library... would like to know if i need to keep pushing (tried with strict tools) and is it expected ?

Cloudflare Docs

OpenAI compatible API endpoints

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions) and text embedding models (/v1/embeddings). This allows you to use the same code as you would for your OpenAI commands, but swap in Workers AI easily.

BblindChicken When using `@cf/meta/llama-3.1-8b-instruct-fp8-fast` or `@cf/meta/llama-3.2-3b-...

Ꭱ

ᎡᴜᴛʜʟᴇssŦᴀᴄᴛɪᴏɴ•7/7/25, 3:42 PM

The docs are generated wrong. I reported and docs team said it was out of their hands. You can see the list of supported models at the bottom of this page however: https://developers.cloudflare.com/workers-ai/features/json-mode/

Interesting that you could coax a json object with

json_object

json_object

however! I'll have to check that out

Cloudflare Docs

JSON Mode

When we want text-generation AI models to interact with databases, services, and external systems programmatically, typically when using tool calling or building AI agents, we must have structured response formats rather than natural language.

Ꭱ

ᎡᴜᴛʜʟᴇssŦᴀᴄᴛɪᴏɴ•7/7/25, 4:20 PM

On a unrealted note:

Anyone have insight as to why I consistently am getting the same output for text-generation for the same prompt? I explicitly have tried setting

{ temperature: 5, top_p: 1, seed: Math.floor(Math.random() * 9999999999) + 1 }

{ temperature: 5, top_p: 1, seed: Math.floor(Math.random() * 9999999999) + 1 }

{ temperature: 5, top_p: 1, seed: Math.floor(Math.random() * 9999999999) + 1 }

{ temperature: 5, top_p: 1, seed: Math.floor(Math.random() * 9999999999) + 1 } to attempt to encourage different results every run... but no luck

M

Marco Aleixo•7/10/25, 2:27 PM

BGE-M3 batch request failed after all retry attempts
    details: {
    "attempts": 4,
    "error": "10000: Authentication error",
    "batchSize": 1
  }

BGE-M3 batch request failed after all retry attempts
    details: {
    "attempts": 4,
    "error": "10000: Authentication error",
    "batchSize": 1
  }

BGE-M3 batch request failed after all retry attempts
    details: {
    "attempts": 4,
    "error": "10000: Authentication error",
    "batchSize": 1
  }

BGE-M3 batch request failed after all retry attempts
    details: {
    "attempts": 4,
    "error": "10000: Authentication error",
    "batchSize": 1
  }

this is getting very common locally, not only on my environment but my colleagues as well. are we miss using something?

Similar Threads