Hello, is there any roadmap for custom model support ? Like on the fly model download from hugging f

G

gabrielstuffOP•6/5/25, 6:19 PM

The only useful info I’ve come across so far is from here:
https://community.cloudflare.com/t/newer-ai-models-bring-your-own-model/597274
https://github.com/cloudflare/cloudflare-docs/issues/11039
https://developers.cloudflare.com/changelog/

At the moment, “bring your own model” is something very few platforms do well. Among the better ones, I’d say Scaleway Managed Inference and Deepinfra are leading the pack.

Being able to plug in your own model, especially for tasks like completion or classification, is a huge deal. It opens the door to running smaller, highly optimized models, including those trained for other languages or specific domains. And for fine-tuned models, this could really set Cloudflare apart if done right !

GitHub

Ai model deployment, can we deploy our models to cloudflare? · Iss...

Which Cloudflare product(s) does this pertain to? Workers Subject Matter AI workers Content Location https://github.com/cloudflare/cloudflare-docs/blob/production/content/workers-ai/models/embeddin...

Cloudflare Docs

Changelogs

Ggabrielstuff Hello, is there any roadmap for custom model support ? Like on the fly model dow...

I

Isaac McFadyen•6/5/25, 7:55 PM

?workers-ai-models

S

SuperHelpflare•6/5/25, 7:55 PM

Workers AI currently only supports popular open-source models provided by the Cloudflare team, as well as your own LoRAs that can be applied on top of the Cloudflare-provided models. You cannot currently upload your own models or use a model from HuggingFace. See the documentation for the list of Cloudflare-provided models: https://developers.cloudflare.com/workers-ai/models/

S

SuperHelpflare•6/6/25, 8:02 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

Z

Zig•6/6/25, 6:16 PM

any plans for adding flux kontext?

ZZig any plans for adding flux kontext?

I

Isaac McFadyen•6/6/25, 7:08 PM

At the moment Flux Kontext is not an open-weight model, so I expect they wouldn't add it until if/when it becomes open-weight the same as Flux is.

Z

Zig•6/6/25, 7:09 PM

Thanks sorry I thought it was

ZZig Thanks sorry I thought it was

I

Isaac McFadyen•6/6/25, 7:10 PM

I believe BFL has announced that they plan to make it open-weight (or, well, a lighter version of it the same as they've done with Flux Dev/Schnell vs Pro) but haven't actually done that yet.

IIsaac McFadyen ?workers-ai-models

G

gabrielstuffOP•6/7/25, 11:14 AM

Yes

A

Advany•6/7/25, 11:31 AM

For agents sdk, using the starter kit, when doing npm update. It started to fail. I had to add

import { defineWorkersConfig } from "@cloudflare/vitest-pool-workers/config";

export default defineWorkersConfig({
environments: {
ssr: {
keepProcessEnv: true,
},
},
test: {
poolOptions: {
workers: {
wrangler: { configPath: "./wrangler.jsonc" },
},
},
},
resolve: {
alias: {
ajv: 'ajv/dist/ajv.min.js',
},
},
optimizeDeps: {
include: ['ajv'],
},
});

To vitest to make it work again!

AAdvany For agents sdk, using the starter kit, when doing npm update. It started to fail...

A

Advany•6/7/25, 11:32 AM

so the way to replicate it is create agents sdk using the starter, then npm update. It will show something like

 FAIL  tests/index.test.ts [ tests/index.test.ts ]
SyntaxError: Unexpected token ':'
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/definition_schema.js?mf_vitest_no_cjs_esm_shim:3:18
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/keyword.js?mf_vitest_no_cjs_esm_shim:5:24
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/ajv.js?mf_vitest_no_cjs_esm_shim:29:21

 FAIL  tests/index.test.ts [ tests/index.test.ts ]
SyntaxError: Unexpected token ':'
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/definition_schema.js?mf_vitest_no_cjs_esm_shim:3:18
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/keyword.js?mf_vitest_no_cjs_esm_shim:5:24
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/ajv.js?mf_vitest_no_cjs_esm_shim:29:21

 FAIL  tests/index.test.ts [ tests/index.test.ts ]
SyntaxError: Unexpected token ':'
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/definition_schema.js?mf_vitest_no_cjs_esm_shim:3:18
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/keyword.js?mf_vitest_no_cjs_esm_shim:5:24
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/ajv.js?mf_vitest_no_cjs_esm_shim:29:21

 FAIL  tests/index.test.ts [ tests/index.test.ts ]
SyntaxError: Unexpected token ':'
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/definition_schema.js?mf_vitest_no_cjs_esm_shim:3:18
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/keyword.js?mf_vitest_no_cjs_esm_shim:5:24
 ❯ Users/advany/Documents/GitHub/generator-agent/node_modules/ajv/lib/ajv.js?mf_vitest_no_cjs_esm_shim:29:21

I fixed it but adding

 resolve: {
    alias: {
      ajv: 'ajv/dist/ajv.min.js',
    },
  },
  optimizeDeps: {
    include: ['ajv'],
  },

 resolve: {
    alias: {
      ajv: 'ajv/dist/ajv.min.js',
    },
  },
  optimizeDeps: {
    include: ['ajv'],
  },

 resolve: {
    alias: {
      ajv: 'ajv/dist/ajv.min.js',
    },
  },
  optimizeDeps: {
    include: ['ajv'],
  },

 resolve: {
    alias: {
      ajv: 'ajv/dist/ajv.min.js',
    },
  },
  optimizeDeps: {
    include: ['ajv'],
  },

LLiam HUGE loss of about 99% of AI requests failing currently...[Last 3 hours]

G

gabrielstuffOP•6/7/25, 1:58 PM

I can’t manage to run ai workers properly on my side

Ggabrielstuff Yes

I

Isaac McFadyen•6/7/25, 3:02 PM

See the bot message right below mine.

S

Stupefystar•6/8/25, 12:04 PM

CURL to

/ai/run/@cf/meta/llama-3.2-11b-vision-instruct

/ai/run/@cf/meta/llama-3.2-11b-vision-instruct

/ai/run/@cf/meta/llama-3.2-11b-vision-instruct

/ai/run/@cf/meta/llama-3.2-11b-vision-instruct with LORA >

"errors": [{
        "message": "AiError: AiError: Lora not compatible with model. (c8afa8e1-48d6-42d4-871d-b04bce9b4c67)",
        "code": 3030
    }]

"errors": [{
        "message": "AiError: AiError: Lora not compatible with model. (c8afa8e1-48d6-42d4-871d-b04bce9b4c67)",
        "code": 3030
    }]

"errors": [{
        "message": "AiError: AiError: Lora not compatible with model. (c8afa8e1-48d6-42d4-871d-b04bce9b4c67)",
        "code": 3030
    }]

"errors": [{
        "message": "AiError: AiError: Lora not compatible with model. (c8afa8e1-48d6-42d4-871d-b04bce9b4c67)",
        "code": 3030
    }]

S

Stupefystar•6/8/25, 12:04 PM

S

Stupefystar•6/8/25, 12:05 PM

Creating new finetune "my-finetune-name" for model "@cf/meta/llama-3.2-11b-vision-instruct".
...

Assets uploaded, finetune "my-finetune-name" is ready to use.

IIsaac McFadyen See the bot message right below mine.

G

gabrielstuffOP•6/9/25, 7:20 AM

Thanks Isaac, per my question, I was asking about a roadmap. Is that something planned, considered or not. The bot successfully answers on the fact that it is not supported.

Thank you !

1

1984 Ford Laser•6/10/25, 10:52 PM

Wondering if there's plans to allow custom/uploaded models in future? Would love some more image detection and action recognition models for some video pipelines Im working on

K

Kraus•6/11/25, 3:53 PM

I keep getting network connection lost errors every few worker ai calls. What would be the reason for this?

S

s•6/12/25, 5:09 AM

Has LORAs support been updated for the models in this announcement? https://blog.cloudflare.com/workers-ai-improvements/

The docs are a bit confusing on this: Llama 3.1 and Mistral Small 3.1 are listed as supported, but the model finder and API docs don't reflect LORA support, and the blog post lists them as soon

The Cloudflare Blog

Workers AI gets a speed boost, batch workload support, more LoRAs, ...

We just made Workers AI inference faster with speculative decoding & prefix caching. Use our new batch inference for handling large request volumes seamlessly. Build tailored AI apps with more LoRA options. Lastly, new models and a refreshed dashboard round out this Developer Week update for Workers AI.

B

bomboclat•6/12/25, 8:10 AM

Yes, it sort of contradicts https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/ also wondering what soon means
edit: PR submitted

Cloudflare Docs

Using LoRA adapters

Upload and use LoRA adapters to get fine-tuned inference on Workers AI.

C

cipherfunk•6/14/25, 1:05 AM

When will containers be released? Still June? Will that allow us to run custom ai models? Specifically video gen?

B

bomboclat•6/17/25, 5:19 AM

I'm also looking out for it too, the press release mentioned end of this month

A

Adam•6/17/25, 2:58 PM

Hola!

I can't get the caching to work. Any hints here?
The doc page says skipCache:false should do it, but I'm seeing slow response times even with it set to 24hours. Does skipCache:false really work?

https://developers.cloudflare.com/ai-gateway/providers/workersai/

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

const aiResponse = await env.AI.run(
  '@cf/meta/llama-3.1-8b-instruct',
  {
    messages: [
      {
        role: 'system',
        content: 'You are a helpful assistant that transforms text into different tones and styles.'
      },
      {
        role: 'user',
        content: fullPrompt
      }
    ],
    max_tokens: 2048
  },
  {
    gateway: {
      skipCache: false,   // Enable caching (default behavior)
      cacheTtl: 86400     // Cache for 24 hours (24 * 60 * 60 seconds)
    }
  }
);

Cloudflare Docs

Workers AI

Use AI Gateway for analytics, caching, and security on requests to Workers AI. Workers AI integrates seamlessly with AI Gateway, allowing you to execute AI inference via API requests or through an environment binding for Workers scripts. The binding simplifies the process by routing requests through your AI Gateway with minimal setup.

AAdam Hola! I can't get the caching to work. Any hints here? The doc page says ski...

A

Adam•6/17/25, 4:37 PM

Ok, I get it. The caching only works for gateways. ChatGPT explained it to me.

Some kind of warning or error would have been nice though.

K

Kate6•6/17/25, 10:26 PM

Is there an example somewhere showing the format for the tools array that needs to be passed into runWithTools when using the embedded function calling? I'm trying to call something that doesn't provide an "openAPI" descriptor...

K

Kate6•6/17/25, 10:31 PM

Keep getting "✘ [ERROR] ERROR: Tool undefined not found, maybe AI hallucinated" in my Wrangler output

X

xingguif1111•6/18/25, 3:58 PM

Can cloudflare ai provide some advanced models instead of some outdated models

Xxingguif1111 Can cloudflare ai provide some advanced models instead of some outdated models

D

deleted_user_29bc0398aM3•6/20/25, 1:50 AM

agree

F

fandy_1•6/20/25, 5:49 AM

Hi team, I'm using traditional function call method, but it seems doesn't work for me . No function/tool could be picked.
Sample :
Request payload:
{"model":"@hf/nousresearch/hermes-2-pro-mistral-7b","messages":[{"role":"user","content":"what is the weather in london"}],"stream":false,"tools":[{"name":"getWeather","description":"Return the weather for a latitude and longitude","parameters":{"type":"object","properties":{"latitude":{"type":"string","description":"The latitude for the given location"},"longitude":{"type":"string","description":"The longitude for the given location"}},"required":["latitude","longitude"]}}]}
Response payload:
{"id":"id-1750398103502","object":"chat.completion","created":1750398103,"model":"@hf/nousresearch/hermes-2-pro-mistral-7b","choices":[{"index":0,"message":{"role":"assistant","content":null},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}

Z

zido•6/21/25, 3:22 AM

Can we add allenai/Molmo-7B-D-0924

Ffandy_1 Hi team, I'm using traditional function call method, but it seems doesn't work f...

K

Kate6•6/23/25, 3:08 PM

I'm having the same problem as you. And I've tried both the "traditional function calling" and the embedded. Neither one seems to work.

X

xCobraax•6/24/25, 9:55 AM

has anyone tried the tts model on spanish? i cant get it to work on spanish

C

cipherfunk•6/24/25, 6:08 PM

According to the livestream today, containers are now available but do not support GPUs and custom ai models. Can we please get a video generation model added to Workers AI? Would it help to request a specific open weights model here?

A

acusti•6/27/25, 4:06 AM

quick follow up on BFL’s Kontext model: they’ve now released the
open weights (“FLUX.1 Kontext [dev]”), which seems like a good candidate for workers AI: https://bfl.ai/models/flux-kontext

Black Forest Labs - Frontier AI Lab

Amazing AI models from the Black Forest.

Aacusti quick follow up on BFL’s Kontext model: they’ve now released the open weights (“...

I

Isaac McFadyen•6/27/25, 1:24 PM

Think the main blocker is going to be the fact that the non-commercial license is quite strict: https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md

I

Isaac McFadyen•6/27/25, 1:24 PM

Non-Commercial Use Only. You may only access, use, Distribute, or create Derivatives of the FLUX.1 [dev] Model or Derivatives for Non-Commercial Purposes. If you want to use a FLUX.1 [dev] Model or a Derivative for any purpose that is not expressly authorized under this License, such as for a commercial activity, you must request a license from Company, which Company may grant to you in Company’s sole discretion and which additional use may be subject to a fee, royalty or other revenue share. Please see www.bfl.ai if you would like a commercial license.

Z

zido•6/27/25, 3:34 PM

Can we add allenai/Molmo-7B-D-0924. It has 150k monthly downloads and Apache 2 license.

IIsaac McFadyen Think the main blocker is going to be the fact that the non-commercial license i...

A

acusti•6/27/25, 8:22 PM

yeah that does look tricky. i wonder how hard it would be for cloudflare to have some kind of checkbox associated with the cloudflare account that we would need to click to confirm that we are using the model for non-commercial purposes, or at least that we understand that the license is only for non-commercial usage, before you could access the model from Workers AI. though i’m also not actually sure that is legally necessary, as long as cloudflare calls out the license and has a statement along the lines of “by using this model, you agree that you will use it only for non-commercial purposes and otherwise abide by BFL’s license” in the docs.

or is the idea that cloudflare itself is using the model for non-commercial purposes by providing it with workers, a product that they can charge for? but if that’s the case, how does a provider like FAL provide paid access to the model?

Aacusti yeah that does look tricky. i wonder how hard it would be for cloudflare to have...

I

Isaac McFadyen•6/27/25, 8:26 PM

Yeah, the issue is that by providing it to customers, Cloudflare is under the commercial terms.

I

Isaac McFadyen•6/27/25, 8:26 PM

I expect FAL has reached out to BFL and actually has a (paid) license.

A

acusti•6/30/25, 5:14 PM

excited to try out the ERNIE 4.5 family of models that were open-sourced today: https://ernie.baidu.com/blog/posts/ernie4.5/

ERNIE Blog

Announcing the Open Source Release of the ERNIE 4.5 Model Family

We introduce ERNIE 4.5, a new family of large-scale multimodal models comprising 10 distinct variants. The model family consist of Mixture-of-Experts (MoE) models with 47B and 3B active parameters, with the largest model having 424B total parameters, as well as a 0.3B dense model.

D

dan•7/2/25, 4:06 PM

i'm getting

Capacity temporarily exceeded, please try again.

Capacity temporarily exceeded, please try again.

Capacity temporarily exceeded, please try again.

Capacity temporarily exceeded, please try again. whenever i try to use whisper, it works briefly then pretty quickly gives me errors

B

blindChicken•7/3/25, 11:47 PM

When using

@cf/meta/llama-3.1-8b-instruct-fp8-fast

@cf/meta/llama-3.1-8b-instruct-fp8-fast

@cf/meta/llama-3.1-8b-instruct-fp8-fast

@cf/meta/llama-3.1-8b-instruct-fp8-fast or

@cf/meta/llama-3.2-3b-instruct

@cf/meta/llama-3.2-3b-instruct

@cf/meta/llama-3.2-3b-instruct

@cf/meta/llama-3.2-3b-instruct
I'm getting the following error:

message: "AiError: 5025: This model doesn't suuport JSON Schema."

message: "AiError: 5025: This model doesn't suuport JSON Schema."

message: "AiError: 5025: This model doesn't suuport JSON Schema."

message: "AiError: 5025: This model doesn't suuport JSON Schema."
JSON_Schema is shown in the CF's API docs for both models. So what's the issue, are the docs wrong? Seems to be possible, there is a typo in the error message

. If the docs are wrong how can I determine which model support JSON Schema? Otherwise, what could be the problem?

B

blindChicken•7/4/25, 12:03 AM

Okay! Found the fix, type must

json_object

json_object

json_object

json_object and not

json_schema

json_schema

json_schema

json_schema like so:

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

response_format: {
  type: 'json_object',
  json_schema: responseSchema
}

So error in the docs?

B

blindChicken•7/4/25, 12:10 AM

While I'm ranting about the docs, it would be nice if the json format of the API Schema would be collapsible, as it is very long and hard to follow indentation down the page. I copy it to VS-code when I need it, but it would be great if would be included on the page as UI element.

S

Sinisha Djukic•7/4/25, 6:41 AM

I have an issue with trying out MeloTTS - it doesn't seem to be in the AiModels list as a constant, even though I'm using @Cloudflare/ai version 1.2.2, which seems to be the latest. Anyone has this issue?

Hello, is there any roadmap for custom model support ? Like on the fly model download from hugging f

Similar Threads