We have thousands of users of our service complaining because the users LLM requests are being block

We have thousands of users of our service complaining because the users LLM requests are being blocked when trying to use Anthropic from a supported country using the AI gateway through a CF Worker, but the issue is they are being routed through a HK server (Which is not supported by anthropic), so the users end up getting blocked. What can we do here to fix this critical issue @Walshy | Workers/Pages ?
We are in talks with Anthropic about this too

JjLynx We have thousands of users of our service complaining because the users LLM requ...

Isaac McFadyen•2/25/25, 8:02 PM

Please don't ping employees, Walshy isn't on the AI team so wouldn't be able to resolve this anyway.

IIsaac McFadyen Please don't ping employees, Walshy isn't on the AI team so wouldn't be able to ...

jLynxOP•2/25/25, 8:03 PM

Sorry, its related to workers/pages too

Isaac McFadyen•2/25/25, 8:04 PM

Walshy specifically works on the Worker deployment team so isn't able to help here. I've sent this to someone on the AI Gateway team though.

IIsaac McFadyen Walshy specifically works on the Worker deployment team so isn't able to help he...

jLynxOP•2/25/25, 8:04 PM

Appreciate that @Isaac McFadyen

stochastic•2/26/25, 3:24 AM

Are there any distributed tracing products that work with Workers AI such that I can see spans and waterfalls?

Nimesh•2/26/25, 5:02 AM

Hi, congratulations on the Agents SDK launch. I checked the source code and it looks like it’s based on Durable Objects and one of the examples makes a fetch request to an AI provider, but aren’t Durable Objects charged based on wall time? Makes me wonder if DOs are really suited for these long running AI requests from a pricing point of view. Is this expected to change anytime soon?

rob•2/26/25, 1:11 PM

I love agents

NNimesh Hi, congratulations on the Agents SDK launch. I checked the source code and it l...

Zetax•2/26/25, 2:42 PM

Would love to know this as well

Aster•2/26/25, 3:47 PM

Hey, do you know if there would be MCP support any time soon

AAster Hey, do you know if there would be MCP support any time soon 🙂 ?

DutchDelite (Ivo)•2/26/25, 3:50 PM

+1 on this one

Original message was deleted

Isaac McFadyen•2/26/25, 4:28 PM

In a recent patch, they removed the systemsystem role from llama-guard-3-8bllama-guard-3-8b since it wasn't intended to be allowed but it seems they forgot to update the playground

IIsaac McFadyen Walshy specifically works on the Worker deployment team so isn't able to help he...

jLynxOP•2/26/25, 8:21 PM

Morning Isaac, any update from the team?

JjLynx Morning Isaac, any update from the team?

Isaac McFadyen•2/26/25, 8:31 PM

I haven't heard anything, sorry.

kfcvivo50q•2/27/25, 5:51 AM

hey guys,I noticed that the response from workers-ai/models/deepseek-r1-distill-qwen-32b is incomplete; it only provides a brief answer before being cut off. Is this a normal occurrence, or could there be an issue with my settings?

rufi•2/27/25, 1:24 PM

Hello - I tried the agents-starter (following yesterday's launch / blog post) and both locally and when deployed, following the instructions, the "agent" never respond to inputs. Any cues? Thanks

Kkfcvivo50q hey guys,I noticed that the response from workers-ai/models/deepseek-r1-distill-...

Isaac McFadyen•2/27/25, 1:45 PM

There was a recent change to start enforcing max_tokens in the API. If you want it to respond with more than 256 tokens then you'll need to specify

max_tokens

max_tokens

yourself.

drdiffie•2/27/25, 1:54 PM

Is it possible to use custom Models which are not listed in the "Models"? Would love to see Phi-4 multimodal.

Isaac McFadyen•2/27/25, 2:00 PM

?workers-ai-models

SuperHelpflare•2/27/25, 2:00 PM

Workers AI currently only supports popular open-source models provided by the Cloudflare team, as well as your own LoRAs that can be applied on top of the Cloudflare-provided models. You cannot currently upload your own models or use a model from HuggingFace. See the documentation for the list of Cloudflare-provided models: https://developers.cloudflare.com/workers-ai/models/

Isaac McFadyen•2/27/25, 2:00 PM

^ TLDR no, not at the moment

drdiffie•2/27/25, 2:11 PM

Sad.

IIsaac McFadyen There was a recent change to start enforcing max_tokens in the API. If you want ...

kfcvivo50q•2/27/25, 3:19 PM

thx

Keebs•2/27/25, 8:09 PM

anyone talk yet about the agents?

KKeebs anyone talk yet about the agents?

Isaac McFadyen•2/27/25, 8:21 PM

#agents

Keebs•2/27/25, 8:49 PM

aaaah... I was looking for one lol

Keebs•2/27/25, 8:49 PM

totally read over it somehow

mr.niko.la•2/27/25, 11:03 PM

whisper-large-v3-turbo

What’s the price per hour for this worker ? I don’t see it anywhere

Mmr.niko.la whisper-large-v3-turbo What’s the price per hour for this worker ? I don’t see...

Isaac McFadyen•2/27/25, 11:10 PM

https://developers.cloudflare.com/workers-ai/platform/pricing/

Isaac McFadyen•2/27/25, 11:10 PM

$0.0005 per audio minute

mr.niko.la•2/27/25, 11:11 PM

0.03$/hr not vs 0.04 for groq

IIsaac McFadyen https://developers.cloudflare.com/workers-ai/platform/pricing/

mr.niko.la•2/27/25, 11:13 PM

@cf/openai/whisper I think that’s is for og whisper large

I don’t see pricing for whisper v3 turbo …

Mmr.niko.la @cf/openai/whisper I think that’s is for og whisper large I don’t see pricing ...

Isaac McFadyen•2/27/25, 11:30 PM

That's for Whisper Turbo, one of the Workers AI engineers confirmed a while back (they don't have non-Turbo)

beelzabub•2/27/25, 11:52 PM

Did anyone figure out how to get workers AI to run with vitest locally? Getting the same issue as workers-ai

beelzabub•2/27/25, 11:52 PM

workerd/server/workerd-api.c++:759: error: wrapped binding module can't be resolved (internal modules only); moduleName = miniflare-internal:wrapped:__WRANGLER_EXTERNAL_AI_WORKER
workerd/jsg/util.c++:331: error: e = workerd/server/workerd-api.c++:789: failed: expected !value.IsEmpty(); global did not produce v8::Value

workerd/server/workerd-api.c++:759: error: wrapped binding module can't be resolved (internal modules only); moduleName = miniflare-internal:wrapped:__WRANGLER_EXTERNAL_AI_WORKER
workerd/jsg/util.c++:331: error: e = workerd/server/workerd-api.c++:789: failed: expected !value.IsEmpty(); global did not produce v8::Value

beelzabub•2/27/25, 11:53 PM

Ah, just looks like a straight up bug: https://github.com/cloudflare/workers-sdk/issues/7434

GitHub

🐛 BUG:Could not start vitest when using Cloudflare Vectorize, AI, a...

Which Cloudflare product(s) does this pertain to? Workers AI, Wrangler, Miniflare What version(s) of the tool(s) are you using? 3.91.0 What version of Node are you using? v22.9.0 What operating sys...

IIsaac McFadyen That's for Whisper Turbo, one of the Workers AI engineers confirmed a while back...

mr.niko.la•2/27/25, 11:57 PM

sorry misread it. i will defintately try it.

currently i am sendig audio to backend then sending to groq then to and llm provider.

this looks like a much simpler way to do it and at the very least a redudant and failover option.

right now taking 5-7s but if i am able to send earlier and depending on the edge vs groq it might be very close.

mr.niko.la•2/28/25, 1:49 AM

where is this i dont see it in the workers template no more.

mr.niko.la•2/28/25, 1:55 AM

https://youtu.be/RvJ-T5BrAzc?si=zapE1xdXpM5MJCAD&t=42
this 2min video is so much simpler than cf dev youtube channel.

YouTubeilovefreesoftwareTV

Run OpenAI’s Whisper on Cloudflare for Free using Workers AI [Speec...

#cloudflare #whisper #openai #speechtotext #machinelearning #workersai

In this video, I will show you How to run OpenAI’s Whisper on Cloudflare using Workers AI.

Before we begin, make sure you hit subscribe and tap the bell icon to get daily tech videos.

Workers AI is a new beta feature of Cloudflare that allows you to run popular AI models ...

mr.niko.la•2/28/25, 1:57 AM

althought i do appreicate their cool examples its just too complicated especially when the frontend is Astro which is another brand new framework. I do want to replicate their last teo demo but once I get this simple worker going first. UGHHHHH

mr.niko.la•2/28/25, 2:08 AM

lol cursed gui

DragoDiKomo•2/28/25, 3:55 PM

Im getting this error: 3040 : Capacity temporarily exceeded, please try again. I'm on a paid plan for AI and with an application in production

DDragoDiKomo Im getting this error: 3040 : Capacity temporarily exceeded, please try again. I...

Isaac McFadyen•2/28/25, 3:56 PM

This isn't a per-account limit but a global limit indicating that Workers AI as a service is above capacity.

IIsaac McFadyen This isn't a per-account limit but a global limit indicating that Workers AI as ...

DragoDiKomo•2/28/25, 3:56 PM

Image generation is working

DragoDiKomo•2/28/25, 3:56 PM

btw

Isaac McFadyen•2/28/25, 3:56 PM

Yeah it'll be per-model.

Isaac McFadyen•2/28/25, 3:56 PM

Just depending on how many replicas of that model they have deployed.

IIsaac McFadyen This isn't a per-account limit but a global limit indicating that Workers AI as ...

DragoDiKomo•2/28/25, 3:56 PM

Yup I understand but it's a problem anyway

Isaac McFadyen•2/28/25, 3:56 PM

For sure, just wanted to make you aware.

DragoDiKomo•2/28/25, 3:58 PM

I need to implement a fallback logic with a model that is providing at least the same quality result

workerd/server/workerd-api.c++:759: error: wrapped binding module can't be resolved (internal modules only); moduleName = miniflare-internal:wrapped:__WRANGLER_EXTERNAL_AI_WORKER workerd/jsg/util.c++:331: error: e = workerd/server/workerd-api.c++:789: failed: expected !value.IsEmpty(); global did not produce v8::Value

We have thousands of users of our service complaining because the users LLM requests are being block

Similar Threads

Similar Threads

Similar Threads