are custom workers ai model supported? i would like to host it myself
are custom workers ai model supported? i would like to host it myself
Cyou can't currently upload custom models if that's what you're asking
C?workers-ai-models
SWorkers AI currently only supports popular open-source models provided by the Cloudflare team, as well as your own LoRAs that can be applied on top of the Cloudflare-provided models. You cannot currently upload your own models or use a model from HuggingFace. See the documentation for the list of Cloudflare-provided models: https://developers.cloudflare.com/workers-ai/models/
JLooks like 
@cf/openai/whisper-large-v3-turbo is available now? 
IYay!
SIs there any way to see the actual pricing of Worker AI? I am using the @cf/baai/bge-large-en-v1.5 model for vector embedding, with each text having 100 input tokens. I ran it for 50000 texts, and based on that, pricing should have been $0.075, but in the worker AI usage dashboard, I see $0.45.
I need to run it for millions of texts and I am worried about getting the huge check at the end
I need to run it for millions of texts and I am worried about getting the huge check at the end
A
BI'm using messages API body, why it requires
prompt?
Sllama-3.3-70b-instruct-fp8-fast is taking 20 sec on avg, that's normal?
VIt's fast relative to CF workers running at edge not so much fast like groq is fast.
VYou could use AI gateway
VI think?
TDo AI gateway requests have 100s timeout limitations like other cloudflare workers
EHey folks, if I'm understanding this correctly, the most recent blog on Anthropics MCP protocol and the workers-mcp project is using a local proxy and some tooling to generate some local config to then proxy to a running worker.
https://blog.cloudflare.com/model-context-protocol/
What are the limitations that drove this design? I looked into this and see that the MCP protocol requires a TCP connection and server-sent events to implement the full server protocol. Which means long lasting connections. Workers DO can hold the TCP connection though so I'm not exactly sure whats getting in the way for anyone to build a full MCP server using just workers without a local proxy like workers-mcp did. Anyone got any insight here?
https://blog.cloudflare.com/model-context-protocol/
What are the limitations that drove this design? I looked into this and see that the MCP protocol requires a TCP connection and server-sent events to implement the full server protocol. Which means long lasting connections. Workers DO can hold the TCP connection though so I'm not exactly sure whats getting in the way for anyone to build a full MCP server using just workers without a local proxy like workers-mcp did. Anyone got any insight here?
The Cloudflare Blog
Want Claude to interact with your app directly? Build an MCP server on Workers. That will enable you to connect your service directly, allowing Claude to understand and run tasks on your behalf.
E@geelen as the author and dev can you comment on the above?
EAwesome thanks for sharing the docs!
DIt was probably asked a few times, but I couldn‘t find an answer: Are the limits (eg text gen 300 req/min) per worker or per account?
Eper account
VAnyone smarter than me that can explain what that Claude MCT is id be forever grateful. Having a hard time conceptualizing
JPlease Cloudflare, reconsider this breaking change to Workers AI types: https://github.com/cloudflare/workerd/pull/3278#pullrequestreview-2524962497
BHeya, I understand that there's been some issues with the NSFW filter flaring up on certain one word inputs. The current solution recommended is to add more words, yet I saw this comment and I was curious if any documentation has been released on disabling it or potentially lowering the NSFW "range" of inputs detected? Just wondering if there has been any updates or documentation I might've missed!
NIt's high time for workers ai lora hosting to move on from Llama 2. No one has trained a fine tune on llama 2 in >1 year. Would love to see Qwen 2.5 7B on this list.
ACode that cloudflare ai creates. Is there a way I can run that for the user as a eval?
Only way I know is using a worker for platform worker and calling that...
Only way I know is using a worker for platform worker and calling that...
Bhi all, how do i track model chat token usage and other numbers so that i know how much i should bill users ? is there an built-in rest api
JTracking usage of things is a common use-case for Analytics Engine: https://developers.cloudflare.com/analytics/analytics-engine/
Cloudflare Docs
Workers Analytics Engine provides unlimited-cardinality analytics at scale, via a built-in API to write data points from Workers, and a SQL API to query that data.
DCloudflare Docs
Cloudflare's AI Gateway allows you to gain visibility and control over your AI apps. By connecting your apps to AI Gateway, you can gather insights on how people are using your application with analytics and logging and then control how your application scales with features such as caching, rate limiting, as well as request retries, model fallba...
Uhello, i'm getting this error:
binding ai of type ai failed to generate: internal error; please try again later or contact support [code: 10021]. is anyone else having the same problem, or is there downtime at the moment?U✘ [ERROR] A request to the Cloudflare API (/accounts/xxx/workers/scripts/xxx/versions) failed.
U╰─± wrangler -v
wrangler 3.99.0
-------------------
i didn't change anything in the code, and it was working fine a few hours ago
wrangler 3.99.0-------------------
i didn't change anything in the code, and it was working fine a few hours ago
JCan you share what your AI binding looks like in your
wrangler.toml?Jnevermind, I can reproduce
Jgoing to escalate
UThanks James!
Sjust wanted to tag on that im also experiencing this @James and thankful im not the only one haha
JThe team are aware and actively looking into this, thanks folks
UYes it does tysm!
VWorkers AI's OpenAI compatible endpoint is missing the
(screenshot is returned)
index property during text chat completion (streaming) https://platform.openai.com/docs/api-reference/chat/streaming(screenshot is returned)
SHey guys, I am trying to use this text generation model but I am getting a typescript error:
const response = await context.env.AI.run("@cf/meta/llama-3.3-70b-instruct-fp8-fast", { messages });
No overload matches this call.
The last overload gave the following error.
Argument of type '"@cf/meta/llama-3.3-70b-instruct-fp8-fast"' is not assignable to parameter of type 'BaseAiImageToTextModels'.ts(2769)
index.d.ts(3720, 3): The last overload is declared here.
Has anyone been through this before? I was able to find an open issue on github but the PR is still open so I assume they haven't pushed a fix for it
const response = await context.env.AI.run("@cf/meta/llama-3.3-70b-instruct-fp8-fast", { messages });
No overload matches this call.
The last overload gave the following error.
Argument of type '"@cf/meta/llama-3.3-70b-instruct-fp8-fast"' is not assignable to parameter of type 'BaseAiImageToTextModels'.ts(2769)
index.d.ts(3720, 3): The last overload is declared here.
Has anyone been through this before? I was able to find an open issue on github but the PR is still open so I assume they haven't pushed a fix for it
JThat should be fixed with the most recent workers-types version, via https://github.com/cloudflare/workerd/pull/3242. Are you up to date?
BHey, the docs are sorta confusing, Are we able to pay for more neurons?
If so, is it monthly, could you provide some help about the neurons?
If so, is it monthly, could you provide some help about the neurons?
BSince the docs say we have 100k requests, but ai says we have 10k neurons
JWorkers AI pricing is all kinds of confusing right now. Docs talk about tokens which is the “new” method that was set to go into action on November 1st but never happened, dashboard talks about neurons, old marketing talks about neurons, etc.
JHopefully we can get some more official clarification and updates soon
BHow are we able to get more neurons?
JNeurons aren’t a thing anymore - it’s token based now, but nothing in the UI reflects that
