Thanks. To be clear, is it true to say that each call to `env.AI.run` = 1 request? Or is it more nua

Thanks. To be clear, is it true to say that each call to

env.AI.run

env.AI.run

env.AI.run

env.AI.run = 1 request? Or is it more nuanced and comes down to the number of generations, as you alude to?

MMitya Thanks. To be clear, is it true to say that each call to `env.AI.run` = 1 reques...

Isaac McFadyen•12/12/24, 3:19 PM

It depends. I'm not super familiar with the Workers AI API—is

env.AI.run

env.AI.run

just the standard generation mechanism or the Functions one?

Isaac McFadyen•12/12/24, 3:20 PM

If it's the standard generation mechanism then yes, unless you're generating more than 1 generation with a single call it's usually 1:1

MityaOP•12/12/24, 3:22 PM

Thanks.

env.AI.run

env.AI.run

is for both standard generation and function calls - it's just that, for the latter, you a toolstools object to the params.

SSuperHelpflare Workers AI currently only supports popular open-source models provided by the Cl...

zcpbx•12/13/24, 3:07 AM

Thank you. Is there a place where we can register interest in future models?

Advany•12/13/24, 11:10 AM

has anyone figured out a way to create workflows using ai and deploy them onto worker for platforms?

AAdvany has anyone figured out a way to create workflows using ai and deploy them onto w...

rob•12/14/24, 1:26 PM

I don't think It's supported on wfp yet

rob•12/14/24, 1:26 PM

How's everyone doing? How's AI?

user•12/15/24, 1:47 AM

is anyone able to get image2image working?

user•12/15/24, 1:47 AM

i have not been able to do it

user•12/15/24, 2:20 AM

i consistently get api error 3001

user•12/15/24, 2:21 AM

all the image-to-text models work fine, but if i pass an image to text-to-image i get error 3001

user•12/15/24, 8:38 PM

ok after some research i figured out its a token issue

user•12/15/24, 8:38 PM

at strength 2 or something, it runs out of tokens very quickly

Ashley•12/16/24, 9:36 AM

Is there an easy way to tell the model size for embedding models? For things like Llama (I know not an embedding model) it’s easy, as it has the size in the name. The BGE Large for example, even Googling, I can’t find its size - it would be really helpful to include this on the model’s page on the CF docs, considering the pricing is based on the model size for embedding models.

AAshley Is there an easy way to tell the model size for embedding models? For things lik...

Raylight•12/16/24, 12:05 PM

Regarding pricing, the docs also state that "Individual model pages will have the pricing listed on them", so I'm kinda waiting for that to happen. In any case large is ~335 million parameters, base ~109 million. You can get a rough idea by looking at the file size of the model (e.g. https://huggingface.co/BAAI/bge-large-en-v1.5/tree/main). Going by the "pricing structure", it looks like base and small would fall in the category <= 150 million parameters.

Ashley•12/16/24, 12:19 PM

Ah cool thank you, I did peek at that page but couldn’t see it but I see it now, thank you!

Uuser at strength 2 or something, it runs out of tokens *very* quickly

Raylight•12/16/24, 12:22 PM

strengthstrength is the fraction of steps where the source image is used, where 1.0 means every step. So it doesn't make sense to set it higher than 1.0. I've noticed that it'll throw an error if you do, however.

user•12/16/24, 1:30 PM

should return a better error than just “internal server error” though

Bossman•12/17/24, 11:51 AM

are custom workers ai model supported? i would like to host it myself

BBossman are custom workers ai model supported? i would like to host it myself

Chaika•12/17/24, 2:30 PM

you can't currently upload custom models if that's what you're asking

Chaika•12/17/24, 2:30 PM

?workers-ai-models

SuperHelpflare•12/17/24, 2:30 PM

Workers AI currently only supports popular open-source models provided by the Cloudflare team, as well as your own LoRAs that can be applied on top of the Cloudflare-provided models. You cannot currently upload your own models or use a model from HuggingFace. See the documentation for the list of Cloudflare-provided models: https://developers.cloudflare.com/workers-ai/models/

James•12/17/24, 10:27 PM

Looks like @cf/openai/whisper-large-v3-turbo@cf/openai/whisper-large-v3-turbo is available now?

JJames Looks like `@cf/openai/whisper-large-v3-turbo` is available now? 👀

Isaac McFadyen•12/17/24, 11:16 PM

Yay!

sam•12/18/24, 8:59 AM

Is there any way to see the actual pricing of Worker AI? I am using the @cf/baai/bge-large-en-v1.5 model for vector embedding, with each text having 100 input tokens. I ran it for 50000 texts, and based on that, pricing should have been $0.075, but in the worker AI usage dashboard, I see $0.45.

I need to run it for millions of texts and I am worried about getting the huge check at the end

antim8•12/19/24, 4:01 AM

there seem to be models on https://developers.cloudflare.com/workers-ai/models/ that are not usable

Cloudflare Docs

Models · Cloudflare Workers AI docs

Brandon Studio•12/19/24, 7:31 AM

I'm using messages API body, why it requires promptprompt?

scotto•12/19/24, 7:35 PM

llama-3.3-70b-instruct-fp8-fast is taking 20 sec on avg, that's normal?

velinxs•12/19/24, 8:38 PM

It's fast relative to CF workers running at edge not so much fast like groq is fast.

BBossman are custom workers ai model supported? i would like to host it myself

velinxs•12/19/24, 8:39 PM

You could use AI gateway

velinxs•12/19/24, 8:39 PM

I think?

tornado•12/21/24, 2:03 AM

Do AI gateway requests have 100s timeout limitations like other cloudflare workers

enkoder•12/21/24, 3:13 PM

Hey folks, if I'm understanding this correctly, the most recent blog on Anthropics MCP protocol and the workers-mcp project is using a local proxy and some tooling to generate some local config to then proxy to a running worker.

https://blog.cloudflare.com/model-context-protocol/

What are the limitations that drove this design? I looked into this and see that the MCP protocol requires a TCP connection and server-sent events to implement the full server protocol. Which means long lasting connections. Workers DO can hold the TCP connection though so I'm not exactly sure whats getting in the way for anyone to build a full MCP server using just workers without a local proxy like workers-mcp did. Anyone got any insight here?

The Cloudflare Blog

Hi Claude, build an MCP server on Cloudflare Workers

Want Claude to interact with your app directly? Build an MCP server on Workers. That will enable you to connect your service directly, allowing Claude to understand and run tasks on your behalf.

enkoder•12/21/24, 3:19 PM

@geelen as the author and dev can you comment on the above?

enkoder•12/22/24, 4:24 PM

Awesome thanks for sharing the docs!

dsds10•12/25/24, 11:41 AM

It was probably asked a few times, but I couldn‘t find an answer: Are the limits (eg text gen 300 req/min) per worker or per account?

Erisa•12/25/24, 11:54 AM

per account

velinxs•12/26/24, 10:04 PM

Anyone smarter than me that can explain what that Claude MCT is id be forever grateful. Having a hard time conceptualizing

James•12/29/24, 3:13 PM

Please Cloudflare, reconsider this breaking change to Workers AI types: https://github.com/cloudflare/workerd/pull/3278#pullrequestreview-2524962497

LLogan Grasby We definitely don't intend to moderate content and we'll hopefully add documenta...

Buno•12/30/24, 1:30 AM

Heya, I understand that there's been some issues with the NSFW filter flaring up on certain one word inputs. The current solution recommended is to add more words, yet I saw this comment and I was curious if any documentation has been released on disabling it or potentially lowering the NSFW "range" of inputs detected? Just wondering if there has been any updates or documentation I might've missed!

n0riskn0r3ward•12/31/24, 2:07 PM

It's high time for workers ai lora hosting to move on from Llama 2. No one has trained a fine tune on llama 2 in >1 year. Would love to see Qwen 2.5 7B on this list.

Advany•12/31/24, 9:44 PM

Code that cloudflare ai creates. Is there a way I can run that for the user as a eval?

Only way I know is using a worker for platform worker and calling that...

BumblebeeSquare•1/1/25, 8:28 PM

hi all, how do i track model chat token usage and other numbers so that i know how much i should bill users ? is there an built-in rest api

James•1/1/25, 8:30 PM

Tracking usage of things is a common use-case for Analytics Engine: https://developers.cloudflare.com/analytics/analytics-engine/

Cloudflare Docs

Workers Analytics Engine · Cloudflare Analytics docs

Workers Analytics Engine provides unlimited-cardinality analytics at scale, via a built-in API to write data points from Workers, and a SQL API to query that data.

BBumblebeeSquare hi all, how do i track model chat token usage and other numbers so that i know h...

dattaproffs•1/2/25, 10:07 PM

look at https://developers.cloudflare.com/ai-gateway/

Cloudflare Docs

AI Gateway · Cloudflare AI Gateway docs

Cloudflare's AI Gateway allows you to gain visibility and control over your AI apps. By connecting your apps to AI Gateway, you can gather insights on how people are using your application with analytics and logging and then control how your application scales with features such as caching, rate limiting, as well as request retries, model fallba...

Thanks. To be clear, is it true to say that each call to `env.AI.run` = 1 request? Or is it more nua

Similar Threads

Similar Threads

Similar Threads