Cloudflare Developers•11mo ago

Cf: <https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/> > $0.50 per

Cf: https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/

$0.50 per 1M input token
$4.88 per 1M output token

DeepSeek: https://api-docs.deepseek.com/quick_start/pricing (deepseek-reasoner)

$0.55 per 1M input token (cache miss; $0.14 if cache hit)
$2.19 per 1M output token

Yeh not sure how to explain that one unfortunately

BBob Hello, I am using the "bge-large-en-v1.5" model from cloudflare and same model r...

Bob•1/29/25, 7:08 PM

finally found the reason why cloudflare ai and hugging face endpoint/my local give different results

hugging face endpoint most likely using "sentence_transformers" and cloudflare ai using "transformers" with normalization.
now I can die in peace

MMattIPv4 Cf: <https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwe...

topmass•1/29/25, 7:32 PM

This is also comparing the Full (non distilled) R1 to 32b distilled R1 reasoning Qwen model. In extensive testing over the past few days I could not confidently recommend anything below 70b distilled variant for production, the thinking tokens are decent on all but when acting upon them the performance is still a bit hodge-podge. You can see this in use cases like asking for simple structured outputs or json function calling - I hope workers ai will consider lowering the price significantly or offering the 70b distilled variant if wanting to charge this high

rob•1/29/25, 10:30 PM

yeaaaaa lets go AGENTS~

rob•1/30/25, 2:18 PM

we want full r1!!! not that "we have r1 at home" version plz

scotto•1/30/25, 3:28 PM

https://mistral.ai/news/mistral-small-3/

Mistral Small 3

Apache 2.0, 81% MMLU, 150 tokens/s

Charles Von Habsburg•1/30/25, 3:41 PM

Hello fellow dev, i have a problem with workers ai. There is no error in the codes and im using hono, im running a simple llama prompt just like the documentation, but when i request the route it gives me: error 10000 Authentication error on the wrangler console

CCharles Von Habsburg Hello fellow dev, i have a problem with workers ai. There is no error in the cod...

Mathis @ blaxel.ai•1/30/25, 5:04 PM

Hey! Are you using worker bindings to interact with your model?

Charles Von Habsburg•1/30/25, 5:05 PM

Yes, heres the code, i wrote the code in help section

CCharles Von Habsburg Yes, heres the code, i wrote the code in help section

Mathis @ blaxel.ai•1/30/25, 5:05 PM

Let me check this out

Charles Von Habsburg•1/30/25, 5:07 PM

Thanks

CCharles Von Habsburg Thanks

Mathis @ blaxel.ai•1/30/25, 5:08 PM

I can't find, are you sure you've posted it?

Charles Von Habsburg•1/30/25, 5:10 PM

Im sorry, its in the q and a section

Charles Von Habsburg•1/30/25, 5:10 PM

I know its in the wrong place

MMathis @ blaxel.ai I can't find, are you sure you've posted it?

Charles Von Habsburg•1/30/25, 5:14 PM

MMattIPv4 Cf: <https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwe...

�

😈 Donkey 💫•1/31/25, 11:06 AM

This price for joining the hype lmao.
CF AI now is a OpenRouter provider

user•2/2/25, 1:04 AM

:frown: very unfortunate to see that the r1 cf has is majorly censored like the chinese hosted one.

user•2/2/25, 1:06 AM

i thought the OSS model one didnt have the major censorship

user•2/2/25, 1:06 AM

only the deepseek's official one

user•2/2/25, 1:06 AM

mightve been misinformed

user•2/2/25, 1:23 AM

i see

rob•2/2/25, 8:08 PM

hit up the PM of workers AI on Twitter , they love hearing what models ppl want

Kyle G•2/3/25, 6:23 PM

Looking for a sanity check on an idea here...is it crazy that I'm considering a quick POC of CF agents + browsing to do basic app testing validation? The dream idea in my head is like "go to this URL and login with these credentials and then use the app, playing music, adding songs to a playlist, your library"

feeling like I might be able to massively and quickly improve testing coverage/bug reporting and also massively decrease the difficulty of "writing tests" but idk if I would be using the tool incorrectly if I go down this path.

the enger•2/3/25, 10:42 PM

is https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/ working? getting 'Request body is not valid json" back

Cloudflare Docs

whisper-large-v3-turbo · Cloudflare Workers AI docs

Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation.

the enger•2/3/25, 10:48 PM

oh got it - docs still show Uint8Array for typescript worker / --data-binary for the curl, but audio should be sent by encoding it with the Base64

merlo•2/4/25, 3:12 PM

Is there any roadmap for models being added to cloudflare's catalogue? I feel like the options available are surprisingly limited. IE only 1 model that supports function calling.

Mitya•2/4/25, 6:03 PM

How does one pick a text generation (or any) model from those available? On the models page, even filtering to "text generation" still results in 10s of models. Some are named such that their purpose is obvious e.g. "deepseek-coder", but of the more general ones, is there one that is considered more performant?

MMitya How does one pick a text generation (or any) model from those available? On the ...

Isaac McFadyen•2/4/25, 6:47 PM

Two main criteria:

The bigger the weight the more intelligent they will (generally) be, i.e. a 70b will usually beat an 8b although newer 8b might come close, but also the slower they will return a response
The provider who trained it, i.e. Mistral or Llama are both good options but it's really just a matter of testing to see which is better for your specific use-case

IIsaac McFadyen Two main criteria: - The bigger the weight the more intelligent they will (gener...

Mitya•2/4/25, 7:00 PM

That's helpful - thanks!

Marius B.•2/5/25, 9:23 AM

Someone please help me understand Workers AI vs Deepseek Pricing. The DeepSeek API costs $0.28 per million output tokens. Running DeepSeek on Workers would cost $4.88 per 1M output token + the unbound worker usage, I assume. But this is a big discrepancy in price. Am I getting something wrong?
https://api-docs.deepseek.com/quick_start/pricing
https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/

MMattIPv4 Cf: <https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwe...

Advany•2/5/25, 10:24 AM

already anwsered, scroll back

rob•2/5/25, 3:29 PM

tldr. CHOYNA

Mitya•2/5/25, 3:46 PM

Does anyone have any view on which is the best text genertion model that reliably outputs JSON? I'm finding that llama-3.3-70b-instruct-fp8-fastllama-3.3-70b-instruct-fp8-fast is reliable about 70% of the time, but will occasionally throw in an invalid bracket here and there. Any other models people are using for this? (Yes I know there's Function Calls as well.)

MMitya Does anyone have any view on which is the best text genertion model that reliabl...

Isaac McFadyen•2/5/25, 6:28 PM

Your best bet is to implement retries, really.

Isaac McFadyen•2/5/25, 6:28 PM

Try parsing the JSON and if it fails re-send the request.

Isaac McFadyen•2/5/25, 6:28 PM

Unless specifically forced to output JSON via low-level logit math (which Workers AI doesn't support, I don't think) then there will always be some cases where it returns invalid JSON.

Isaac McFadyen•2/5/25, 6:29 PM

You can try to improve it by using specific system messages but it'll never be 100% without forced logits.

IIsaac McFadyen Your best bet is to implement retries, really.

Mitya•2/6/25, 9:27 AM

Thanks. Yeah this is what I'm doing currently. It's helping a lot, but as you say, there's always some cases. I note that the mistral model (CF's single function calls-capable model, at least according to the filters on the models page) has a "JSON mode". But in the input params, this is not mentioned, so supposedly there's a way to force it to return JSON but I'm unsure whether there's a param missing from the docs.

Mitya•2/6/25, 9:28 AM

Also, this looks interesting re: structured output: https://js.langchain.com/docs/how_to/structured_output/

How to return structured data from a model | 🦜️🔗 Langchain

It is often useful to have a model return output that matches some

MMitya Thanks. Yeah this is what I'm doing currently. It's helping a lot, but as you sa...

Isaac McFadyen•2/6/25, 1:01 PM

I don't believe there is. That's for when you're running it locally.

MMitya Thanks. Yeah this is what I'm doing currently. It's helping a lot, but as you sa...

thomash•2/6/25, 2:04 PM

if you are using the OpenAI compatible API, I think you can enable JSON mode by passing the correct response_formatresponse_format. Since it is possible to access of CloudFlare's models via the OpenAI API.

Tthomash if you are using the OpenAI compatible API, I think you can enable JSON mode by ...

Isaac McFadyen•2/6/25, 2:15 PM

Cloudflare's API does not respect the JSON mode setting iirc

Isaac McFadyen•2/6/25, 2:15 PM

I tried the other day and it ignored it.

Torianna•2/6/25, 2:16 PM

to be fair, json mode is only supported on a small amount of models

Torianna•2/6/25, 2:16 PM

since it requires modifying how the model responds

TTorianna to be fair, json mode is only supported on a small amount of models

Isaac McFadyen•2/6/25, 2:17 PM

Kind of? Like, the models themselves have "JSON mode" but it's not actually that hard to force it to be JSON, it just needs some forced logits but theoretically any backend that does that will work with any model.

Isaac McFadyen•2/6/25, 2:17 PM

I just don't think CF's backend has that setting or if it does it's not enabled.

Cf: <https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/> > $0.50 per

Similar Threads