Hello, I am using the "bge-large-en-v1.5" model from cloudflare and same model running in local that

Hello, I am using the "bge-large-en-v1.5" model from cloudflare and same model running in local that I got from huggingface via SentenceTransformer.
For same string I am getting different vectors between cloudflare hosted model and local (screenshots - cloudflare ai in postman vs local with https://huggingface.co/BAAI/bge-large-en-v1.5)

Shouldn't these be exactly same?
One possible reason can be that cloudflare is using a older or different version than the hugging face one.
Also, what is the strategy from cloudflare to support these models? If I use the model for my current data set, I should have access to the model for a very long time to query too.

BobOP•1/27/25, 11:40 AM

cosine similarity score between them is [0.9773116111755371]. so they are not very very close

BBob Hello, I am using the "bge-large-en-v1.5" model from cloudflare and same model r...

Isaac McFadyen•1/27/25, 12:04 PM

CUDA is not deterministic meaning you won't get the same outputs every time, especially across GPU models.

BobOP•1/27/25, 12:07 PM

oh damn. I didnt know that. thanks for the reply

BBob oh damn. I didnt know that. thanks for the reply

Isaac McFadyen•1/27/25, 12:15 PM

You also aren't passing

normalize_embeddings=True

normalize_embeddings=True

locally which the docs recommend (https://huggingface.co/BAAI/bge-large-en-v1.5#using-sentence-transformers).

BAAI/bge-large-en-v1.5 · Hugging Face

Isaac McFadyen•1/27/25, 12:15 PM

I expect if you pass that flag to encode()encode() you might get closer outputs.

BobOP•1/27/25, 12:26 PM

still same results. I think cloudflare runs the the models in different architecture or via different libraries than what I am doing.
Chatgpt says pytorch has some different settings for deterministic algo on different GPUs which might be different depending on implementation.

It is not a big deal, I just wanted to have a backup in case cloudflare discontinue this model in future and I might still need to query it for sometime while I re calculate the embeddings with newer model

BobOP•1/27/25, 12:38 PM

I ran same query on huggingface now via their api.
it is exact same result that I get in local.
Then I guess cloudflare's architecture/libraries are different

scotto•1/27/25, 12:49 PM

It would be good to know cloud flare preprocess and tokenization process, result should be the same

BobOP•1/27/25, 12:59 PM

yes some details around their setup in docs will be helpful for noobs like me

goris29•1/27/25, 5:06 PM

Is there an outage at the moment? I'm getting this returned from my workers AI binding:

{
  "httpCode": 500,
  "message": "Error: Network connection lost.",
  "name": "AiError"
}

{
  "httpCode": 500,
  "message": "Error: Network connection lost.",
  "name": "AiError"
}

I'm seeing this both in the response in my application and in AI Gateway.

It was working fine on Friday when I last tested it.

I'm using the @cf/llava-hf/llava-1.5-7b-hf@cf/llava-hf/llava-1.5-7b-hf model.

Ggotfredsen Can we get DeepSeek R1 on cloudflare Workers AI today ? 🙂 https://github.com/d...

�

😈 Donkey 💫•1/27/25, 6:23 PM

CF AI seems slept for a while, with many many ancient AI models

rob•1/28/25, 12:26 PM

what happened

Ian_X12•1/28/25, 3:38 PM

Is this product dead? Not seeing any recent models.

James•1/28/25, 7:32 PM

Looks like @cf/deepseek-ai/deepseek-r1-distill-qwen-32b@cf/deepseek-ai/deepseek-r1-distill-qwen-32b just landed in Workers AI

Advany•1/29/25, 10:41 AM

did you add it to toml? [ai]
binding = "AI"

topmass•1/29/25, 5:33 PM

Can someone explain the pricing for deepseek-r1-distill-qwen-32b? - most distillations below 70b are a bit underwhelming on performance and the 32b version is priced higher than the full R1 on deepseek api ?

Ttopmass Can someone explain the pricing for deepseek-r1-distill-qwen-32b? - most distill...

MattIPv4•1/29/25, 7:03 PM

Cf: https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwen-32b/

$0.50 per 1M input token
$4.88 per 1M output token

DeepSeek: https://api-docs.deepseek.com/quick_start/pricing (deepseek-reasoner)

$0.55 per 1M input token (cache miss; $0.14 if cache hit)
$2.19 per 1M output token

Yeh not sure how to explain that one unfortunately

BBob Hello, I am using the "bge-large-en-v1.5" model from cloudflare and same model r...

BobOP•1/29/25, 7:08 PM

finally found the reason why cloudflare ai and hugging face endpoint/my local give different results

hugging face endpoint most likely using "sentence_transformers" and cloudflare ai using "transformers" with normalization.
now I can die in peace

MMattIPv4 Cf: <https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwe...

topmass•1/29/25, 7:32 PM

This is also comparing the Full (non distilled) R1 to 32b distilled R1 reasoning Qwen model. In extensive testing over the past few days I could not confidently recommend anything below 70b distilled variant for production, the thinking tokens are decent on all but when acting upon them the performance is still a bit hodge-podge. You can see this in use cases like asking for simple structured outputs or json function calling - I hope workers ai will consider lowering the price significantly or offering the 70b distilled variant if wanting to charge this high

rob•1/29/25, 10:30 PM

yeaaaaa lets go AGENTS~

rob•1/30/25, 2:18 PM

we want full r1!!! not that "we have r1 at home" version plz

scotto•1/30/25, 3:28 PM

https://mistral.ai/news/mistral-small-3/

Mistral Small 3

Apache 2.0, 81% MMLU, 150 tokens/s

Charles Von Habsburg•1/30/25, 3:41 PM

Hello fellow dev, i have a problem with workers ai. There is no error in the codes and im using hono, im running a simple llama prompt just like the documentation, but when i request the route it gives me: error 10000 Authentication error on the wrangler console

CCharles Von Habsburg Hello fellow dev, i have a problem with workers ai. There is no error in the cod...

Mathis @ blaxel.ai•1/30/25, 5:04 PM

Hey! Are you using worker bindings to interact with your model?

Charles Von Habsburg•1/30/25, 5:05 PM

Yes, heres the code, i wrote the code in help section

CCharles Von Habsburg Yes, heres the code, i wrote the code in help section

Mathis @ blaxel.ai•1/30/25, 5:05 PM

Let me check this out

Charles Von Habsburg•1/30/25, 5:07 PM

Thanks

CCharles Von Habsburg Thanks

Mathis @ blaxel.ai•1/30/25, 5:08 PM

I can't find, are you sure you've posted it?

Charles Von Habsburg•1/30/25, 5:10 PM

Im sorry, its in the q and a section

Charles Von Habsburg•1/30/25, 5:10 PM

I know its in the wrong place

MMathis @ blaxel.ai I can't find, are you sure you've posted it?

Charles Von Habsburg•1/30/25, 5:14 PM

MMattIPv4 Cf: <https://developers.cloudflare.com/workers-ai/models/deepseek-r1-distill-qwe...

�

😈 Donkey 💫•1/31/25, 11:06 AM

This price for joining the hype lmao.
CF AI now is a OpenRouter provider

user•2/2/25, 1:04 AM

:frown: very unfortunate to see that the r1 cf has is majorly censored like the chinese hosted one.

user•2/2/25, 1:06 AM

i thought the OSS model one didnt have the major censorship

user•2/2/25, 1:06 AM

only the deepseek's official one

user•2/2/25, 1:06 AM

mightve been misinformed

user•2/2/25, 1:23 AM

i see

rob•2/2/25, 8:08 PM

hit up the PM of workers AI on Twitter , they love hearing what models ppl want

Kyle G•2/3/25, 6:23 PM

Looking for a sanity check on an idea here...is it crazy that I'm considering a quick POC of CF agents + browsing to do basic app testing validation? The dream idea in my head is like "go to this URL and login with these credentials and then use the app, playing music, adding songs to a playlist, your library"

feeling like I might be able to massively and quickly improve testing coverage/bug reporting and also massively decrease the difficulty of "writing tests" but idk if I would be using the tool incorrectly if I go down this path.

the enger•2/3/25, 10:42 PM

is https://developers.cloudflare.com/workers-ai/models/whisper-large-v3-turbo/ working? getting 'Request body is not valid json" back

Cloudflare Docs

whisper-large-v3-turbo · Cloudflare Workers AI docs

Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation.

the enger•2/3/25, 10:48 PM

oh got it - docs still show Uint8Array for typescript worker / --data-binary for the curl, but audio should be sent by encoding it with the Base64

merlo•2/4/25, 3:12 PM

Is there any roadmap for models being added to cloudflare's catalogue? I feel like the options available are surprisingly limited. IE only 1 model that supports function calling.

Mitya•2/4/25, 6:03 PM

How does one pick a text generation (or any) model from those available? On the models page, even filtering to "text generation" still results in 10s of models. Some are named such that their purpose is obvious e.g. "deepseek-coder", but of the more general ones, is there one that is considered more performant?

Hello, I am using the "bge-large-en-v1.5" model from cloudflare and same model running in local that

Similar Threads

Similar Threads

Similar Threads