the deepseek model: `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` response sometime got empty re

the deepseek model:

@cf/deepseek-ai/deepseek-r1-distill-qwen-32b

@cf/deepseek-ai/deepseek-r1-distill-qwen-32b

response sometime got empty response:

data: {"response":""}

data: {"response":""}

data: {"response":"","usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1}}

data: {"response":""}

data: {"response":""}

data: {"response":"","usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1}}

data: {"response":""}

data: {"response":""}

data: {"response":"","usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1}}

data: {"response":""}

data: {"response":""}

data: {"response":"","usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1}}

and sometimes it works all right, but missing the opening <think> tag. any others rushed into same problem?

Ccyrilis the deepseek model: `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` response s...

cyrilisOP•3/27/25, 2:24 PM

Or maybe it's just because I enabled smart placement? and a few AI models are not ready in some region ?

Ccyrilis the deepseek model: `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` response s...

Isaac McFadyen•3/27/25, 3:21 PM

missing the opening <think> tag

This is due to a quirk in the prompt template, which forces the <think> tag in order to require the model to think through it's response, but which results in this behavior

Isaac McFadyen•3/27/25, 3:21 PM

@michelle ^ been reported many times here, might want to fix it eventually

Isaac McFadyen•3/27/25, 3:22 PM

Not sure if it was always the case or whether it was a change but regardless, it's different from how other providers generally do it

Pytato•3/28/25, 6:58 AM

I think I have found a bug with Cloudflare's Worker Ai

I'm basically making a super simple Stable Diffusion wrapper with Pages and Worker Ai. While making the UI and testing it, I probably made 50 - 80
50 - 80
images with it. But my Daily Neurons Limits is still 0
0
(as in nothing used)

So, I checked the Pricing and found no mention of Stable Diffusion's neuron pricing.

Could it be they forgot to consider these model to add to the daily usage limit?
OR is 80
80
images too low to be show in the dashboard?

Pytato•3/28/25, 7:03 AM

Oh... I'm stupid

mr.niko.la•3/28/25, 2:56 PM

@cf/openai/whisper-large-v3-turbo

@cf/openai/whisper-large-v3-turbo

is there no sample worker code for this for something basic like r2 audio file to transcribe ?

I am using a worker to send it to 3rd party api to transcribe using queue

But I would like use worker ai whisper turbo as a failover. Without over engineering or breaking my webapp.

Do I just implement a new queue instead of a separate worker?

Perhaps eventually cf workflow but for now I don’t want to break my current webapp by reengineering anything too much.

Just want to do backup for the rare times 3rd api fails.

Mistic92•3/29/25, 5:34 PM

I'm getting 3001 error when trying to use Whisper :/

{"errors":[{"message":"AiError: AiError: Unknown internal error (8e2b7fc6-c9be-4abd-b5da-82def70cb5be)","code":3001}],"success":false,"result":{},"messages":[]}%

{"errors":[{"message":"AiError: AiError: Unknown internal error (8e2b7fc6-c9be-4abd-b5da-82def70cb5be)","code":3001}],"success":false,"result":{},"messages":[]}%

Jaka•3/30/25, 11:25 AM

I need help with pricing for creating embendings, storing them in vectorize for almost 280k md files about Romanian legislation. i have them in R2. i want to create a rag that is capable of answering very complex questions about Romanian legislation. i did some estimations uisng chatgpt but i hope it is wrong( almost 8k was the estimation for embedings).

Brandon Studio•3/31/25, 8:18 AM

Currently stream APIs (including official API and OpenAI compatible API) do not properly handle tool_call response; the tool call content is embedded in the

response

response

content

content

field.
Could the stream API provide the same behavior as the non-stream API? For example,

"response“: null, "tool_calls": [...]

"response“: null, "tool_calls": [...]

Brandon Studio•3/31/25, 8:19 AM

Also, OpenAI compatible API only returns

null

null

in non-stream mode

vivsam•3/31/25, 8:38 AM

It'd be nice if you guys add Zero-Shot-Text-Classification models like https://huggingface.co/facebook/bart-large-mnli

facebook/bart-large-mnli · Hugging Face

sku•3/31/25, 5:46 PM

I've tried to make requests to several different models today over the REST API and I've only gotten this response. Are there any known issues?

{
    "result": null,
    "success": false,
    "errors": [
        {
            "code": 7010,
            "message": "Service unavailable"
        }
    ],
    "messages": []
}

{
    "result": null,
    "success": false,
    "errors": [
        {
            "code": 7010,
            "message": "Service unavailable"
        }
    ],
    "messages": []
}

BBrandon Studio Currently stream APIs (including official API and OpenAI compatible API) do not ...

Isaac McFadyen•3/31/25, 8:28 PM

Is this for AI Gateway or Workers AI?

Isaac McFadyen•3/31/25, 8:28 PM

And what models does this occur on?

nclevenger•3/31/25, 9:49 PM

hey - this new model is amazing ... it's SOTA on SWE Bench for a 32b model, scoring 37% which is SHOCKING for a model this small ... would love to see this on Workers AI https://huggingface.co/all-hands/openhands-lm-32b-v0.1

all-hands/openhands-lm-32b-v0.1 · Hugging Face

IIsaac McFadyen Is this for AI Gateway or Workers AI?

Brandon Studio•4/1/25, 2:09 AM

Workers AI, behaviors with and without AI Gateway are same

IIsaac McFadyen And what models does this occur on?

Brandon Studio•4/1/25, 2:12 AM

I see the two models (

@hf/nousresearch/hermes-2-pro-mistral-7b

@hf/nousresearch/hermes-2-pro-mistral-7b

and

@cf/meta/llama-3.3-70b-instruct-fp8-fast

@cf/meta/llama-3.3-70b-instruct-fp8-fast

) behaves differently.
Response of hermes does not handle

<tool_call>

<tool_call>

label in stream mode, as shown in Image 1.
Response of llama 3.3 does not distinguish text content and tool call, as shown in Image 2.

Brandon Studio•4/1/25, 2:14 AM

I expect the tool call response to present in

tool_calls

tool_calls

field rather than

response

response

field, just like non-stream mode

Ssku I've tried to make requests to several different models today over the REST API ...

glncy•4/1/25, 4:52 AM

same i am gettings this error on my local. but when i try to deploy a sample workers ai, it's working.

Original message was deleted

Isaac McFadyen•4/1/25, 3:44 PM

That is for Cloudflare Stream which is a different product.

Isaac McFadyen•4/1/25, 3:44 PM

I know for sure it accepts WAV 16khz mono, but I'm not sure about other formats.

glncy•4/1/25, 6:10 PM

how do i access workers ai locally? i have this nextjs app. using bindings and it returns me service unavailable 7010. even on rest api, it returns me the same response. i've been stuck for two days about this :3

Ssku I've tried to make requests to several different models today over the REST API ...

glncy•4/1/25, 7:06 PM

hi @sku want to know if you already figured this out? thanks

Mmichelle this seems to be a bug from an upstream service, we're taking a look

glncy•4/1/25, 7:34 PM

hi! thank you for your help. i already sent my account ID to you.

mr.niko.la•4/2/25, 8:16 PM

The whisper v3 turbo has 25mb file limit ? Would be nice if the website has this details ……

Spencer Hurst•4/3/25, 9:10 PM

Hello. Does anybody have experience using a custom lora on Workers AI? I uploaded the lora files and the finetune is listed via "wrangler ai finetune list", but when I try and do inference via curl with the finetune ID reported by wrangler, it says finetune not found. The adapter works on huggingface with the same files, and I can do inference on the base model (mistral 7b instruct v0.2) via curl fine. Does anybody know the right way to do this?

This is the error I'm getting (all redacted fields filled in properly):

$ curl https://api.cloudflare.com/client/v4/accounts/MY_ACCOUNT_ID/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora \
  -H 'Authorization: Bearer MY_API_TOKEN' \
  -d '{
    "messages": [{"role": "user", "content": "Hello world"}],
    "raw": true,
    "lora": "MY_LORA_ID"
  }'
{"errors":[{"message":"AiError: AiError: Finetune MY_LORA_ID not found (07af68df-18b6-4a5d-bfeb-3453830da3be)","code":3037}],"success":false,"result":{},"
messages":[]}

$ curl https://api.cloudflare.com/client/v4/accounts/MY_ACCOUNT_ID/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora \
  -H 'Authorization: Bearer MY_API_TOKEN' \
  -d '{
    "messages": [{"role": "user", "content": "Hello world"}],
    "raw": true,
    "lora": "MY_LORA_ID"
  }'
{"errors":[{"message":"AiError: AiError: Finetune MY_LORA_ID not found (07af68df-18b6-4a5d-bfeb-3453830da3be)","code":3037}],"success":false,"result":{},"
messages":[]}

s•4/4/25, 4:57 AM

sillah•4/4/25, 10:30 AM

Hello, I have recently deployed cloudfare worker. All seems to work well however I can't seem to set max tokens, it always crops output to 256 tokens. Anyone who has resolved this issue curl -X POST https://xxx \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "user", "content": "explain c++ in detail?"}
],
"max_tokens": 500
}'
Response :
{"inputs":{"messages":[{"role":"user","content":"explain c++ in detail?"}]},"response":{"response":"<think>\nOkay, so I'm trying to understand C++ in detail. I've heard it's a powerful programming language, but I'm not exactly sure how it all fits together. Let me start by breaking down what I know and what I need to figure out.\n\nFirst, I know that C++ is an extension of C. So, it includes everything from C but adds more features. But what exactly are those features? I remember something about classes and objects, which makes me think it's object-oriented. That must mean it's good for building large applications where you can structure code more cleanly.\n\nThe assistant mentioned that C++ is compiled, which I think means it's faster because it's converted directly into machine code. But I'm not entirely sure how that process works compared to interpreted languages. Maybe I should look into the compilation steps, like preprocessing, compiling, assembling, and linking. Understanding that could help me see why C++ is efficient.\n\nI'm also a bit confused about the Standard Template Library (STL). I know it's part of C++, but how does it help? It sounds like it provides a lot of useful data structures and algorithms, which would save time when coding. But I'm not sure how to use it effectively or what the main","usage":{"prompt_tokens":13,"completion_tokens":256,"total_tokens":269}}}

nclevenger•4/6/25, 6:43 AM

Will Llama 4 Scout on Workers AI be offered via OpenRouter?

https://openrouter.ai/meta-llama/llama-4-scout
https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/

Llama 4 Scout - API, Providers, Stats

Llama 4 Scout 17B Instruct (16E) is a mixture-of-experts (MoE) language model developed by Meta, activating 17 billion parameters out of a total of 109B. It supports native multimodal input (text and image) and multilingual output (text and code) across 12 supported languages. Run Llama 4 Scout with API

Cloudflare Docs

llama-4-scout-17b-16e-instruct · Cloudflare Workers AI docs

Meta's Llama 4 Scout is a 17 billion parameter model with 16 experts that is natively multimodal. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding.

Nnclevenger Will Llama 4 Scout on Workers AI be offered via OpenRouter? https://openrouter....

Silvan•4/6/25, 2:29 PM

i wondered the same but workers ai is currently not working (anymore) with openrouter

Silvan•4/6/25, 2:30 PM

im assuming it will be when it returns

Jason Barry•4/7/25, 1:39 AM

humble request to redesign https://developers.cloudflare.com/workers-ai/models/ into a simple table that shows model name, context window size, price per input/output token, support for JSON mode, etc. I'm finding I need to drill into each detail page to compare models. I'd prefer to compare at a birds-eye level.

JJason Barry humble request to redesign https://developers.cloudflare.com/workers-ai/models/ ...

mr.niko.la•4/7/25, 7:17 AM

Ya very confusing.

I can’t even find whisper v3 turbo sample code on how to test it or even what the file size limit is or if I will need to chunk it….

namlongday•4/7/25, 10:51 AM

hello claufalre plan business f

LrGian•4/7/25, 4:46 PM

Hello, I want to do a proof of concept (PoC) on an AI worker. How much budget do you think I can make for at least 2 models? I'm looking to find out if it meets the data I need for my application.an approximate

r4v4rg4s•4/7/25, 6:46 PM

Are there any plans for AutoRag service to support multimodal embeddings?

Mmr.niko.la Ya very confusing. I can’t even find whisper v3 turbo sample code on how to tes...

mr.niko.la•4/7/25, 8:12 PM

WHAT IS THE WHISPER V3 TURBO FILE SIZE LIMIT ON WORKERS AI!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

面

面条•4/8/25, 2:22 AM

Are there plans to support some Qwen LLM models?

Juan•4/8/25, 7:07 PM

Hello, what's the best channel to ask questions about the Agents API?

a4amaan•4/8/25, 10:28 PM

Hello, i have mixed text

AMAAN KHAN Mobile: (+92) 321 322 578 Email: a4amaan@asd.com SUMMARY: ▪ 7+ Years of Software Development Experience. ▪ Experience in design, development of web based applications. ▪ Experience in Python: Django, Flask, Django Rest Framework, JWT Authentication, SaaS Based Applications. ▪ Experience in Web Development: HTML, CSS, Javascript, Nodejs, Express, Mysql, Mongodb ▪ Experience Developing SPA in VUE Js, VUEX, LocalStorage along with Rest APIs in Django Rest Framework. ▪ Experience in

AMAAN KHAN Mobile: (+92) 321 322 578 Email: a4amaan@asd.com SUMMARY: ▪ 7+ Years of Software Development Experience. ▪ Experience in design, development of web based applications. ▪ Experience in Python: Django, Flask, Django Rest Framework, JWT Authentication, SaaS Based Applications. ▪ Experience in Web Development: HTML, CSS, Javascript, Nodejs, Express, Mysql, Mongodb ▪ Experience Developing SPA in VUE Js, VUEX, LocalStorage along with Rest APIs in Django Rest Framework. ▪ Experience in

is there anyway to get specific data from this text like name email and contact using any ai model? i mean in proper json format

andyt•4/9/25, 8:47 AM

Hey is there a file size limit for toMarkdown? I have tried a few larger files that are failing

Aa4amaan Hello, i have mixed text ```AMAAN KHAN Mobile: (+92) 321 322 578 Email: a4amaan@...

andyt•4/9/25, 8:47 AM

i think this is what your looking for, https://developers.cloudflare.com/workers-ai/features/json-mode/

Cloudflare Docs

JSON Mode · Cloudflare Workers AI docs

When we want text-generation AI models to interact with databases, services, and external systems programmatically, typically when using tool calling or building AI agents, we must have structured response formats rather than natural language.

mongj•4/9/25, 9:57 AM

hi! what's the difference between @cf/openai/whisper-large-v3-turbo and @cf/openai/whisper? which whisper model is the latter using?

Aandyt i think this is what your looking for, https://developers.cloudflare.com/workers...

a4amaan•4/9/25, 11:11 AM

Really helpfull thanks...

Muhammadreza Haghiri•4/9/25, 12:12 PM

it seems flux schnell is down.

Muhammadreza Haghiri•4/9/25, 12:24 PM

It takes so long and nothing happens.

Aandyt Hey is there a file size limit for toMarkdown? I have tried a few larger files t...

andyt•4/9/25, 1:49 PM

found this, assuming these limist for AutoRag https://developers.cloudflare.com/autorag/configuration/data-source/

Cloudflare Docs

Data source · AutoRAG

AutoRAG currently supports Cloudflare R2 as the data source for storing your knowledge base. To get started, configure an R2 bucket containing your data.

the deepseek model: `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` response sometime got empty re

Similar Threads

Similar Threads

Similar Threads