Seems like all the models on Workers AI have such small Context WIndows, is there any that will be a

S

si(dot).xth•3/20/25, 11:30 AM

my pillow fixed my issue:

raw data for simplicity and efficiency in handling large images, this method may lead to more API calls if the image is re-generated each time it's accessed (Caching handles that last part.)

base64 encoding to minimize API calls over performance (Easier to implement)

Raw data + cache is the only way to go

Original: Blob not supported for workers ai? URL.CreateObjectURL() is not implemented

AAlex Patterson Seems like all the models on Workers AI have such small Context WIndows, is ther...

S

Silvan•3/20/25, 12:01 PM

Having the same issue here. Trying to create a story is fairly hard this way but some models do have bigger context windows but many of the "good ones" are less than 10k

J

JMPA•3/20/25, 9:14 PM

hello, is inpaiting ai workers working currently?

J

JMPA•3/20/25, 9:15 PM

i tried 2 models and they dont works.

J

JMPA•3/20/25, 9:16 PM

stable-diffusion-v1-5-inpainting
API response status: 400 body: {"errors":[{"message":"AiError: Bad input: Error: Type mismatch of '/mask', 'array' not in 'string'","code":5006}],"success":false,"result":{},"messages":[]}
API error: 400 - {"errors":[{"message":"AiError: Bad input: Error: Type mismatch of '/mask', 'array' not in 'string'","code":5006}],"success":false,"result":{},"messages":[]}
Response with status 400 in 1363 ms.

J

JMPA•3/20/25, 10:19 PM

anyone there?

J

James•3/20/25, 10:27 PM

Is this something that worked previously and now doesn't? Can you provide any code to reproduce?

JJMPA hello, is inpaiting ai workers working currently?

R

Raylight•3/22/25, 6:59 AM

As James implied, it would be easier to help if you have a snippet showing the call to the model, but in general: The docs show the same set of parameters (e.g.

image

image

image

image,

mask

mask

mask

mask) for several models, but the only model that does inpainting is

@cf/runwayml/stable-diffusion-v1-5-inpainting

@cf/runwayml/stable-diffusion-v1-5-inpainting

@cf/runwayml/stable-diffusion-v1-5-inpainting

@cf/runwayml/stable-diffusion-v1-5-inpainting.
Make sure to supply both

image

image

image

image and

mask

mask

mask

mask when calling it. The format is

mask: [...new Uint8Array(await maskImage.arrayBuffer())]

mask: [...new Uint8Array(await maskImage.arrayBuffer())]

mask: [...new Uint8Array(await maskImage.arrayBuffer())]

mask: [...new Uint8Array(await maskImage.arrayBuffer())], where

maskImage

maskImage

maskImage

maskImage could be the result of e.g.

await fetch(/* mask image url */)

await fetch(/* mask image url */)

await fetch(/* mask image url */)

await fetch(/* mask image url */).

S

Silvan•3/23/25, 11:09 AM

I really hope we get some “better” text generation models on here soon. i tried every single one for story creation and nothing came even remotely close to claude atm.

O

ochrons•3/23/25, 3:37 PM

I'm using

@cf/openai/whisper-large-v3-turbo

@cf/openai/whisper-large-v3-turbo

@cf/openai/whisper-large-v3-turbo

@cf/openai/whisper-large-v3-turbo to convert audio into text. It works fine when recording on Pixel 9 in AAC format, but fails when recording on an older Oneplus 7T with AAC (I'm using Flutter to build the app). Using an AMR WB format works fine, though. Also all the AAC formats work fine when given to Google Gemini Flash/Pro models

O

ochrons•3/23/25, 3:38 PM

I tried both

MPEG-4 AAC Low complexity

MPEG-4 AAC Low complexity

MPEG-4 AAC Low complexity

MPEG-4 AAC Low complexity and

MPEG-4 High Efficiency AAC

MPEG-4 High Efficiency AAC

MPEG-4 High Efficiency AAC

MPEG-4 High Efficiency AAC formats with identical results

O

ochrons•3/23/25, 3:54 PM

ffprobe

ffprobe

ffprobe

ffprobe says this about the one recorded on Oneplus,

[aac @ 0x560e66ce38c0] Number of scalefactor bands in group (46) exceeds limit (43).

[aac @ 0x560e66ce38c0] Number of scalefactor bands in group (46) exceeds limit (43).

[aac @ 0x560e66ce38c0] Number of scalefactor bands in group (46) exceeds limit (43).

[aac @ 0x560e66ce38c0] Number of scalefactor bands in group (46) exceeds limit (43).

while there are no errors with the AAC coming from pixel 9

0

0xCC•3/25/25, 4:17 AM

Hi the Daily Usage Limits for ai workder is not reset today
(my account 4631e4e4e57a2e43b16a0a5a1490d7c3 )

Original message was deleted

D

DaniFoldi•3/25/25, 8:35 AM

Argh I thought I got them all, thanks

N

nuno•3/25/25, 10:40 AM

Hey guys, I'm trying to generate embeddings using the

@cf/baai/bge-m3

@cf/baai/bge-m3

@cf/baai/bge-m3

@cf/baai/bge-m3 model but I keep getting the error below. Previously I was using the

@cf/baai/bge-base-en-v1.5

@cf/baai/bge-base-en-v1.5

@cf/baai/bge-base-en-v1.5

@cf/baai/bge-base-en-v1.5 model and everything was just fine.

err : AiError: 5006: Error: required properties at '/' are 'contexts'

err : AiError: 5006: Error: required properties at '/' are 'contexts'

err : AiError: 5006: Error: required properties at '/' are 'contexts'

err : AiError: 5006: Error: required properties at '/' are 'contexts'

Does anyone have any idea why is this happening? Any guidance here is much appreciated.

/

/dev/null•3/27/25, 9:51 AM

the cost of flux schnell is 172 neurons per image when I set with 1 step.

C

cyrilis•3/27/25, 2:22 PM

the deepseek model:

@cf/deepseek-ai/deepseek-r1-distill-qwen-32b

@cf/deepseek-ai/deepseek-r1-distill-qwen-32b

@cf/deepseek-ai/deepseek-r1-distill-qwen-32b

@cf/deepseek-ai/deepseek-r1-distill-qwen-32b

response sometime got empty response:

data: {"response":""}

data: {"response":""}

data: {"response":"","usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1}}

data: {"response":""}

data: {"response":""}

data: {"response":"","usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1}}

data: {"response":""}

data: {"response":""}

data: {"response":"","usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1}}

data: {"response":""}

data: {"response":""}

data: {"response":"","usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1}}

and sometimes it works all right, but missing the opening <think> tag. any others rushed into same problem?

Ccyrilis the deepseek model: `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` response s...

C

cyrilis•3/27/25, 2:24 PM

Or maybe it's just because I enabled smart placement? and a few AI models are not ready in some region ?

Ccyrilis the deepseek model: `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` response s...

I

Isaac McFadyen•3/27/25, 3:21 PM

missing the opening <think> tag

This is due to a quirk in the prompt template, which forces the <think> tag in order to require the model to think through it's response, but which results in this behavior

I

Isaac McFadyen•3/27/25, 3:21 PM

@michelle ^ been reported many times here, might want to fix it eventually

I

Isaac McFadyen•3/27/25, 3:22 PM

Not sure if it was always the case or whether it was a change but regardless, it's different from how other providers generally do it

P

Pytato•3/28/25, 6:58 AM

I think I have found a bug with Cloudflare's Worker Ai

I'm basically making a super simple Stable Diffusion wrapper with Pages and Worker Ai. While making the UI and testing it, I probably made 50 - 80
50 - 80
50 - 80
50 - 80 images with it. But my Daily Neurons Limits is still 0
0
0
0 (as in nothing used)

So, I checked the Pricing and found no mention of Stable Diffusion's neuron pricing.

Could it be they forgot to consider these model to add to the daily usage limit?
OR is 80
80
80
80 images too low to be show in the dashboard?

P

Pytato•3/28/25, 7:03 AM

Oh... I'm stupid

M

mr.niko.la•3/28/25, 2:56 PM

@cf/openai/whisper-large-v3-turbo

@cf/openai/whisper-large-v3-turbo

@cf/openai/whisper-large-v3-turbo

@cf/openai/whisper-large-v3-turbo

is there no sample worker code for this for something basic like r2 audio file to transcribe ?

I am using a worker to send it to 3rd party api to transcribe using queue

But I would like use worker ai whisper turbo as a failover. Without over engineering or breaking my webapp.

Do I just implement a new queue instead of a separate worker?

Perhaps eventually cf workflow but for now I don’t want to break my current webapp by reengineering anything too much.

Just want to do backup for the rare times 3rd api fails.

M

Mistic92•3/29/25, 5:34 PM

I'm getting 3001 error when trying to use Whisper :/

{"errors":[{"message":"AiError: AiError: Unknown internal error (8e2b7fc6-c9be-4abd-b5da-82def70cb5be)","code":3001}],"success":false,"result":{},"messages":[]}%

{"errors":[{"message":"AiError: AiError: Unknown internal error (8e2b7fc6-c9be-4abd-b5da-82def70cb5be)","code":3001}],"success":false,"result":{},"messages":[]}%

{"errors":[{"message":"AiError: AiError: Unknown internal error (8e2b7fc6-c9be-4abd-b5da-82def70cb5be)","code":3001}],"success":false,"result":{},"messages":[]}%

{"errors":[{"message":"AiError: AiError: Unknown internal error (8e2b7fc6-c9be-4abd-b5da-82def70cb5be)","code":3001}],"success":false,"result":{},"messages":[]}%

J

Jaka•3/30/25, 11:25 AM

I need help with pricing for creating embendings, storing them in vectorize for almost 280k md files about Romanian legislation. i have them in R2. i want to create a rag that is capable of answering very complex questions about Romanian legislation. i did some estimations uisng chatgpt but i hope it is wrong( almost 8k was the estimation for embedings).

B

Brandon Studio•3/31/25, 8:18 AM

Currently stream APIs (including official API and OpenAI compatible API) do not properly handle tool_call response; the tool call content is embedded in the

response

response

response

response or

content

content

field.
Could the stream API provide the same behavior as the non-stream API? For example,

"response“: null, "tool_calls": [...]

"response“: null, "tool_calls": [...]

"response“: null, "tool_calls": [...]

"response“: null, "tool_calls": [...]

B

Brandon Studio•3/31/25, 8:19 AM

Also, OpenAI compatible API only returns

null

null

in non-stream mode

V

vivsam•3/31/25, 8:38 AM

It'd be nice if you guys add Zero-Shot-Text-Classification models like https://huggingface.co/facebook/bart-large-mnli

facebook/bart-large-mnli · Hugging Face

S

sku•3/31/25, 5:46 PM

I've tried to make requests to several different models today over the REST API and I've only gotten this response. Are there any known issues?

{
    "result": null,
    "success": false,
    "errors": [
        {
            "code": 7010,
            "message": "Service unavailable"
        }
    ],
    "messages": []
}

{
    "result": null,
    "success": false,
    "errors": [
        {
            "code": 7010,
            "message": "Service unavailable"
        }
    ],
    "messages": []
}

{
    "result": null,
    "success": false,
    "errors": [
        {
            "code": 7010,
            "message": "Service unavailable"
        }
    ],
    "messages": []
}

{
    "result": null,
    "success": false,
    "errors": [
        {
            "code": 7010,
            "message": "Service unavailable"
        }
    ],
    "messages": []
}

BBrandon Studio Currently stream APIs (including official API and OpenAI compatible API) do not ...

I

Isaac McFadyen•3/31/25, 8:28 PM

Is this for AI Gateway or Workers AI?

I

Isaac McFadyen•3/31/25, 8:28 PM

And what models does this occur on?

N

nclevenger•3/31/25, 9:49 PM

hey - this new model is amazing ... it's SOTA on SWE Bench for a 32b model, scoring 37% which is SHOCKING for a model this small ... would love to see this on Workers AI https://huggingface.co/all-hands/openhands-lm-32b-v0.1

all-hands/openhands-lm-32b-v0.1 · Hugging Face

IIsaac McFadyen Is this for AI Gateway or Workers AI?

B

Brandon Studio•4/1/25, 2:09 AM

Workers AI, behaviors with and without AI Gateway are same

IIsaac McFadyen And what models does this occur on?

B

Brandon Studio•4/1/25, 2:12 AM

I see the two models (

@hf/nousresearch/hermes-2-pro-mistral-7b

@hf/nousresearch/hermes-2-pro-mistral-7b

@hf/nousresearch/hermes-2-pro-mistral-7b

@hf/nousresearch/hermes-2-pro-mistral-7b and

@cf/meta/llama-3.3-70b-instruct-fp8-fast

@cf/meta/llama-3.3-70b-instruct-fp8-fast

@cf/meta/llama-3.3-70b-instruct-fp8-fast

@cf/meta/llama-3.3-70b-instruct-fp8-fast) behaves differently.
Response of hermes does not handle

<tool_call>

<tool_call>

<tool_call>

<tool_call> label in stream mode, as shown in Image 1.
Response of llama 3.3 does not distinguish text content and tool call, as shown in Image 2.

B

Brandon Studio•4/1/25, 2:14 AM

I expect the tool call response to present in

tool_calls

tool_calls

tool_calls

tool_calls field rather than

response

response

response

response field, just like non-stream mode

Ssku I've tried to make requests to several different models today over the REST API ...

G

glncy•4/1/25, 4:52 AM

same i am gettings this error on my local. but when i try to deploy a sample workers ai, it's working.

Original message was deleted

I

Isaac McFadyen•4/1/25, 3:44 PM

That is for Cloudflare Stream which is a different product.

I

Isaac McFadyen•4/1/25, 3:44 PM

I know for sure it accepts WAV 16khz mono, but I'm not sure about other formats.

G

glncy•4/1/25, 6:10 PM

how do i access workers ai locally? i have this nextjs app. using bindings and it returns me service unavailable 7010. even on rest api, it returns me the same response. i've been stuck for two days about this :3

Ssku I've tried to make requests to several different models today over the REST API ...

G

glncy•4/1/25, 7:06 PM

hi @sku want to know if you already figured this out? thanks

Mmichelle this seems to be a bug from an upstream service, we're taking a look

G

glncy•4/1/25, 7:34 PM

hi! thank you for your help. i already sent my account ID to you.

M

mr.niko.la•4/2/25, 8:16 PM

The whisper v3 turbo has 25mb file limit ? Would be nice if the website has this details ……

S

Spencer Hurst•4/3/25, 9:10 PM

Hello. Does anybody have experience using a custom lora on Workers AI? I uploaded the lora files and the finetune is listed via "wrangler ai finetune list", but when I try and do inference via curl with the finetune ID reported by wrangler, it says finetune not found. The adapter works on huggingface with the same files, and I can do inference on the base model (mistral 7b instruct v0.2) via curl fine. Does anybody know the right way to do this?

This is the error I'm getting (all redacted fields filled in properly):

$ curl https://api.cloudflare.com/client/v4/accounts/MY_ACCOUNT_ID/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora \
  -H 'Authorization: Bearer MY_API_TOKEN' \
  -d '{
    "messages": [{"role": "user", "content": "Hello world"}],
    "raw": true,
    "lora": "MY_LORA_ID"
  }'
{"errors":[{"message":"AiError: AiError: Finetune MY_LORA_ID not found (07af68df-18b6-4a5d-bfeb-3453830da3be)","code":3037}],"success":false,"result":{},"
messages":[]}

$ curl https://api.cloudflare.com/client/v4/accounts/MY_ACCOUNT_ID/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora \
  -H 'Authorization: Bearer MY_API_TOKEN' \
  -d '{
    "messages": [{"role": "user", "content": "Hello world"}],
    "raw": true,
    "lora": "MY_LORA_ID"
  }'
{"errors":[{"message":"AiError: AiError: Finetune MY_LORA_ID not found (07af68df-18b6-4a5d-bfeb-3453830da3be)","code":3037}],"success":false,"result":{},"
messages":[]}

$ curl https://api.cloudflare.com/client/v4/accounts/MY_ACCOUNT_ID/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora \
  -H 'Authorization: Bearer MY_API_TOKEN' \
  -d '{
    "messages": [{"role": "user", "content": "Hello world"}],
    "raw": true,
    "lora": "MY_LORA_ID"
  }'
{"errors":[{"message":"AiError: AiError: Finetune MY_LORA_ID not found (07af68df-18b6-4a5d-bfeb-3453830da3be)","code":3037}],"success":false,"result":{},"
messages":[]}

$ curl https://api.cloudflare.com/client/v4/accounts/MY_ACCOUNT_ID/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora \
  -H 'Authorization: Bearer MY_API_TOKEN' \
  -d '{
    "messages": [{"role": "user", "content": "Hello world"}],
    "raw": true,
    "lora": "MY_LORA_ID"
  }'
{"errors":[{"message":"AiError: AiError: Finetune MY_LORA_ID not found (07af68df-18b6-4a5d-bfeb-3453830da3be)","code":3037}],"success":false,"result":{},"
messages":[]}

S

s•4/4/25, 4:57 AM

S

sillah•4/4/25, 10:30 AM

Hello, I have recently deployed cloudfare worker. All seems to work well however I can't seem to set max tokens, it always crops output to 256 tokens. Anyone who has resolved this issue curl -X POST https://xxx \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "user", "content": "explain c++ in detail?"}
],
"max_tokens": 500
}'
Response :
{"inputs":{"messages":[{"role":"user","content":"explain c++ in detail?"}]},"response":{"response":"<think>\nOkay, so I'm trying to understand C++ in detail. I've heard it's a powerful programming language, but I'm not exactly sure how it all fits together. Let me start by breaking down what I know and what I need to figure out.\n\nFirst, I know that C++ is an extension of C. So, it includes everything from C but adds more features. But what exactly are those features? I remember something about classes and objects, which makes me think it's object-oriented. That must mean it's good for building large applications where you can structure code more cleanly.\n\nThe assistant mentioned that C++ is compiled, which I think means it's faster because it's converted directly into machine code. But I'm not entirely sure how that process works compared to interpreted languages. Maybe I should look into the compilation steps, like preprocessing, compiling, assembling, and linking. Understanding that could help me see why C++ is efficient.\n\nI'm also a bit confused about the Standard Template Library (STL). I know it's part of C++, but how does it help? It sounds like it provides a lot of useful data structures and algorithms, which would save time when coding. But I'm not sure how to use it effectively or what the main","usage":{"prompt_tokens":13,"completion_tokens":256,"total_tokens":269}}}

N

nclevenger•4/6/25, 6:43 AM

Will Llama 4 Scout on Workers AI be offered via OpenRouter?

https://openrouter.ai/meta-llama/llama-4-scout
https://developers.cloudflare.com/workers-ai/models/llama-4-scout-17b-16e-instruct/

Llama 4 Scout - API, Providers, Stats

Llama 4 Scout 17B Instruct (16E) is a mixture-of-experts (MoE) language model developed by Meta, activating 17 billion parameters out of a total of 109B. It supports native multimodal input (text and image) and multilingual output (text and code) across 12 supported languages. Run Llama 4 Scout with API

Cloudflare Docs

llama-4-scout-17b-16e-instruct · Cloudflare Workers AI docs

Meta's Llama 4 Scout is a 17 billion parameter model with 16 experts that is natively multimodal. These models leverage a mixture-of-experts architecture to offer industry-leading performance in text and image understanding.

Seems like all the models on Workers AI have such small Context WIndows, is there any that will be a

Similar Threads

Similar Threads

Similar Threads