@Cloudflare When to support qwen3

AAiden @Cloudflare When to support qwen3

I

Isaac McFadyen•5/6/25, 2:34 PM

?pings

S

SuperHelpflare•5/6/25, 2:34 PM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

I

Isaac McFadyen•5/6/25, 2:35 PM

That is a bot but the point stands, do not try to ping people please

C

cipherfunk•5/7/25, 1:14 AM

Does the Cloudflare implementation of MeloTTS support any voice other than the default? Any language code I have tried returns the same voice. Is lang: 'EN-AU' the correct formatting?

M

matti•5/8/25, 10:05 AM

Hello! as of today all my local dev queries to cloudflare ai return the following error message, but it works in production. Has anyone noticed a similar thing?

M

matti•5/8/25, 10:05 AM

X [ERROR] InferenceUpstreamError: 10000: Authentication error

at Ai._parseError (cloudflare-internal:ai-api:107:24)
at async Ai.run (cloudflare-internal:ai-api:82:19)

B

blindChicken•5/8/25, 3:36 PM

I'm getting

Error: 8001: Invalid input

Error: 8001: Invalid input

when I try to call tools using

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct

. If I call the model without tools it works fines, but if add the tools array it throws the error. Here is my tools array:

[
    {
        "name":"applyFilter",
        "description":"Function that applies the filter...",
        "parameters":{
            "type":"object",
            "properties":{
                "query":{
                    "type":"object",
                    "description":"The query object..."
                },
                "filterDesc":{
                    "type":"string",
                    "description":"Description, for the user, of the filter..."
                }
            },
            "required":["query","filterDesc"]
        }
    }
]

[
    {
        "name":"applyFilter",
        "description":"Function that applies the filter...",
        "parameters":{
            "type":"object",
            "properties":{
                "query":{
                    "type":"object",
                    "description":"The query object..."
                },
                "filterDesc":{
                    "type":"string",
                    "description":"Description, for the user, of the filter..."
                }
            },
            "required":["query","filterDesc"]
        }
    }
]

[
    {
        "name":"applyFilter",
        "description":"Function that applies the filter...",
        "parameters":{
            "type":"object",
            "properties":{
                "query":{
                    "type":"object",
                    "description":"The query object..."
                },
                "filterDesc":{
                    "type":"string",
                    "description":"Description, for the user, of the filter..."
                }
            },
            "required":["query","filterDesc"]
        }
    }
]

[
    {
        "name":"applyFilter",
        "description":"Function that applies the filter...",
        "parameters":{
            "type":"object",
            "properties":{
                "query":{
                    "type":"object",
                    "description":"The query object..."
                },
                "filterDesc":{
                    "type":"string",
                    "description":"Description, for the user, of the filter..."
                }
            },
            "required":["query","filterDesc"]
        }
    }
]

Here is the call to the model:

await AI.run( model, {  messages: messages, tools: tools });

await AI.run( model, {  messages: messages, tools: tools });

B

blindChicken•5/8/25, 3:59 PM

If I use

@cf/meta/llama-3.3-70b-instruct-fp8-fast

@cf/meta/llama-3.3-70b-instruct-fp8-fast

instead of

llama-4-scout

llama-4-scout

the tool call works, but then the models responds incorrectly. Has there been a change in how tools are call between llama-3 and llama-4? Should I use

Functions

Functions

instead.

B

blindChicken•5/8/25, 4:06 PM

If I change it to call functions instead of tools, I get this cryptic error:

Error: 5006: Error: oneOf at '/' not met, 0 matches: required properties at '/' are 'prompt', Type mismatch of '/messages/0/content', 'array' not in 'string', Type mismatch of '/messages/1/content', 'array' not in 'string', required properties at '/functions/0' are 'name,code'

Error: 5006: Error: oneOf at '/' not met, 0 matches: required properties at '/' are 'prompt', Type mismatch of '/messages/0/content', 'array' not in 'string', Type mismatch of '/messages/1/content', 'array' not in 'string', required properties at '/functions/0' are 'name,code'

Error: 5006: Error: oneOf at '/' not met, 0 matches: required properties at '/' are 'prompt', Type mismatch of '/messages/0/content', 'array' not in 'string', Type mismatch of '/messages/1/content', 'array' not in 'string', required properties at '/functions/0' are 'name,code'

Error: 5006: Error: oneOf at '/' not met, 0 matches: required properties at '/' are 'prompt', Type mismatch of '/messages/0/content', 'array' not in 'string', Type mismatch of '/messages/1/content', 'array' not in 'string', required properties at '/functions/0' are 'name,code'

B

bscofield•5/8/25, 7:12 PM

Howdy! I'm getting an

AiError: 3043: Internal server error

AiError: 3043: Internal server error

on some calls -- has anyone successfully debugged those?

B

blindChicken•5/8/25, 8:18 PM

I found a solution to my problem above, instead of using tool calling, I used

response_format

response_format

and as result I get response as JSON that can be easily parsed and then used to call my functions. It basically provides the same result, with the exception of allowing the model to select the tool, but one could easily set a prop in the JSON for function selection. It still sucks that tool calling appears to be broken.

Bbscofield Howdy! I'm getting an `AiError: 3043: Internal server error` on some calls -- ha...

M

matti•5/8/25, 10:24 PM

I've been getting that too, I think something is off today

Mmatti I've been getting that too, I think something is off today

B

bscofield•5/8/25, 11:05 PM

I got different results when I changed models -- maybe worth a shot for you?

M

matti•5/9/25, 7:56 AM

I know yesterday I wasn't able to develop locally, I was getting auth errors all day long. That seems fixed. And I was also getting some AiError 3043 in prod on one certain flow.

D

dust•5/9/25, 8:14 AM

hi guys. I have an issue fetching the api with 401 error. i fetched with vanilla js. i am not sure this is serve side issue or client side but the browser said the error401 is coming from the server. I ran the python code with the same token and works well. i have allowed the control origin but still error. hope anyone can help.

const API_BASE_URL = "https://api.cloudflare.com/client/v4/accounts/{myID}/ai/run/"
const API_AUTH_TOKEN = "{myTOKEN}" //process.env.API_AUTH_TOKEN;
const model = "@cf/meta/llama-2-7b-chat-int8"
const headers = {
        'Authorization':`Bearer ${API_AUTH_TOKEN}`, 
        //'Content-type':'application/type',
        "Access-Control-Allow-Origin": "*",
        "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
        "Access-Control-Max-Age": "86400"
    }

    if (!API_BASE_URL || !API_AUTH_TOKEN) {
        throw new Error('API credential is wrong or not configured from Github Action')
    }

    const inputs = [
        {'role':'system', 'content':systemPrompt},
        {'role':'user', 'content':userPrompt},
    ]

    const payload = {
        message: inputs
    }

    try {
        console.log("Requesting to LLM...")
        const response = await fetch(`${API_BASE_URL}${model}`, {
            method: 'POST', 
            headers:headers, 
            body: JSON.stringify(payload), 
            mode: 'no-cors'
        }
        );

        if (!response.ok) {
            throw new Error (`Error request from LLM: ${response.status}`)
        }
        console.log("Requesting completed. Waiting for output...")
        const output = await response.json(); // output
        console.log(output)
    }
    catch (error) {
        console.log("API Error", error);
        throw error;
    }

const API_BASE_URL = "https://api.cloudflare.com/client/v4/accounts/{myID}/ai/run/"
const API_AUTH_TOKEN = "{myTOKEN}" //process.env.API_AUTH_TOKEN;
const model = "@cf/meta/llama-2-7b-chat-int8"
const headers = {
        'Authorization':`Bearer ${API_AUTH_TOKEN}`, 
        //'Content-type':'application/type',
        "Access-Control-Allow-Origin": "*",
        "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
        "Access-Control-Max-Age": "86400"
    }

    if (!API_BASE_URL || !API_AUTH_TOKEN) {
        throw new Error('API credential is wrong or not configured from Github Action')
    }

    const inputs = [
        {'role':'system', 'content':systemPrompt},
        {'role':'user', 'content':userPrompt},
    ]

    const payload = {
        message: inputs
    }

    try {
        console.log("Requesting to LLM...")
        const response = await fetch(`${API_BASE_URL}${model}`, {
            method: 'POST', 
            headers:headers, 
            body: JSON.stringify(payload), 
            mode: 'no-cors'
        }
        );

        if (!response.ok) {
            throw new Error (`Error request from LLM: ${response.status}`)
        }
        console.log("Requesting completed. Waiting for output...")
        const output = await response.json(); // output
        console.log(output)
    }
    catch (error) {
        console.log("API Error", error);
        throw error;
    }

const API_BASE_URL = "https://api.cloudflare.com/client/v4/accounts/{myID}/ai/run/"
const API_AUTH_TOKEN = "{myTOKEN}" //process.env.API_AUTH_TOKEN;
const model = "@cf/meta/llama-2-7b-chat-int8"
const headers = {
        'Authorization':`Bearer ${API_AUTH_TOKEN}`, 
        //'Content-type':'application/type',
        "Access-Control-Allow-Origin": "*",
        "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
        "Access-Control-Max-Age": "86400"
    }

    if (!API_BASE_URL || !API_AUTH_TOKEN) {
        throw new Error('API credential is wrong or not configured from Github Action')
    }

    const inputs = [
        {'role':'system', 'content':systemPrompt},
        {'role':'user', 'content':userPrompt},
    ]

    const payload = {
        message: inputs
    }

    try {
        console.log("Requesting to LLM...")
        const response = await fetch(`${API_BASE_URL}${model}`, {
            method: 'POST', 
            headers:headers, 
            body: JSON.stringify(payload), 
            mode: 'no-cors'
        }
        );

        if (!response.ok) {
            throw new Error (`Error request from LLM: ${response.status}`)
        }
        console.log("Requesting completed. Waiting for output...")
        const output = await response.json(); // output
        console.log(output)
    }
    catch (error) {
        console.log("API Error", error);
        throw error;
    }

const API_BASE_URL = "https://api.cloudflare.com/client/v4/accounts/{myID}/ai/run/"
const API_AUTH_TOKEN = "{myTOKEN}" //process.env.API_AUTH_TOKEN;
const model = "@cf/meta/llama-2-7b-chat-int8"
const headers = {
        'Authorization':`Bearer ${API_AUTH_TOKEN}`, 
        //'Content-type':'application/type',
        "Access-Control-Allow-Origin": "*",
        "Access-Control-Allow-Methods": "GET,HEAD,POST,OPTIONS",
        "Access-Control-Max-Age": "86400"
    }

    if (!API_BASE_URL || !API_AUTH_TOKEN) {
        throw new Error('API credential is wrong or not configured from Github Action')
    }

    const inputs = [
        {'role':'system', 'content':systemPrompt},
        {'role':'user', 'content':userPrompt},
    ]

    const payload = {
        message: inputs
    }

    try {
        console.log("Requesting to LLM...")
        const response = await fetch(`${API_BASE_URL}${model}`, {
            method: 'POST', 
            headers:headers, 
            body: JSON.stringify(payload), 
            mode: 'no-cors'
        }
        );

        if (!response.ok) {
            throw new Error (`Error request from LLM: ${response.status}`)
        }
        console.log("Requesting completed. Waiting for output...")
        const output = await response.json(); // output
        console.log(output)
    }
    catch (error) {
        console.log("API Error", error);
        throw error;
    }

A

ajgeiss0702•5/9/25, 7:24 PM

has anyone had any success using custom LoRAs recently? Last time i tried they would just fail with a generic error for seemingly no reason

M

matti•5/10/25, 10:51 AM

Has anyone used Qwen qwq-32b : https://developers.cloudflare.com/workers-ai/models/qwq-32b/

M

matti•5/10/25, 10:52 AM

it always sends me back his thinking

M

matti•5/10/25, 10:52 AM

"I've been asked hello by the user, I think I should say hello back"

M

matti•5/10/25, 10:53 AM

is there a good way to filter it out ? (maybe just the first text until the first \n)

M

matti•5/10/25, 11:47 AM

(found the answer myself, I can remove things on the stream until the </think> marker).

A

absolutejam•5/10/25, 7:41 PM

Anyone know how to get the text-to-img models working with the Go cloudflare client? I've tried

@cf/bytedance/stable-diffusion-xl-lightning

@cf/bytedance/stable-diffusion-xl-lightning

and

@cf/black-forest-labs/flux-1-schnell

@cf/black-forest-labs/flux-1-schnell

but both produce the error:

expected destination type of 'string' or '[]byte' for responses with content-type 'image/png' that is not 'application/json'

Which is because they're returning bytes instead of a JSON object (tested with cURL).

resp, err := client.AI.Run(ctx, model, ai.AIRunParams{
    AccountID: cloudflare.F(cfAccountID),
    Body: ai.AIRunParamsBodyTextToImage{
        Prompt: cloudflare.F(prompt),
    },
})
if err != nil {
    return err
}

resp, err := client.AI.Run(ctx, model, ai.AIRunParams{
    AccountID: cloudflare.F(cfAccountID),
    Body: ai.AIRunParamsBodyTextToImage{
        Prompt: cloudflare.F(prompt),
    },
})
if err != nil {
    return err
}

resp, err := client.AI.Run(ctx, model, ai.AIRunParams{
    AccountID: cloudflare.F(cfAccountID),
    Body: ai.AIRunParamsBodyTextToImage{
        Prompt: cloudflare.F(prompt),
    },
})
if err != nil {
    return err
}

resp, err := client.AI.Run(ctx, model, ai.AIRunParams{
    AccountID: cloudflare.F(cfAccountID),
    Body: ai.AIRunParamsBodyTextToImage{
        Prompt: cloudflare.F(prompt),
    },
})
if err != nil {
    return err
}

A

absolutejam•5/10/25, 7:54 PM

Or should I report this as an issue?

C

cipherfunk•5/11/25, 5:52 PM

When will we get a video generation model? Does Cloudflare have enough GPUs to support this?

G

gabrielstuff•5/12/25, 11:13 AM

Hello! I hope you’re doing well. I’m experimenting with the Cloudflare AI-Worker and attempting to parallelize my AI calls, but I keep encountering the following error:

AiError: 3040: Capacity temporarily exceeded, please try again.

AiError: 3040: Capacity temporarily exceeded, please try again.

Is this expected behavior when scaling up request volume? If so, are there recommended strategies or configuration tweaks to improve throughput?

Any guidance would be greatly appreciated—thank you!

G

gabrielstuff•5/12/25, 11:56 AM

here is an example :

{
exception:
 
{
stack:
 
"AiError: 3040: Capacity temporarily exceeded, please try again. at Ai._parseError (cloudflare-internal:ai-api:102:24) at async Ai.run (cloudflare-internal:ai-api:82:19) at async Object.fetch (index.js:21:22)",
name:
 
"Error",
message:
 
"3040: Capacity temporarily exceeded, please try again.",
timestamp:
 
1747050909290,
},
message:
 
"3040: Capacity temporarily exceeded, please try again.",
$workers:
 
{
truncated:
 
false,
event:
 
{
request:
 
{
url:
 
"https://classify-ai.workers.dev/",
method:
 
"POST",
path:
 
"/"
}
},
outcome:
 
"exception",
scriptName:
 
"classify-ai",
eventType:
 
"fetch",
executionModel:
 
"stateless",
scriptVersion:
 
{
id:
 
"1678a794-81e9-48fd-8d8f-43f9d95d4235"
},
requestId:
 
"93e9b9b6a8b802ad"
},
$metadata:
 
{
id:
 
"01JV25JKKAQGM3BZG6KQDG9AA7",
requestId:
 
"93e9b9b6a8b802ad",
trigger:
 
"POST /",
service:
 
"classify-ai",
level:
 
"error",
error:
 
"3040: Capacity temporarily exceeded, please try again.",
message:
 
"3040: Capacity temporarily exceeded, please try again.",
account:
 
"f1c3db27dc01771c8c27ec175b404598",
type:
 
"cf-worker",
fingerprint:
 
"4bc40a10bc8b1073f4883173eeb61c8a",
origin:
 
"fetch",
messageTemplate:
 
"<NUM>: Capacity temporarily exceeded, please try again.",
errorTemplate:
 
"<NUM>: Capacity temporarily exceeded, please try again.",
}
}

{
exception:
 
{
stack:
 
"AiError: 3040: Capacity temporarily exceeded, please try again. at Ai._parseError (cloudflare-internal:ai-api:102:24) at async Ai.run (cloudflare-internal:ai-api:82:19) at async Object.fetch (index.js:21:22)",
name:
 
"Error",
message:
 
"3040: Capacity temporarily exceeded, please try again.",
timestamp:
 
1747050909290,
},
message:
 
"3040: Capacity temporarily exceeded, please try again.",
$workers:
 
{
truncated:
 
false,
event:
 
{
request:
 
{
url:
 
"https://classify-ai.workers.dev/",
method:
 
"POST",
path:
 
"/"
}
},
outcome:
 
"exception",
scriptName:
 
"classify-ai",
eventType:
 
"fetch",
executionModel:
 
"stateless",
scriptVersion:
 
{
id:
 
"1678a794-81e9-48fd-8d8f-43f9d95d4235"
},
requestId:
 
"93e9b9b6a8b802ad"
},
$metadata:
 
{
id:
 
"01JV25JKKAQGM3BZG6KQDG9AA7",
requestId:
 
"93e9b9b6a8b802ad",
trigger:
 
"POST /",
service:
 
"classify-ai",
level:
 
"error",
error:
 
"3040: Capacity temporarily exceeded, please try again.",
message:
 
"3040: Capacity temporarily exceeded, please try again.",
account:
 
"f1c3db27dc01771c8c27ec175b404598",
type:
 
"cf-worker",
fingerprint:
 
"4bc40a10bc8b1073f4883173eeb61c8a",
origin:
 
"fetch",
messageTemplate:
 
"<NUM>: Capacity temporarily exceeded, please try again.",
errorTemplate:
 
"<NUM>: Capacity temporarily exceeded, please try again.",
}
}

{
exception:
 
{
stack:
 
"AiError: 3040: Capacity temporarily exceeded, please try again. at Ai._parseError (cloudflare-internal:ai-api:102:24) at async Ai.run (cloudflare-internal:ai-api:82:19) at async Object.fetch (index.js:21:22)",
name:
 
"Error",
message:
 
"3040: Capacity temporarily exceeded, please try again.",
timestamp:
 
1747050909290,
},
message:
 
"3040: Capacity temporarily exceeded, please try again.",
$workers:
 
{
truncated:
 
false,
event:
 
{
request:
 
{
url:
 
"https://classify-ai.workers.dev/",
method:
 
"POST",
path:
 
"/"
}
},
outcome:
 
"exception",
scriptName:
 
"classify-ai",
eventType:
 
"fetch",
executionModel:
 
"stateless",
scriptVersion:
 
{
id:
 
"1678a794-81e9-48fd-8d8f-43f9d95d4235"
},
requestId:
 
"93e9b9b6a8b802ad"
},
$metadata:
 
{
id:
 
"01JV25JKKAQGM3BZG6KQDG9AA7",
requestId:
 
"93e9b9b6a8b802ad",
trigger:
 
"POST /",
service:
 
"classify-ai",
level:
 
"error",
error:
 
"3040: Capacity temporarily exceeded, please try again.",
message:
 
"3040: Capacity temporarily exceeded, please try again.",
account:
 
"f1c3db27dc01771c8c27ec175b404598",
type:
 
"cf-worker",
fingerprint:
 
"4bc40a10bc8b1073f4883173eeb61c8a",
origin:
 
"fetch",
messageTemplate:
 
"<NUM>: Capacity temporarily exceeded, please try again.",
errorTemplate:
 
"<NUM>: Capacity temporarily exceeded, please try again.",
}
}

{
exception:
 
{
stack:
 
"AiError: 3040: Capacity temporarily exceeded, please try again. at Ai._parseError (cloudflare-internal:ai-api:102:24) at async Ai.run (cloudflare-internal:ai-api:82:19) at async Object.fetch (index.js:21:22)",
name:
 
"Error",
message:
 
"3040: Capacity temporarily exceeded, please try again.",
timestamp:
 
1747050909290,
},
message:
 
"3040: Capacity temporarily exceeded, please try again.",
$workers:
 
{
truncated:
 
false,
event:
 
{
request:
 
{
url:
 
"https://classify-ai.workers.dev/",
method:
 
"POST",
path:
 
"/"
}
},
outcome:
 
"exception",
scriptName:
 
"classify-ai",
eventType:
 
"fetch",
executionModel:
 
"stateless",
scriptVersion:
 
{
id:
 
"1678a794-81e9-48fd-8d8f-43f9d95d4235"
},
requestId:
 
"93e9b9b6a8b802ad"
},
$metadata:
 
{
id:
 
"01JV25JKKAQGM3BZG6KQDG9AA7",
requestId:
 
"93e9b9b6a8b802ad",
trigger:
 
"POST /",
service:
 
"classify-ai",
level:
 
"error",
error:
 
"3040: Capacity temporarily exceeded, please try again.",
message:
 
"3040: Capacity temporarily exceeded, please try again.",
account:
 
"f1c3db27dc01771c8c27ec175b404598",
type:
 
"cf-worker",
fingerprint:
 
"4bc40a10bc8b1073f4883173eeb61c8a",
origin:
 
"fetch",
messageTemplate:
 
"<NUM>: Capacity temporarily exceeded, please try again.",
errorTemplate:
 
"<NUM>: Capacity temporarily exceeded, please try again.",
}
}

G

gabrielstuff•5/12/25, 12:26 PM

opened an isue : https://community.cloudflare.com/t/gemma-aierror-3040-capacity-temporarily-exceeded-please-try-again/797352

J

John Guthrie•5/12/25, 3:31 PM

Hi Cloudflare community,

I'm running into an issue while trying to create a new AutoRAG pipeline through the Cloudflare dashboard. I've successfully gone through the steps:

Connect your data source
Generate indexes
Configure retrieval and generation
However, when I reach the final step (4. Finalize), provide a name for the RAG, and choose "Create one for me" for the Service API Token (it generated AutoRAG Token - 2025-05-06), and then click the "Create" button, I immediately get the following error message:

D1 TYPE ERROR: Type 'object' not supported for value '[object Object]'

It seems like something during the finalization process is trying to pass a JavaScript object [object Object] directly into a D1 database operation where it's not supported.

Has anyone else encountered this specific D1 type error when setting up AutoRAG? Any ideas what configuration step or data aspect might be causing this, or how I can further debug it?

Thanks for any help or pointers!

B

blindChicken•5/12/25, 4:49 PM

I just read this post on CF-Blog about prefix caching: https://blog.cloudflare.com/workers-ai-improvements/#what-is-prefix-caching

The Cloudflare Blog

Workers AI gets a speed boost, batch workload support, more LoRAs, ...

We just made Workers AI inference faster with speculative decoding & prefix caching. Use our new batch inference for handling large request volumes seamlessly. Build tailored AI apps with more LoRA options. Lastly, new models and a refreshed dashboard round out this Developer Week update for Workers AI.

B

blindChicken•5/12/25, 4:50 PM

Is there any detailed info on how to implement / manage this?

A

ajgeiss0702•5/12/25, 8:44 PM

is it not possible to reliably use tools with streaming?

I've been messing with it, and it seems incosistant.

When no tools are specified and streaming is enabled, each chunk includes a

"tool_calls":[]

"tool_calls":[]

object.

When I keep streaming enabled and add a tool to the request, the

"tool_calls":[]

"tool_calls":[]

goes away and instead some json is output is streamed in the

response

response

string.
And that json isnt consistant either. Sometimes it includes

"type": "function"

"type": "function"

and sometimes not.

A

acusti•5/13/25, 3:58 PM

update: this was not actually an error from cloudflare. my assessment of the issue was incorrect, so the following is not accurate.

i’ve been using

@cf/black-forest-labs/flux-1-schnell

@cf/black-forest-labs/flux-1-schnell

via the workers AI binding for text-to-image, which recently stopped working for me, giving me this response:

Monthly usage limit for image reached for your plan. Please upgrade.

Monthly usage limit for image reached for your plan. Please upgrade.

i couldn’t find any mention of monthly limits in the Limits doc or the Pricing doc. what’s the best way to increase my monthly usage limit for flux schnell?

note: i’m currently on the Workers Paid plan

J

joesepi•5/13/25, 4:38 PM

How does Cloudflare choose the models it makes available on the Cloudflare Workers AI platform? I'd like to try using the open source IBM Granite models but they are not listed. (Disclosure: I work at IBM) Thanks!!!

B

blindChicken•5/13/25, 7:02 PM

I've been getting some strange responses intermittently. The responses have a bunch spaces and line feed characters after an otherwise fine response. I get this a few times in a row and then it stops for while. I'm using Qwen

@cf/qwen/qwen2.5-coder-32b-instruct

@cf/qwen/qwen2.5-coder-32b-instruct

. This really sucks when your paying for tokens, a couple of hundred token for nothing per request can start to add up.

{{{{ ...the expected response... } } ] } } } \n' +
    '\n' +
    '   \n' +
    '  \n' +
    '  \n' +
    '  \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    ' ',

{{{{ ...the expected response... } } ] } } } \n' +
    '\n' +
    '   \n' +
    '  \n' +
    '  \n' +
    '  \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    ' ',

{{{{ ...the expected response... } } ] } } } \n' +
    '\n' +
    '   \n' +
    '  \n' +
    '  \n' +
    '  \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    ' ',

{{{{ ...the expected response... } } ] } } } \n' +
    '\n' +
    '   \n' +
    '  \n' +
    '  \n' +
    '  \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    '    \n' +
    '  \n' +
    ' ',

I

Ian•5/14/25, 12:17 AM

does cloudflare still support onnx custom models ?

I

Isaac McFadyen•5/14/25, 12:43 AM

?workers-ai-models TLDR: No

S

SuperHelpflare•5/14/25, 12:43 AM

Workers AI currently only supports popular open-source models provided by the Cloudflare team, as well as your own LoRAs that can be applied on top of the Cloudflare-provided models. You cannot currently upload your own models or use a model from HuggingFace. See the documentation for the list of Cloudflare-provided models: https://developers.cloudflare.com/workers-ai/models/

Ggabrielstuff Hello! I hope you’re doing well. I’m experimenting with the Cloudflare AI-Worker...

G

gabrielstuff•5/15/25, 9:19 AM

Hello, any news on this topic ?

G

gabrielstuff•5/15/25, 9:20 AM

I'm a paying customer and having no response from either the support or someone here, is a bit worrying.

Ggabrielstuff Hello! I hope you’re doing well. I’m experimenting with the Cloudflare AI-Worker...

A

ajgeiss0702•5/15/25, 6:42 PM

yeah, that happens sometimes. just re-try.
if its something not time critical, you can use the batch api to avoid exceeding cf's capacity

Cloudflare Docs

Asynchronous Batch API

Asynchronous batch processing lets you send a collection (batch) of inference requests in a single call. Instead of expecting immediate responses for every request, the system queues them for processing and returns the results later.

Aajgeiss0702 yeah, that happens sometimes. just re-try. if its something not time critical, y...

G

gabrielstuff•5/15/25, 7:12 PM

Ok lets retry but I don’t get how’s that supposed to scale with with millions of users …

F

felix_m•5/15/25, 9:00 PM

Hi! Any plans to have an image embedding model available?
I'd love to move my CLIP inferences to CF. That model has 31M downloads last month - it's pretty popular.
https://huggingface.co/openai/clip-vit-large-patch14

openai/clip-vit-large-patch14 · Hugging Face

Ggabrielstuff Ok lets retry but I don’t get how’s that supposed to scale with with millions of...

A

ajgeiss0702•5/15/25, 11:18 PM

in my experience, more than 1 retry a second later has never been needed, but I don't have millions of users

✎

✎ℳ㎕Joy485@新歲錦時•5/16/25, 2:18 PM

I made an chat AI worker project
Recommended: https://github.com/qyjoy/Gemini-CFworker

GitHub

GitHub - qyjoy/Gemini-CFworker: 🌐 A simple Cloudflare Worker - S...

A simple Cloudflare Worker - Serverless &amp; Cost-Effective: No need to rent or manage your own server. Cloudflare Workers operate on a serverless model, and the free tier is often sufficien...

J

JZ•5/17/25, 7:35 AM

Anyone successfully run the example of MCP with auth in this doc?
https://developers.cloudflare.com/agents/guides/remote-mcp-server/

I keep getting error below:

⎔ Starting local server...
[wrangler:inf] Ready on http://localhost:8788
✘ [ERROR] OAuth error response: 401 invalid_token - Missing or invalid access token


[wrangler:inf] POST /sse 401 Unauthorized (6ms)
[wrangler:inf] GET /.well-known/oauth-authorization-server 200 OK (1ms)
[wrangler:inf] GET /authorize 302 Found (68ms)
[wrangler:inf] GET /callback 302 Found (871ms)
[wrangler:inf] GET /.well-known/oauth-authorization-server 200 OK (2ms)
✘ [ERROR] OAuth error response: 401 invalid_client - Client not found


[wrangler:inf] POST /token 401 Unauthorized (3ms)

⎔ Starting local server...
[wrangler:inf] Ready on http://localhost:8788
✘ [ERROR] OAuth error response: 401 invalid_token - Missing or invalid access token


[wrangler:inf] POST /sse 401 Unauthorized (6ms)
[wrangler:inf] GET /.well-known/oauth-authorization-server 200 OK (1ms)
[wrangler:inf] GET /authorize 302 Found (68ms)
[wrangler:inf] GET /callback 302 Found (871ms)
[wrangler:inf] GET /.well-known/oauth-authorization-server 200 OK (2ms)
✘ [ERROR] OAuth error response: 401 invalid_client - Client not found


[wrangler:inf] POST /token 401 Unauthorized (3ms)

⎔ Starting local server...
[wrangler:inf] Ready on http://localhost:8788
✘ [ERROR] OAuth error response: 401 invalid_token - Missing or invalid access token


[wrangler:inf] POST /sse 401 Unauthorized (6ms)
[wrangler:inf] GET /.well-known/oauth-authorization-server 200 OK (1ms)
[wrangler:inf] GET /authorize 302 Found (68ms)
[wrangler:inf] GET /callback 302 Found (871ms)
[wrangler:inf] GET /.well-known/oauth-authorization-server 200 OK (2ms)
✘ [ERROR] OAuth error response: 401 invalid_client - Client not found


[wrangler:inf] POST /token 401 Unauthorized (3ms)

⎔ Starting local server...
[wrangler:inf] Ready on http://localhost:8788
✘ [ERROR] OAuth error response: 401 invalid_token - Missing or invalid access token


[wrangler:inf] POST /sse 401 Unauthorized (6ms)
[wrangler:inf] GET /.well-known/oauth-authorization-server 200 OK (1ms)
[wrangler:inf] GET /authorize 302 Found (68ms)
[wrangler:inf] GET /callback 302 Found (871ms)
[wrangler:inf] GET /.well-known/oauth-authorization-server 200 OK (2ms)
✘ [ERROR] OAuth error response: 401 invalid_client - Client not found


[wrangler:inf] POST /token 401 Unauthorized (3ms)

Cloudflare Docs

Build a Remote MCP server

This guide will show you how to deploy your own remote MCP server on Cloudflare, with two options:

JJZ Anyone successfully run the example of MCP with auth in this doc? https://devel...

I

Isaac McFadyen•5/17/25, 12:52 PM

?crossposting

S

SuperHelpflare•5/17/25, 12:52 PM

Please do not post your question in multiple channels/post it multiple times per the rules at #

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

Similar Threads

Similar Threads

Similar Threads