That is just how LLMs work and how every provider does it, not just Workers AI. OpenAI, Gemini, Clau

IIsaac McFadyen That is just how LLMs work and how every provider does it, not just Workers AI. ...

A

Ashkan•10/5/25, 12:14 AM

ok, I will try it. thanks for help

AAshkan Hey, how can I have message history such that ai model will have a message histo...

A

Anon•10/5/25, 2:16 AM

Use Durable Obejcts to store the history

A

Anon•10/5/25, 2:17 AM

At least that's what's recommended you could really store them anywhere

C

Comrade Nalaxone•10/5/25, 4:44 PM

For god sake can you all please update the docs to match the portal this drives me nuts every single time:

https://developers.cloudflare.com/workers-ai/get-started/rest-api/#1-get-api-token-and-account-id

Cloudflare Docs

REST API

Use the Cloudflare Workers AI REST API to deploy a large language model (LLM).

C

Comrade Nalaxone•10/5/25, 4:44 PM

Too big of a company for that to happen

L

LordSilver•10/6/25, 9:21 PM

what is the default model being used?

LLordSilver what is the default model being used?

I

Isaac McFadyenOP•10/6/25, 9:32 PM

There is no default, you pick whatever model on Workers AI you want to use yourself.

IIsaac McFadyen There is no default, you pick whatever model on Workers AI you want to use yours...

L

LordSilver•10/6/25, 9:37 PM

but i didn't pick anything

I

Isaac McFadyenOP•10/6/25, 9:38 PM

If you're using Workers AI you must have? Are you using another product of some sort?

L

LordSilver•10/6/25, 9:38 PM

ok it uses llama by default

IIsaac McFadyen If you're using Workers AI you must have? Are you using another product of some ...

L

LordSilver•10/6/25, 9:38 PM

i use Workes AI and i don't specify any model

LLordSilver i use Workes AI and i don't specify any model

I

Isaac McFadyenOP•10/6/25, 9:39 PM

You must be using a product that uses Workers AI and picks the model for you, there are no default models in Workers AI. I saw in #workers-and-pages-discussions that you're using Stagehand, correct? I expect that picks llama-3.3 for you.

L

LordSilver•10/6/25, 9:40 PM

mmm ok

S

sodepr•10/9/25, 6:15 AM

Hi,

Does Cloudflare support legacy endpoints (

/completions

/completions

/completions

/completions,

/chat/completions

/chat/completions

/chat/completions

/chat/completions) for new models (Eg: gpt-oss-20b)?

The document said that:

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions)

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions)

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions)

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions)

But I cannot use any new model.

A

Ashkan•10/9/25, 9:49 AM

doesn't

runWithTools

runWithTools

runWithTools

runWithTools support

gateway

gateway

gateway

gateway configuration?

Ssodepr Hi, Does Cloudflare support legacy endpoints (`/completions`, `/chat/completion...

K

Kevin•10/9/25, 9:52 AM

Use responses endpoint.

KKevin Use responses endpoint.

S

sodepr•10/9/25, 2:46 PM

Applications hasn't support new endpoint yet.

Ssodepr Applications hasn't support new endpoint yet.

K

Kevin•10/9/25, 2:51 PM

at least ai gateway does support it ai-gateway I have not tried workers ai directly

H

Hew•10/9/25, 7:27 PM

@sodepr The Cloudflare blog said in August that completions support was "coming soon" to the gpt-oss models. I haven't seen any update from them since, though. It would certainly be nice to have because, as you've pointed out, not everything supports the responses endpoint yet.

R

Razmjoo•10/9/25, 8:44 PM

Hi, is there any way to use

@cf/tomarkdown

@cf/tomarkdown

@cf/tomarkdown

@cf/tomarkdown /

@cf/markdown

@cf/markdown

@cf/markdown

@cf/markdown using the AI Gateway?

M

Manqo•10/10/25, 11:27 PM

Hello! I Been looking at worker ai For txt2img. Im quite new with these different Image models, i Have usually just used Gemini or chatgpt Image generation. But to cut down on spending im intrested of using worker ai to generate images. So My question which of The avaivable Image models IS best to create Hand draw pictures? I dont need realism. I tried Flux schell from huggingface but that tend to not create images that Look like drawings.

C

crazyjack12•10/11/25, 11:19 PM

Maybe stable-diffusion-xl-base-1.0

S

Sudhan•10/12/25, 11:56 AM

hey guys i have question. Lets say i have an worker endpoint for processing a audio file. The audio file is 500mb
The endpoint recieves the url and downloads during the endpoint runtime, and then transcribe the audio to text and then vectorize it.. I mean it takes a lot of time cpu compute so is it possible in workers?

MManqo Hello! I Been looking at worker ai For txt2img. Im quite new with these differen...

S

samjs•10/12/25, 6:02 PM

You could try lucid-origin? It generates some nice images in my experience. e.g. here's a short example:

lucidOrigin.get('/', async (c) => {
    // Generate the image
    const response = await c.env.AI.run(
        "@cf/leonardo/lucid-origin",
        {
            prompt: "picture in a hand-drawn style of a mini goldendoodle dog playing fetch in a field"
        }
    );
    console.log(response);

    // Embed the image directly as a data URL
    const imageDataUrl = `data:image/jpeg;base64,${response.image}`;

    const html = `
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>AI Image Generator</title>
    <style>
        // styles
    </style>
</head>
<body>
    <div class="container">
        <h1>🎨 Leonardo AI Image Generator</h1>
        <p>Generating: <em>A mini goldendoodle dog playing fetch in a field (hand-drawn style)</em></p>
        <div class="image-container">
            <img src="${imageDataUrl}" alt="AI Generated Image" />
        </div>
    </div>
</body>
</html>
    `;
    return c.html(html);
});

lucidOrigin.get('/', async (c) => {
    // Generate the image
    const response = await c.env.AI.run(
        "@cf/leonardo/lucid-origin",
        {
            prompt: "picture in a hand-drawn style of a mini goldendoodle dog playing fetch in a field"
        }
    );
    console.log(response);

    // Embed the image directly as a data URL
    const imageDataUrl = `data:image/jpeg;base64,${response.image}`;

    const html = `
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>AI Image Generator</title>
    <style>
        // styles
    </style>
</head>
<body>
    <div class="container">
        <h1>🎨 Leonardo AI Image Generator</h1>
        <p>Generating: <em>A mini goldendoodle dog playing fetch in a field (hand-drawn style)</em></p>
        <div class="image-container">
            <img src="${imageDataUrl}" alt="AI Generated Image" />
        </div>
    </div>
</body>
</html>
    `;
    return c.html(html);
});

lucidOrigin.get('/', async (c) => {
    // Generate the image
    const response = await c.env.AI.run(
        "@cf/leonardo/lucid-origin",
        {
            prompt: "picture in a hand-drawn style of a mini goldendoodle dog playing fetch in a field"
        }
    );
    console.log(response);

    // Embed the image directly as a data URL
    const imageDataUrl = `data:image/jpeg;base64,${response.image}`;

    const html = `
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>AI Image Generator</title>
    <style>
        // styles
    </style>
</head>
<body>
    <div class="container">
        <h1>🎨 Leonardo AI Image Generator</h1>
        <p>Generating: <em>A mini goldendoodle dog playing fetch in a field (hand-drawn style)</em></p>
        <div class="image-container">
            <img src="${imageDataUrl}" alt="AI Generated Image" />
        </div>
    </div>
</body>
</html>
    `;
    return c.html(html);
});

lucidOrigin.get('/', async (c) => {
    // Generate the image
    const response = await c.env.AI.run(
        "@cf/leonardo/lucid-origin",
        {
            prompt: "picture in a hand-drawn style of a mini goldendoodle dog playing fetch in a field"
        }
    );
    console.log(response);

    // Embed the image directly as a data URL
    const imageDataUrl = `data:image/jpeg;base64,${response.image}`;

    const html = `
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>AI Image Generator</title>
    <style>
        // styles
    </style>
</head>
<body>
    <div class="container">
        <h1>🎨 Leonardo AI Image Generator</h1>
        <p>Generating: <em>A mini goldendoodle dog playing fetch in a field (hand-drawn style)</em></p>
        <div class="image-container">
            <img src="${imageDataUrl}" alt="AI Generated Image" />
        </div>
    </div>
</body>
</html>
    `;
    return c.html(html);
});

SSudhan hey guys i have question. Lets say i have an worker endpoint for processing a au...

S

samjs•10/12/25, 6:07 PM

Hey @Sudhan . There are a couple of things that might help here:

1. you can pass in a stream of bytes for the inputs in many of the models, e.g. in the nova-3 example you can fetch and pass in the body directly (without needing to await it: https://developers.cloudflare.com/workers-ai/models/nova-3/)
2. Some models (like nova-3) support our async batch API (https://developers.cloudflare.com/workers-ai/features/batch-api/). So you can submit the job for running the inference and then poll for the response.

In both cases, it shouldn't be using much compute in the worker itself.

J

John Spurlock•10/12/25, 11:10 PM

Does anyone know what the

"p"

"p"

"p"

"p" property represents when streaming results back from

@cf/google/gemma-3-12b-it

@cf/google/gemma-3-12b-it

@cf/google/gemma-3-12b-it

@cf/google/gemma-3-12b-it ? I can't find any docs on it

P

Peps•10/12/25, 11:28 PM

its padding

JJohn Spurlock Does anyone know what the `"p"` property represents when streaming results back ...

P

Peps•10/12/25, 11:28 PM

the reason for the padding is more covered here: https://blog.cloudflare.com/ai-side-channel-attack-mitigated/

The Cloudflare Blog

Mitigating a token-length side-channel attack in our AI products

The Workers AI and AI Gateway team recently collaborated closely with security researchers at Ben Gurion University regarding a report submitted through our Public Bug Bounty program. Through this process, we discovered and fully patched a vulnerability affecting all LLM providers. Here’s the story

PPeps the reason for the padding is more covered here: https://blog.cloudflare.com/ai-...

J

John Spurlock•10/12/25, 11:29 PM

thanks!

Ssamjs Hey @Sudhan . There are a couple of things that might help here: 1. you can pa...

S

Sudhan•10/13/25, 5:05 AM

Thank you for the help

Z

zsueo•10/13/25, 5:24 PM

does workers ai json mode respect patterns in the json schema?

Z

zsueo•10/13/25, 5:25 PM

ie if i include regex will it follow the regex?

Zzsueo does workers ai json mode respect patterns in the json schema?

A

Ashkan•10/13/25, 9:22 PM

I've tried it, the answer is almost Yes but for some models. as far as I tested,

llama-4

llama-4

llama-4

llama-4 and

gpt-oss-120b

gpt-oss-120b

gpt-oss-120b

gpt-oss-120b response in provided JSON schema. But keep it mind that schema Must well defined and emphasize in your prompt that use JSON schema.

AAshkan I've tried it, the answer is almost Yes but for some models. as far as I tested,...

I

Isaac McFadyenOP•10/13/25, 9:37 PM

I think they are specifically asking about regex support not JSON schema support in general

IIsaac McFadyen I think they are specifically asking about regex support not JSON schema support...

A

Ashkan•10/13/25, 9:44 PM

Yea I see, I thought she means pattern of schema but it is pattern of regex. thanks for clarification.

A

antim8•10/14/25, 6:49 PM

are onnx custom models supported / available somehow? Wasm seems supported: https://blog.cloudflare.com/webgpu-in-workers/

The Cloudflare Blog

You can now use WebGPU in Cloudflare Workers

Today, we are introducing WebGPU support to Cloudflare Workers. This blog will explain why it's important, why we did it, how you can use it, and what comes next.

Aantim8 are onnx custom models supported / available somehow? Wasm seems supported: htt...

I

Isaac McFadyenOP•10/14/25, 6:50 PM

Unfortunately WebGPU only ever landed in local dev and is not a thing in production Workers. And no, Workers AI does not currently support custom models.

I

Isaac McFadyenOP•10/14/25, 6:51 PM

https://developers.cloudflare.com/durable-objects/api/webgpu/

A

antim8•10/14/25, 6:51 PM

is it possible to use the onnx through rust workers which are built on top of wasm to my understanding?

I

Isaac McFadyenOP•10/14/25, 6:52 PM

You might be able to? It would be CPU-only and likely quite slow but maybe usable for very small models. Keep in mind that Workers have a size limit of 10mb at deployment time and while you could load in models at runtime you're limited to 128mb of RAM total, shared among all incoming requests to that specific machine.

A

antim8•10/14/25, 6:53 PM

yes, its an incredibly small model, (at least the inference), would provide my trained onnx file through r2 and the inference on a worker

C

Calumk•10/14/25, 8:03 PM

Having a very werid issue with AutoRag (search)

Set it up to index ~ 100 MD files in R2

They reference lots of URLS like https://exampleurl.com/course/901izaxkdas1u4r

The search works really well, but edits ALL of these reference urls, to remove numbers.... the search spits out the urls like https://exampleurl.com/course/izaxkdasur (missing 901__1_4)

Its really consistant. it knows everything in all the MD files, but perfectly edits out all numbers referenced in these urls
(and also edits them out from places where they are not specifically in a url)

B

bosuutap•10/15/25, 5:51 AM

How many requests can do with 10k neurons?

Bbosuutap How many requests can do with 10k neurons?

S

sodepr•10/19/25, 8:40 PM

Depend on model and input/output token

10K neurons ~ 0,11$

If you uses

gpt-oss-120b

gpt-oss-120b

gpt-oss-120b

gpt-oss-120b,
If each request use ~10K input tokens and ~1K output tokens (avg of most AI agent)
=>

10000/1000000*0.35 + 1000/1000000*0.75 = $0.00425/request

10000/1000000*0.35 + 1000/1000000*0.75 = $0.00425/request

10000/1000000*0.35 + 1000/1000000*0.75 = $0.00425/request

10000/1000000*0.35 + 1000/1000000*0.75 = $0.00425/request
=> $0.11 / $0.00425 = ~25 requests

If you just chat, each request spends ~100 input token and ~100 output token
=>

100/1000000*0.35 + 100/1000000*0.75 = $0.00011/request

100/1000000*0.35 + 100/1000000*0.75 = $0.00011/request

100/1000000*0.35 + 100/1000000*0.75 = $0.00011/request

100/1000000*0.35 + 100/1000000*0.75 = $0.00011/request
=> $0.11 / $0.00011 = ~1000 request

K

koalaspider•10/20/25, 2:09 AM

Both my dev and deploy version for AI worker used to work perfectly fine. But last few days, the deploy version Ai worker stop working. Have been trying troubleshoot for the last two days. The dev version still working fine. Anyone know why?

Kkoalaspider Both my dev and deploy version for AI worker used to work perfectly fine. But l...

S

samjs•10/20/25, 9:56 PM

Hey @koalaspider . Can you expand a little on what has stopped working, e.g. is it the inference requests? something else?

B

blindChicken•10/20/25, 10:21 PM

I'm getting the error:

AiError: 3040: Capacity temporarily exceeded, please try again.

AiError: 3040: Capacity temporarily exceeded, please try again.

AiError: 3040: Capacity temporarily exceeded, please try again.

AiError: 3040: Capacity temporarily exceeded, please try again. This appears to happen intermittently, I have had this occur in the past, and it eventually resolves itself. Is there any information as to the cause, how to prevent it and what can be done to resolve it when these errors occur?

BblindChicken I'm getting the error:`AiError: 3040: Capacity temporarily exceeded, please try ...

C

Chaika•10/20/25, 11:44 PM

as far as been said in the past, they just need to scale
When I use worker ai, I get an error.
workers-ai
might be helpful for them if you include the model name here.

CChaika as far as been said in the past, they just need to scale https://discord.com/cha...

B

blindChicken•10/21/25, 12:52 AM

In this case I was calling

"@cf/meta/llama-4-scout-17b-16e-instruct"

"@cf/meta/llama-4-scout-17b-16e-instruct"

"@cf/meta/llama-4-scout-17b-16e-instruct"

"@cf/meta/llama-4-scout-17b-16e-instruct"

That is just how LLMs work and how every provider does it, not just Workers AI. OpenAI, Gemini, Clau

Similar Threads

Similar Threads

Similar Threads