For LLM inference?

PPhoenixSpace For LLM inference?

Z

zsueo•11/18/25, 1:20 PM

Yeah

Zzsueo Yeah

P

PhoenixSpaceOP•11/18/25, 2:18 PM

Google vertex and Amazon bedrock running the same model is usually a bit cheaper, same with fireworks AI and deepinfra

P

PhoenixSpaceOP•11/18/25, 2:19 PM

Similar in pricing is groq (not to be confused with grok!) but faster output

P

PhoenixSpaceOP•11/18/25, 2:19 PM

This is why I’m stoked about Replicate, I think this is gonna be huge

Original message was deleted

Q

QuentinCody•11/18/25, 2:30 PM

Yes, I'm working on a chatbot+MCP Client that comes pre-loaded with custom MCP Servers I've built

V

Vivek_m4_•11/20/25, 6:06 AM

Hi, I built a huge AI tool using Cloudflare Workers (Spider AI).
I want to discuss potential collaboration or GPU credits.
Whom should I contact from the Cloudflare team?

Bblah Requesting help with my community help request: "Request AI Streaming Binding Ac...

V

Vivek_m4_•11/20/25, 6:19 AM

export default {
async fetch(req, env) {
const model = "@cf/meta/llama-2-7b-chat-int8";

const stream = await env.DOCTALK_AI.run(model, {
prompt: "What is a tomato?",
stream: true,
});

return new Response(stream, {
headers: {
"Content-Type": "text/event-stream",
},
});
},
};
Hello bro this is correct code

V

vcc•11/21/25, 7:37 AM

fetch(`https://api.cloudflare.com/client/v4/accounts/${this.env.CF_ACCOUNT}/ai/run/@cf/deepgram/nova-3?${new URLSearchParams({
            encoding: 'linear16',
            sample_rate: '16000',
                        language:  ?
}).toString()}`)

fetch(`https://api.cloudflare.com/client/v4/accounts/${this.env.CF_ACCOUNT}/ai/run/@cf/deepgram/nova-3?${new URLSearchParams({
            encoding: 'linear16',
            sample_rate: '16000',
                        language:  ?
}).toString()}`)

fetch(`https://api.cloudflare.com/client/v4/accounts/${this.env.CF_ACCOUNT}/ai/run/@cf/deepgram/nova-3?${new URLSearchParams({
            encoding: 'linear16',
            sample_rate: '16000',
                        language:  ?
}).toString()}`)

fetch(`https://api.cloudflare.com/client/v4/accounts/${this.env.CF_ACCOUNT}/ai/run/@cf/deepgram/nova-3?${new URLSearchParams({
            encoding: 'linear16',
            sample_rate: '16000',
                        language:  ?
}).toString()}`)

Can I specify a language for it? and how many languages support it?

VVivek_m4_export default { async fetch(req, env) { const model = "@cf/meta/llama-2-7...

B

blah•11/21/25, 5:43 PM

Thanks Vivek

A

Ahmed bat•11/21/25, 6:09 PM

Image models return 0 bytes but text models work

A

Ahmed bat•11/21/25, 6:10 PM

hey there im trying to use leonardo ai to create images from cloudflare workers ai ... i ve tested it on my n8n workflow it wroks good with other models but with leonardo models i ve got thiserror : Image models return 0 bytes but text models work

C

Catkin038•11/22/25, 10:24 AM

Hello everyone, my Workers AI returned Error 1101, and the reason in the statistics is Uncaught Exception. How should I troubleshoot this?

C

Catkin038•11/22/25, 10:39 AM

I found the problem, 5028: This model was deprecated on 2025-10-01. Please use an alternative model.

C

Catkin038•11/22/25, 10:42 AM

I need to confirm whether @cf/qwen/qwen2.5-coder-32b-instruct still accepts support.

Original message was deleted

C

Catkin038•11/22/25, 10:42 AM

Thank you

C

Catkin038•11/22/25, 10:42 AM

I have left that server because I found it to be new people, not any official

C

Catkin038•11/22/25, 10:46 AM

Why did he deceive me? Even if I mistakenly joined that server, I should have been able to realize it was fake quickly.

C

Catkin038•11/22/25, 10:53 AM

Hello everyone, I've noticed that my Workers AI sometimes encounters the 5028 issue, but querying @cf/qwen/qwen2.5-coder-32b-instruct is still supported. After receiving an email in September, I updated the model referenced in my code. Is it possible that some of Cloudflare's data centers are not using the latest code?

P

plutch•11/22/25, 1:40 PM

Support request for code embedding model SFR-Embedding-Code or CodeSage-Large-v2

R

Razmjoo•11/22/25, 5:09 PM

Hi, I am using the batch api for the Markdown API; I am getting a 50% error rate when converting images to Markdown. I am using 10/batch and 10/concurrent requests. example errors I am getting:

{'name': '123_image_5.png', 'mimeType': 'image/png', 'format': 'error', 'error': 'Error converting image to markdown text: 3040: Capacity temporarily exceeded, please try again.'}

{'name': '123_image_5.png', 'mimeType': 'image/png', 'format': 'error', 'error': 'Error converting image to markdown text: 3040: Capacity temporarily exceeded, please try again.'}

{'name': '123_image_5.png', 'mimeType': 'image/png', 'format': 'error', 'error': 'Error converting image to markdown text: 3040: Capacity temporarily exceeded, please try again.'}

{'name': '123_image_5.png', 'mimeType': 'image/png', 'format': 'error', 'error': 'Error converting image to markdown text: 3040: Capacity temporarily exceeded, please try again.'}

S

shotan•11/23/25, 11:42 PM

Hi, I'm using the OpenAI api for embeddings and getting a lot of 500 errors with the qwen3-embedding-0.6b model. Around 100 requests per minute, 20 inputs per batch, small chunks. Seems ok for 5 minutes then the errors start to pile up. My app handles 3 retries.

B

blah•11/24/25, 3:50 PM

Hey Cloudflare AI team / #workers-ai,
We're hitting a wall with high Time-to-First-Token (TTFT) latency on one of our streaming applications using Workers AI. Need some insight if this is expected for the model we're using.

The Problem:
Our application is experiencing an approx 4.0 second delay before the LLM starts streaming any output.

Our Setup:
Model: @cf/meta/llama-3.1-8b-instruct-fp8
Worker Region: MIA (Miami)
Prompt Size: approx 1,144 input characters (RAG context).Worker Performance:Total Wall Time: 4,474 ms
Worker CPU Time: 16 ms
(Conclusion: Almost all time is spent waiting on the model to start generation, not our code.)

Questions:
Is a 4.0+ second TTFT common for the llama-3.1-8b-instruct model at this prompt length?
Are there specific provisioning tips, or is the *-fast variant recommended solely for reducing this TTFT?

Any advice on further optimizing TTFT on Workers AI is appreciated!

Thanks!

Ssamjs Hey @Fabian . Streaming for the gpt-oss models is not currently supported, but w...

A

acusti•11/24/25, 4:04 PM

just checking in on this. is streaming supported yet for the gpt-oss models? i actually thought until i happened to come across this message that it was supported, at least when using the workers AI binding, based on this message thread: workers-ai

C

Celestial Rose•11/24/25, 9:56 PM

loving the new Models page ! step in the right direction, now we just miss better capabilities !

C

Celestial Rose•11/25/25, 4:47 PM

kinda lost, isnt flux2dev a year old ?

CCelestial Rose kinda lost, isnt flux2dev a year old ?

I

Isaac McFadyen•11/25/25, 8:57 PM

That's FLUX 1

I

Isaac McFadyen•11/25/25, 8:57 PM

This is FLUX 2

C

Celestial Rose•11/25/25, 8:57 PM

so whats https://flux2.io/flux2-model-family/

FLUX.2poloz.igor@gmail.com

FLUX2 Model Family

Black Forest Labs offers FLUX.2 in three distinct variants, each tailored for different use cases.

CCelestial Rose so whats https://flux2.io/flux2-model-family/

I

Isaac McFadyen•11/25/25, 8:59 PM

That's FLUX 2

I

Isaac McFadyen•11/25/25, 8:59 PM

Which was released today.

C

Celestial Rose•11/25/25, 8:59 PM

look at the date

C

Celestial Rose•11/25/25, 8:59 PM

oh sorry its not an official page

I

Isaac McFadyen•11/25/25, 8:59 PM

That is not the official BFL site, and the information is wrong.

C

Celestial Rose•11/25/25, 8:59 PM

wth

C

Celestial Rose•11/25/25, 8:59 PM

dang it

I

Isaac McFadyen•11/25/25, 8:59 PM

Yeah, there's no Flux 2 Schnell

I

Isaac McFadyen•11/25/25, 8:59 PM

You want https://bfl.ai/models/flux-2

C

Celestial Rose•11/25/25, 9:00 PM

awesome thank you

C

Celestial Rose•11/26/25, 5:15 PM

hey team! i dont see a Email channel yet, where can i request an access for the private beta for Cloudflare Email please ?

CCelestial Rose hey team! i dont see a Email channel yet, where can i request an access for the ...

I

Isaac McFadyen•11/26/25, 5:16 PM

https://forms.gle/BX6ECfkar3oVLQxs7

C

Celestial Rose•11/26/25, 5:16 PM

thanks !

C

Celestial Rose•11/26/25, 5:18 PM

sent

S

Sb•11/27/25, 3:38 AM

Can I use different models than the current provided.
Like wan 2.2 or veo 3 or other open source model by paying for the compute or neurons?

S

Sb•11/27/25, 3:41 AM

Cloudflare is already using GPU for other models like flux and llama,
Will they also provide us with the direct usage?

M

muskan•11/27/25, 9:44 AM

Hi! I wanted to confirm something about Cloudflare AI. Does it support parsing PDF files directly?
Right now, it looks like I must extract the text from the PDF myself before sending it to the AI model. Is there any built-in support (or upcoming support) for uploading a PDF and having the AI read/parse it automatically?

A

Adnan A.•11/27/25, 1:23 PM

Hi,

I’m encountering an issue with

Ai-Search (Auto RAG)

Ai-Search (Auto RAG)

Ai-Search (Auto RAG)

Ai-Search (Auto RAG) with folder-based filtering using the

gt

gt

gt

gt,

gte

gte

gte

gte, and

eq

eq

eq

eq filter types. Filtering works as expected when targeting a top-level folder, but fails when specifying a sub-folder.

Below is an example payload that works and returns all files inside

customer-a

customer-a

customer-a

customer-a as well as its sub-folders:

{
  "query": "Cat",
  "filters": {
    "type": "gt",
    "key": "folder",
    "value": "customer-a//"
  }
}

{
  "query": "Cat",
  "filters": {
    "type": "gt",
    "key": "folder",
    "value": "customer-a//"
  }
}

{
  "query": "Cat",
  "filters": {
    "type": "gt",
    "key": "folder",
    "value": "customer-a//"
  }
}

{
  "query": "Cat",
  "filters": {
    "type": "gt",
    "key": "folder",
    "value": "customer-a//"
  }
}

However, the following payloads do not return any results, although the specified sub-folder exists:

{
  "filters": {
    "type": "gt",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "gt",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "gt",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "gt",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "eq",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "eq",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "eq",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "eq",
    "key": "folder",
    "value": "customer-a/sub-folder"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a/sub-folder//"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a/sub-folder//"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a/sub-folder//"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a/sub-folder//"
  }
}

The following variant does work, but only returns items from the folder itself — not from its deeper sub-folders:

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a//sub-folder"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a//sub-folder"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a//sub-folder"
  }
}

{
  "filters": {
    "type": "gte",
    "key": "folder",
    "value": "customer-a//sub-folder"
  }
}

What I need

I would like to search inside a specific sub-folder, ideally including its nested sub-folders, similar to how filtering works for the top-level folder.

Could you please advise on the correct filtering method or confirm whether this is a known limitation or bug?

Thank you in advance for your assistance.

Mmuskan Hi! I wanted to confirm something about Cloudflare AI. Does it support parsing P...

S

Sb•11/27/25, 2:05 PM

Pdf to markdown in cf

Mmuskan Hi! I wanted to confirm something about Cloudflare AI. Does it support parsing P...

B

blah•11/27/25, 3:13 PM

google has stuff like that out of the box that you can work into your pipeline.

M

muskan•11/28/25, 6:01 AM

Hi! Thanks for the replies
I have one follow-up question:

I want to upload the whole PDF file directly to the AI model and get back a clean, readable JSON — without having to parse the PDF myself first.

Basically:
No pre-processing
No pdf-parse / PDF.js / external extraction
Just: Upload PDF → AI extracts → returns structured output

Is there a recommended Cloudflare AI model / workflow that already supports this type of direct PDF→JSON parsing?
A model similar to “document AI” (like Google’s), where you upload a PDF and get structured data back?

Thanks again!

Mmuskan Hi! Thanks for the replies I have one follow-up question: 👉 I want to upload t...

S

Sb•11/28/25, 12:49 PM

Check cloudflare pdf to markdown for worker ai.
It does exactly that.

What I need

Similar Threads

Similar Threads

What I need

Similar Threads