will gpt-oss-120b support stream ?

Aakazwz will gpt-oss-120b support stream ?

Chaika•8/7/25, 3:37 PM

It already does support streaming via openai's new response format streaming:
https://platform.openai.com/docs/guides/streaming-responses?api-mode=responses

akazwzOP•8/7/25, 5:00 PM

I was using the response api. without stream param, it works, with stream: true, server error, return 500

akazwzOP•8/7/25, 5:00 PM

Aakazwz I was using the response api. without stream param, it works, with stream: true,...

Chaika•8/7/25, 5:20 PM

Have to use Workers binding for streaming

Chaika•8/7/25, 5:23 PM

I don't think the direct api has supported streaming for any model? Not 100% sure as I don't use the api myself, but yea that same request via binding streaming looks fine

Dev2xl•8/9/25, 10:35 PM

Hello,
I’m exploring options for running inference on private TensorFlow models with Cloudflare’s products. Is it possible to achieve this using Workers or any other Cloudflare service? Any technical guidance or references would be greatly appreciated. Thank you!

Isaac McFadyen•8/10/25, 4:16 PM

You could try Containers but they're currently ill-suited for most ML things as they have fairly low resource allocation and no GPUs.

ixixix•8/12/25, 6:44 AM

I know the docs don't mention it, but just to confirm, the /audio/transcriptions endpoint isn't supported as an openai compatible endpoint in Workers AI or AI Gateway for the cloudflare whisper models, right? I'm getting

{"errors":[{"code":7000,"message":"No route for that URI"}]}

{"errors":[{"code":7000,"message":"No route for that URI"}]}

and

{"code":2002,"message":"Internal server error"}

{"code":2002,"message":"Internal server error"}

tomerzcod•8/13/25, 7:29 AM

Hey, i've been following the cloudflare docs, I have a worker with ai bindings set I'm logged in but whatever i do I get the 10000: authentication error.

What am I missing? can't find any info how to fix this

Nhat Bui•8/14/25, 1:42 AM

Hey guys, I don't know why this prompt is detected as NSFW content

xugik•8/14/25, 11:51 AM

Hi, does anyone happen to know when the Chat Completions API will come to the gpt-oss models?

Bryan•8/15/25, 4:38 AM

Does anyone have advice for getting worker-ai function calling models to work with traditional tool calls, while using the open-ai compatibility layer *and * streaming?
I can't get any model to both send back tool recommendations in the correct format and summarize a tool call.

Mistral keeps recommending tools via XML in regular text content, but when given a chat history where the most recent message is a tool execution, it will correctly summarize the tool execution.
llama-4-scout will correctly recommended a tool, but when given a chat history where the most recent message is a tool execution, it will recommend the tool again instead of summarizing.
The other 2 function calling models aren't any better.

If i turn off streaming, Mistral at least works correctly, but streaming is strongly preferred.

Does anyone know what I'm missing?

AAhmed is there a way to get sparse embeddings from `@cf/baai/bge-m3`

Nipsuli•8/15/25, 11:15 PM

yeah, this would be nice

SmallShen•8/16/25, 10:58 PM

can we have qwen embedding model in worker ai?

mdanowar2•8/19/25, 12:30 PM

scotto•8/19/25, 2:12 PM

openai model dont work with chat/completion API, that can be fixed? from model change standpoint it's good to have the option to switch only changing the string name of the model

DonJonko•8/19/25, 3:49 PM

Heya pretty people! Im looking for a way to set a 'spending limit' on my worker ai usage. Does that exist ?

RS•8/20/25, 7:13 AM

Can we have https://github.com/QwenLM/Qwen-Image models for image generation? The current available models are outdated.

GitHub

GitHub - QwenLM/Qwen-Image: Qwen-Image is a powerful image generati...

Qwen-Image is a powerful image generation foundation model capable of complex text rendering and precise image editing. - QwenLM/Qwen-Image

MichaelC•8/20/25, 11:29 AM

How to use the current image generation models that CF offers? So far in my tests I have good results with https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/. However there is no dimension control which renders it pretty much unusable for what I want to offer. Then I dont see any other upscaling models available either unfortunately.

Cloudflare Docs

flux-1-schnell

FLUX.1 [schnell] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions.

Zzsueo hey im noticing that sometimes the ai does not call the tool properly leading to...

ayad99•8/20/25, 2:02 PM

hey i noticed that too, specially with the llama-4-scout model. did you find any solution or workaround?

Horiwix•8/20/25, 6:35 PM

Is there any AI model available that has internet access and could search for realtime/live data? I don't think so, couldn't find one, but asking just in case

HHoriwix Is there any AI model available that has internet access and could search for re...

Isaac McFadyen•8/20/25, 7:58 PM

Not really, Workers AI is more low-level and if you wanted an LLM to do this you'd need to orchestrate yourself or use Sandboxes: https://developers.cloudflare.com/changelog/2025-06-24-announcing-sandboxes/

yinxingmaiming6409•8/21/25, 2:11 AM

How can AI be used in Cherry Studio? It seems impossible.

Yyinxingmaiming6409 How can AI be used in Cherry Studio? It seems impossible.

xugik•8/21/25, 7:17 AM

https://youtu.be/3XOJvW3einQ?si=BmuUEasTmeHRoCgp&t=447 I followed the guide here and was able to get the api key and Account ID to use it in Cherry Studio

YouTubeQuorify

200+ FREE AI Model APIs You Can Use RIGHT NOW! (Unlimited Access)

Are you a developer, student, or AI enthusiast tired of hitting paywalls and restrictive rate limits? In this ultimate 2025 guide, we reveal the secrets to accessing the world's most powerful AI models—including GPT-4, Llama 3, and Gemini—for FREE. We go end-to-end, showing you exactly how to sign up, get your API keys, and make your first c...

Aayad99 hey i noticed that too, specially with the llama-4-scout model. did you find any...

zsueo•8/23/25, 2:48 PM

no unfortnately, the best i can do is modify the system prompt to minimize it happening

Himanshu•8/25/25, 3:47 PM

Is there an ETA for function calling support for gpt oss models in workers ai?

yureckey•8/27/25, 9:29 AM

I see several new models introduced:
aura-1 Text-to-Speech
lucid-origin Text-to-Image
phoenix-1.0 Text-to-Image
when will the docs be updated with parameters of these models?

James•8/27/25, 11:21 AM

given its currently AI week at cloudflare and with the timing of those releases, I'd wait for today's announcements for more info there

JJames given its currently AI week at cloudflare and with the timing of those releases,...

Mirza•8/27/25, 1:05 PM

Hey James, I believe you're from the cloudflare team.

I wanted to know why are no claude models in cloudflare-ai ? It would help me so much by not depending on any external services than cloudflare for the whole platform (my app).

Do you think I can bring this up in the office hours today ? I never attended those but I feel there's a huge need for the current leading models in the models we have, it would help in many ways.

Thanks.

MMirza Hey James, I believe you're from the cloudflare team. I wanted to know why are...

James•8/27/25, 1:09 PM

I'm not on the Cloudflare team, just a community champ/mvp

I'm not sure there's any office hours today, especially not with the WAI team, but I know the want for more flagship code models has been talked about a lot recently with the current selection being a little lacklustre.

James•8/27/25, 1:10 PM

The claude models aren't freely available, which is why I imagine we've not seen them on WAI. But even something like Kimi would be a huge step up over the current selection, I entirely agree.

JJames I'm not on the Cloudflare team, just a community champ/mvp 🙂 I'm not sure there...

Mirza•8/27/25, 1:11 PM

Ohh i mistaken the cloudflare community champ tag on you as a team member.

I'm having startup credits from cloudflare and with that I've got weekly office hours access where founders come and maybe discuss about what can help them to do better etc....

JJames The claude models aren't freely available, which is why I imagine we've not seen...

Mirza•8/27/25, 1:12 PM

Yeah there are only the open source models in the list, maybe a partnership with claude would help us as builders.

James•8/27/25, 1:12 PM

Ahh, yeah you could bring it up there! A few new processing models were released today, but nothing for the code side yet: https://developers.cloudflare.com/changelog/2025-08-27-partner-models/

Cloudflare Docs

Deepgram and Leonardo partner models now available on Workers AI

State-of-the-art TTS, STT and image generation models, hosted on Workers AI infrastructure

JJames Ahh, yeah you could bring it up there! A few new processing models were released...

Mirza•8/27/25, 1:13 PM

Yeah saw it just now, thanks

FFabian Hi everyone, I would like to stream the response from `@cf/openai/gpt-oss-20b` b...

samjs•8/27/25, 4:47 PM

Hey @Fabian . Streaming for the gpt-oss models is not currently supported, but we're working on it and hope to have an update soon!

samjs•8/27/25, 4:48 PM

Hey folks! Happy AI Week. I'm on the Workers AI team and will be around to help answer any questions you all have

HHimanshu Is there an ETA for function calling support for gpt oss models in workers ai?

samjs•8/27/25, 4:52 PM

Hey @Himanshu ! Function calling should work with the gpt-oss models. Here's an example of that with our sandbox SDK: https://github.com/cloudflare/sandbox-sdk/blob/main/examples/code-interpreter/src/index.ts#L73-L91

We've also tested against the compatibilty suite here: https://github.com/openai/gpt-oss/tree/main/compatibility-test and are getting ~93% successes. Do you have an example of what you were trying that didn't work?

We're also actively working on integrating support for the builtin

code_interpreter

code_interpreter

tool to make some of this even easier.

MMirza Hey James, I believe you're from the cloudflare team. I wanted to know why are...

samjs•8/27/25, 4:56 PM

Hi @Mirza

currently we've mostly been running open source models on our platform. Though as of today we have new partner models we're supporting as James linked above.

Definitely feel free to share your feedback in your office hours today. Sharing here is also a good way for us to know what is important to folks. Are you open to sharing more about your use case? I'd also love to hear if you've tried the gpt-oss models for your use case and how those compare.

Ssamjs Hi @Mirza 👋 currently we've mostly been running open source models on our pla...

Mirza•8/27/25, 4:58 PM

Yeah, let me DM you as I'm pre-occupied today so won't be able to attend the office hours ig.

Himanshu•8/27/25, 5:01 PM

Getting this error when trying out deepgram/nova-3:

{
"message": "Error: Network connection lost.",
"code": 6001
}

HHimanshu Getting this error when trying out deepgram/nova-3: { "mess...

samjs•8/27/25, 5:02 PM

Taking a look on our side

Ssamjs Hey @Himanshu ! Function calling should work with the gpt-oss models. Here's an ...

Himanshu•8/27/25, 5:05 PM

Amazing! I was trying it out with ai sdk, but the normal tool call invocation that worked with other function calling models like Llama 3.3, didn't work. I checked the docs, and there was no mention of tool call support for these models, so I took a back seat. I'll retry and update you. Thanks again.

Himanshu•8/27/25, 5:44 PM

Leonardo lucid-origin is a fantastic model!!

Prompt: Ultra-realistic 3D render of Times Square at night with neon lights and bustling atmosphere. A massive digital billboard dominates the scene, displaying the Cloudflare logo with the words ‘Workers AI’ in bold white font on a vibrant orange background. The billboard is crisp, glowing brightly against the dark sky, reflecting on nearby glass buildings. Camera shot captured from across the street at a slightly low angle, showing street lights, cars, and people walking, adding scale and realism. Photorealistic textures, cinematic lighting, and a dramatic depth-of-field emphasize the billboard as the centerpiece.

samjs•8/27/25, 6:08 PM

Love it! Thanks for sharing

HHimanshu Getting this error when trying out deepgram/nova-3: { "mess...

samjs•8/27/25, 8:33 PM

FYI we have a fix going out for this issue shortly

akazwzOP•8/28/25, 11:04 AM

Workers AI released so many models, which is exciting! Some models support WebSocket, which is awesome, but I can't find any documentation on how to use them with WebSocket. Can anyone help?

John Spurlock•8/28/25, 2:09 PM

pricing for the deepgram nova models should be based on audio input, right? Here is a quick PR to tweak the pricing page wording https://github.com/cloudflare/cloudflare-docs/pull/24763

GitHub

Prices should be per input for audio input models by johnspurlock-s...

Aakazwz Workers AI released so many models, which is exciting! Some models support WebSo...

samjs•8/28/25, 2:55 PM

I think we're still working on the documentation! But in short it should look something like:

export default {
  async fetch(request, env, ctx): Promise<Response> {
    const resp = await env.AI.run("@cf/deepgram/aura-1", {}, {
      websocket: true
    });
    return resp;
  },
} satisfies ExportedHandler<Env>;

export default {
  async fetch(request, env, ctx): Promise<Response> {
    const resp = await env.AI.run("@cf/deepgram/aura-1", {}, {
      websocket: true
    });
    return resp;
  },
} satisfies ExportedHandler<Env>;

export default { async fetch(request, env, ctx): Promise<Response> { const resp = await env.AI.run("@cf/deepgram/aura-1", {}, { websocket: true }); return resp; }, } satisfies ExportedHandler<Env>;

will gpt-oss-120b support stream ?

Similar Threads

Similar Threads

Similar Threads