Cloudflare Developers•5mo ago

Also it looks like you have the messages array backwards. Lower in the array (higher index) is more

Also it looks like you have the messages array backwards. Lower in the array (higher index) is more recent.

Isaac McFadyenOP•7/29/25, 7:38 PM

I.e. system, then user, then assistant tool call, then tool result.

IIsaac McFadyen Also it looks like you have the messages array backwards. Lower in the array (hi...

zsueo•7/29/25, 7:38 PM

huh the example had it higher in the array as first, easier for me to manage

IIsaac McFadyen Yes, both the tool result and the tool call need to be included as a message.

zsueo•7/29/25, 7:39 PM

the input schema for this model doesnt have anything like that i guess the docs are a bit behind

Zzsueo huh the example had it higher in the array as first, easier for me to manage

zsueo•7/29/25, 7:39 PM

https://github.com/craigsdennis/lightbulb-moment-tool-calling/blob/main/src/index.ts <- this is the example i was referencing

GitHub

lightbulb-moment-tool-calling/src/index.ts at main · craigsdennis/...

Workers AI Tool Calling Example. Contribute to craigsdennis/lightbulb-moment-tool-calling development by creating an account on GitHub.

Zzsueo https://github.com/craigsdennis/lightbulb-moment-tool-calling/blob/main/src/inde...

Isaac McFadyenOP•7/29/25, 7:41 PM

Yeah, so the

unshift

unshift

adds the system message to the start of the array

zsueo•7/29/25, 7:42 PM

oh right myb i got it mixed up

IIsaac McFadyen Yes, both the tool result and the tool call need to be included as a message.

zsueo•7/29/25, 7:42 PM

is there a recommended schema for this?

zsueo•7/29/25, 7:42 PM

i cant find anything on cloudflares docs

Isaac McFadyenOP•7/29/25, 7:45 PM

I'm not 100% sure, but I expect it mirrors the OpenAI schema, so it would be an assistant message but with

tool_calls

tool_calls

instead of string

content

content

: https://platform.openai.com/docs/api-reference/chat/create

Isaac McFadyenOP•7/29/25, 7:45 PM

Worth a try, if it's wrong you'll get an error that might help troubleshoot further

zsueo•7/29/25, 7:46 PM

thanks so much!

IIsaac McFadyen I'm not 100% sure, but I expect it mirrors the OpenAI schema, so it would be an ...

zsueo•7/29/25, 9:02 PM

it worked!! just incase anyone wants to do something similar this is the schema i used

   {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "039920ecd55b4b5ba689d692933602cc",
          "function": {
            "name": "Query Vector Database",
            "arguments": "{\"Query\": \"joe\"}"
          },
          "type": "function"
        }
      ],
      "content": ""
    }

   {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "039920ecd55b4b5ba689d692933602cc",
          "function": {
            "name": "Query Vector Database",
            "arguments": "{\"Query\": \"joe\"}"
          },
          "type": "function"
        }
      ],
      "content": ""
    }

zsueo•7/30/25, 6:02 PM

hey im noticing that sometimes the ai does not call the tool properly leading to json just floating in the reply, what is the expected format the ai is supposed to output in so i can try and force that?

ed•7/30/25, 7:54 PM

when's the next models update? would love to have kimi-k2, GLM, etc

rob•7/31/25, 6:16 PM

apparently maybe K2 soon

https://x.com/_mchenco/status/1947761683339825207

michelle (@_mchenco)

i ate my words. who's got a use case for kimi on workers ai? let's chat

•

7/22/25, 8:52 PM

Shadi•8/1/25, 6:09 AM

i kindly pray by my bedside that we support

allminiLmv6

allminiLmv6

in cloudflare workers ai.

would acc be so insane.

im using it for my current embeddings and i don't think its best practice to use 2 different embedding models, as I see we only support bge.

Shadi•8/1/25, 4:35 PM

lmao

R3AP3R editz•8/3/25, 3:06 PM

can anyone say how can we change the embdding model in autorag?
and also does autorag support multilanguae?

ed•8/4/25, 3:57 PM

✘ [ERROR] Uncaught (in promise) InferenceUpstreamError: 10000: Authentication error

      at _parseError (cloudflare-internal:ai-api:107:24)

✘ [ERROR] Uncaught (in promise) InferenceUpstreamError: 10000: Authentication error

      at _parseError (cloudflare-internal:ai-api:107:24)

Anyone else getting this error? Tried logging in and out over and over again

iqorex•8/5/25, 7:09 PM

Is there any chance we will see this model soon in workers ai

? https://openai.com/index/introducing-gpt-oss/

Introducing gpt-oss

gpt-oss-120b and gpt-oss-20b push the frontier of open-weight reasoning models

Original message was deleted

Isaac McFadyenOP•8/5/25, 9:15 PM

It's definitely not 0 :lul:
https://blog.cloudflare.com/openai-gpt-oss-on-workers-ai/

The Cloudflare Blog

Partnering with OpenAI to bring their new open models onto Cloudfla...

OpenAI’s newest open-source models are now available on Cloudflare Workers AI on Day 0, with support for Responses API, Code Interpreter and Web Search (coming soon).

iqorex•8/6/25, 8:36 AM

damn that was fast

�

𝗞enneth•8/6/25, 6:03 PM

Hi Cloudflare team,

Does the workers ai @cf/openai/gpt-oss-120b model support structured outputs with JSON schemas?

I need to force the model to return responses in a specific JSON format using response_format parameter. I don't see this mentioned in the docs.

�𝗞enneth Hi Cloudflare team, Does the workers ai @cf/openai/gpt-oss-120b model support ...

Chaika•8/6/25, 6:19 PM

They're using the new responses openai format:
https://platform.openai.com/docs/api-reference/responses/create

which supports that under the text object looks like. Tested it quickly and it seems to work fine for me

�

𝗞enneth•8/6/25, 6:22 PM

thanks @Chaika

akazwz•8/7/25, 10:26 AM

will gpt-oss-120b support stream ?

Aakazwz will gpt-oss-120b support stream ?

Chaika•8/7/25, 3:37 PM

It already does support streaming via openai's new response format streaming:
https://platform.openai.com/docs/guides/streaming-responses?api-mode=responses

akazwz•8/7/25, 5:00 PM

I was using the response api. without stream param, it works, with stream: true, server error, return 500

akazwz•8/7/25, 5:00 PM

Aakazwz I was using the response api. without stream param, it works, with stream: true,...

Chaika•8/7/25, 5:20 PM

Have to use Workers binding for streaming

Chaika•8/7/25, 5:23 PM

I don't think the direct api has supported streaming for any model? Not 100% sure as I don't use the api myself, but yea that same request via binding streaming looks fine

Dev2xl•8/9/25, 10:35 PM

Hello,
I’m exploring options for running inference on private TensorFlow models with Cloudflare’s products. Is it possible to achieve this using Workers or any other Cloudflare service? Any technical guidance or references would be greatly appreciated. Thank you!

Isaac McFadyenOP•8/10/25, 4:16 PM

You could try Containers but they're currently ill-suited for most ML things as they have fairly low resource allocation and no GPUs.

ixixix•8/12/25, 6:44 AM

I know the docs don't mention it, but just to confirm, the /audio/transcriptions endpoint isn't supported as an openai compatible endpoint in Workers AI or AI Gateway for the cloudflare whisper models, right? I'm getting {"errors":[{"code":7000,"message":"No route for that URI"}]}{"errors":[{"code":7000,"message":"No route for that URI"}]} and {"code":2002,"message":"Internal server error"}{"code":2002,"message":"Internal server error"}

tomerzcod•8/13/25, 7:29 AM

Hey, i've been following the cloudflare docs, I have a worker with ai bindings set I'm logged in but whatever i do I get the 10000: authentication error.

What am I missing? can't find any info how to fix this

Nhat Bui•8/14/25, 1:42 AM

Hey guys, I don't know why this prompt is detected as NSFW content

xugik•8/14/25, 11:51 AM

Hi, does anyone happen to know when the Chat Completions API will come to the gpt-oss models?

Bryan•8/15/25, 4:38 AM

Does anyone have advice for getting worker-ai function calling models to work with traditional tool calls, while using the open-ai compatibility layer *and * streaming?
I can't get any model to both send back tool recommendations in the correct format and summarize a tool call.

Mistral keeps recommending tools via XML in regular text content, but when given a chat history where the most recent message is a tool execution, it will correctly summarize the tool execution.
llama-4-scout will correctly recommended a tool, but when given a chat history where the most recent message is a tool execution, it will recommend the tool again instead of summarizing.
The other 2 function calling models aren't any better.

If i turn off streaming, Mistral at least works correctly, but streaming is strongly preferred.

Does anyone know what I'm missing?

AAhmed is there a way to get sparse embeddings from `@cf/baai/bge-m3`

Nipsuli•8/15/25, 11:15 PM

yeah, this would be nice

SmallShen•8/16/25, 10:58 PM

can we have qwen embedding model in worker ai?

mdanowar2•8/19/25, 12:30 PM

scotto•8/19/25, 2:12 PM

openai model dont work with chat/completion API, that can be fixed? from model change standpoint it's good to have the option to switch only changing the string name of the model

DonJonko•8/19/25, 3:49 PM

Heya pretty people! Im looking for a way to set a 'spending limit' on my worker ai usage. Does that exist ?

RS•8/20/25, 7:13 AM

Can we have https://github.com/QwenLM/Qwen-Image models for image generation? The current available models are outdated.

GitHub

GitHub - QwenLM/Qwen-Image: Qwen-Image is a powerful image generati...

Qwen-Image is a powerful image generation foundation model capable of complex text rendering and precise image editing. - QwenLM/Qwen-Image

MichaelC•8/20/25, 11:29 AM

How to use the current image generation models that CF offers? So far in my tests I have good results with https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/. However there is no dimension control which renders it pretty much unusable for what I want to offer. Then I dont see any other upscaling models available either unfortunately.

Cloudflare Docs

flux-1-schnell

FLUX.1 [schnell] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions.

Zzsueo hey im noticing that sometimes the ai does not call the tool properly leading to...

ayad99•8/20/25, 2:02 PM

hey i noticed that too, specially with the llama-4-scout model. did you find any solution or workaround?

Horiwix•8/20/25, 6:35 PM

Is there any AI model available that has internet access and could search for realtime/live data? I don't think so, couldn't find one, but asking just in case

HHoriwix Is there any AI model available that has internet access and could search for re...

Isaac McFadyenOP•8/20/25, 7:58 PM

Not really, Workers AI is more low-level and if you wanted an LLM to do this you'd need to orchestrate yourself or use Sandboxes: https://developers.cloudflare.com/changelog/2025-06-24-announcing-sandboxes/

yinxingmaiming6409•8/21/25, 2:11 AM

How can AI be used in Cherry Studio? It seems impossible.

Yyinxingmaiming6409 How can AI be used in Cherry Studio? It seems impossible.

xugik•8/21/25, 7:17 AM

https://youtu.be/3XOJvW3einQ?si=BmuUEasTmeHRoCgp&t=447 I followed the guide here and was able to get the api key and Account ID to use it in Cherry Studio

YouTubeQuorify

200+ FREE AI Model APIs You Can Use RIGHT NOW! (Unlimited Access)

Are you a developer, student, or AI enthusiast tired of hitting paywalls and restrictive rate limits? In this ultimate 2025 guide, we reveal the secrets to accessing the world's most powerful AI models—including GPT-4, Llama 3, and Gemini—for FREE. We go end-to-end, showing you exactly how to sign up, get your API keys, and make your first c...

Also it looks like you have the messages array backwards. Lower in the array (higher index) is more

Similar Threads