do you have the actual example? or do you have a specific question? - Cloudflare Developers

© 2026 Hedgehog Software, LLC

More

Communities Docs About Terms Privacy

creet

Ccreet do you have the actual example? or do you have a specific question?

NekoKenma

N

NekoKenma•7/21/25, 11:35 AM

The last post of the gh thread documents exactly what I'm facing, basically the OpenAI compatible endpoint, when plugged as a model for the langchain agent with tools, does not work.

NekoKenma

NNekoKenma The last post of the gh thread documents exactly what I'm facing, basically the ...

creet

C

creetOP•7/22/25, 12:32 AM

i see, looking at the docs Workers AI doesnt natively support "Tool Calling" the way the openai SDK or others does. where tool calling in other SDKs parses the <tool_call> portion of the generated text for you. its pretty straightforward to do yourself tho, im not sure if/how you would do that in n8n but with code you would just have a method that wraps the llm generation, collects the text between the <tool_call> and </tool_call> tags, and then streams the remainder as the text response.

creet

C

creetOP•7/22/25, 12:33 AM

oh nevermind it looks like the docs does say that they support openai tool calling

creet

C

creetOP•7/22/25, 12:34 AM

okay yeah so it is broken but a workaround is like how i mentioned above

creet

Ccreet okay yeah so it is broken but a workaround is like how i mentioned above

NekoKenma

N

NekoKenma•7/22/25, 12:36 AM

Actually, I think it's not exactly broken - but can be a feature request maybe. You see, the openai compatible endpoint page clearly states its for text generation and embedding models only, so I suppose that doesn't include tool calling.

Thanks, ig I'll just have to do what you said, make a wrapper worker.

Cloudflare Docs

OpenAI compatible API endpoints

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions) and text embedding models (/v1/embeddings). This allows you to use the same code as you would for your OpenAI commands, but swap in Workers AI easily.

creet

C

creetOP•7/22/25, 12:53 AM

https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/ this is what youre looking for, response.tool_calls is supposed to be the automatic tool call parsing

Cloudflare Docs

This page shows how you can do traditional function calling, as defined by industry standards. Workers AI also offers embedded function calling, which is drastically easier than traditional function calling.

scotto

S

scotto•7/22/25, 12:57 AM

can we please add some of the qwen3 reranker models?

top in the leaderboard for all ranking tasks

scotto

S

scotto•7/22/25, 12:58 AM

also atm cloudflare dosent have any multilingual reranker

nclevenger

Nnclevenger is there a problem with AI.toMarkdown from HTML? I'm just getting raw HTML when...

ldenoue

L

ldenoue•7/23/25, 8:07 AM

@nclevenger I have the same issue. it works for PDFs, but not for HTML. Have you found an answer since?

Elliot Hesp

E

Elliot Hesp•7/24/25, 10:32 AM

Am I reading the pricing correclty of workers AI - they ONLY charge for Neurons, not Neurons + tokens?

Cause the LLM model pricing table shows both, so me scanning it assumed that you paid for both

usualdev

U

usualdev•7/24/25, 8:41 PM

For the AutoRAG:

1. How can I change the prompt to refer to the URL of the article instead of the file itself? For example, I converted my website into markdown, now the answer is citing the actual file in R2. Is there a way to cite the url itself?
2. Can I use a different model for generation instead of the default ones? such as GPT4o-mini?

Jehrome

J

Jehrome•7/25/25, 4:07 PM

Hi. I'm seeing an error in my logs >> AI analysis failed: AiError: 3040: Capacity temporarily exceeded, please try again.
Unsure what this means, no idea what my capacity actually is.. Could someone help? thanks

Laurence

L

Laurence•7/25/25, 4:17 PM

https://github.com/cloudflare/cloudflare-docs/issues/23308 I am also hitting this error. Quick search on Google there's a number of people with the same problem. How do we get it "seen" and fixed?

English works perfectly fine, for example with the below request I get a 1 second .mp3
{
"prompt": "Hello, this is a test",
"lang": "en"
}

But if I do the same with French (which melotts & cloudflare docs says is supported)
{
"prompt": "Hello, this is a test",
"lang": "fr"
}

I get this back:
{
"httpCode": 404,
"internalCode": 8002,
"message": "AiError: AiError: Invalid input",
"name": "AiError",
"skipSentry": true,
"description": "Invalid input"
}

cosbgn

C

cosbgn•7/25/25, 8:29 PM

Does anyone has latency data for whisper via workers AI? I'm looking to figure out where workers are in this chart.

I'm working on a real-time agent and I need the fastest there is

cosbgn

Ccosbgn Does anyone has latency data for whisper via workers AI? I'm looking to figure o...

creet

C

creetOP•7/25/25, 8:39 PM

you definitely dont need "fastest there is" for real time applications. per the figure out just posted those show inference times of 375-35x real time. truly you could have 1x and still build realtime applications if you did a good job of doing realtime inference and transcribe throughout the user talking

creet

C

creetOP•7/25/25, 8:40 PM

the figure you shared is more applicable to batch processing where it would actually be useful to transcribe 375 seconds of audio in 1 second

cosbgn

C

cosbgn•7/25/25, 8:41 PM

I actually chunk it so I process only very short sentences

creet

C

creetOP•7/25/25, 8:42 PM

yes exactly if you are processing chunks of speech as the user is talking you can still respond immediately with little to no delay

creet

C

creetOP•7/25/25, 8:42 PM

but to answer your original question idk the latency of whisper with workers ai but its probably 'fast enough'

cosbgn

C

cosbgn•7/25/25, 8:42 PM

These are some stats of my setup:

[6:27:34 PM] read_form_data: 25.065ms
[6:27:39 PM] transcribe: 4.907s
[6:27:40 PM] generate_fist_text_chunk: 763.861ms
[6:27:41 PM] generate_speech_chunk_0: 1.476s
[6:27:42 PM] generate_speech_chunk_1: 901.312ms


[6:28:02 PM] read_form_data: 19.01ms
[6:28:06 PM] transcribe: 3.367s
[6:28:06 PM] generate_fist_text_chunk: 612.575ms
[6:28:07 PM] generate_speech_chunk_0: 1.148s
[6:28:09 PM] generate_speech_chunk_1: 1.428s
[6:28:10 PM] generate_speech_chunk_2: 1.582s
[6:28:12 PM] generate_speech_chunk_3: 1.267s


[6:28:26 PM] read_form_data: 5.596ms
[6:28:27 PM] transcribe: 1.593s
[6:28:28 PM] generate_fist_text_chunk: 798.88ms
[6:28:29 PM] generate_speech_chunk_0: 1.294s
[6:28:30 PM] generate_speech_chunk_1: 698.474ms
[6:28:31 PM] generate_speech_chunk_2: 920.976ms

[6:27:34 PM] read_form_data: 25.065ms
[6:27:39 PM] transcribe: 4.907s
[6:27:40 PM] generate_fist_text_chunk: 763.861ms
[6:27:41 PM] generate_speech_chunk_0: 1.476s
[6:27:42 PM] generate_speech_chunk_1: 901.312ms


[6:28:02 PM] read_form_data: 19.01ms
[6:28:06 PM] transcribe: 3.367s
[6:28:06 PM] generate_fist_text_chunk: 612.575ms
[6:28:07 PM] generate_speech_chunk_0: 1.148s
[6:28:09 PM] generate_speech_chunk_1: 1.428s
[6:28:10 PM] generate_speech_chunk_2: 1.582s
[6:28:12 PM] generate_speech_chunk_3: 1.267s


[6:28:26 PM] read_form_data: 5.596ms
[6:28:27 PM] transcribe: 1.593s
[6:28:28 PM] generate_fist_text_chunk: 798.88ms
[6:28:29 PM] generate_speech_chunk_0: 1.294s
[6:28:30 PM] generate_speech_chunk_1: 698.474ms
[6:28:31 PM] generate_speech_chunk_2: 920.976ms

cosbgn

C

cosbgn•7/25/25, 8:43 PM

transcribe is the one which takes the most. (These are using openai endpoints)

cosbgn

C

cosbgn•7/25/25, 8:44 PM

After generate_speech_chunk_0generate_speech_chunk_0 that is already sent back to the client, so the other chunks don't really matter

cosbgn

C

cosbgn•7/25/25, 8:44 PM

Ideally I would like to get transcribe to a few hundreds of milliseconds, not 5 seconds.

cosbgn

Ccosbgn These are some stats of my setup: ``` [6:27:34 PM] read_form_data: 25.065ms [6:...

Isaac McFadyen

I

Isaac McFadyen•7/25/25, 8:47 PM

How long is your audio?

cosbgn

C

cosbgn•7/25/25, 8:47 PM

a sentence

cosbgn

C

cosbgn•7/25/25, 8:47 PM

couple of seconds, like 10 seconds maybe

creet

C

creetOP•7/26/25, 5:14 AM

Hmm yeah 2x transcription speed is pretty bad

creet

C

creetOP•7/26/25, 5:14 AM

Make sure you’re using whisper v3 large turbo or tiny en

creet

C

creetOP•7/26/25, 5:15 AM

Not the base whisper

zsueo

Z

zsueo•7/29/25, 4:33 PM

hey im using llama-4-scout-17b-16e-instruct for tool calling and it seems to just get stuck in a call loop? like it just keeps calling the tool over and over

zsueo

Z

zsueo•7/29/25, 6:54 PM

it seems to be doing this for both this and the hermes model

zsueo

Z

zsueo•7/29/25, 6:54 PM

im sending in requests like this

{
  "messages": [
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-04bad465677b4783b6797433a8deb05b",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-5d73b086fe964ae2900cdd527a659f1f",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-0140345ec2584ae0b8d4d6d1e3c6585e",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "user",
      "content": "hey can you fetch data relating to joe?"
    },
    {
      "role": "system",
      "content": "You are a database assistant, you will query the database if a user requests information from it and respond with a human readable output, please only call the tool once and respond each time after a tool call, all tool responses have the role tool"
    }
  ],
  "max_tokens": 1024,
  "raw": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "Query Vector Database",
        "description": "Queries the Vector Database for legal documents matching the provided query",
        "parameters": {
          "type": "object",
          "properties": {
            "Query": {
              "type": "string",
              "description": "The query to the Vector db"
            }
          },
          "required": ["Query"]
        }
      }
    }
  ]
}

{
  "messages": [
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-04bad465677b4783b6797433a8deb05b",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-5d73b086fe964ae2900cdd527a659f1f",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-0140345ec2584ae0b8d4d6d1e3c6585e",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "user",
      "content": "hey can you fetch data relating to joe?"
    },
    {
      "role": "system",
      "content": "You are a database assistant, you will query the database if a user requests information from it and respond with a human readable output, please only call the tool once and respond each time after a tool call, all tool responses have the role tool"
    }
  ],
  "max_tokens": 1024,
  "raw": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "Query Vector Database",
        "description": "Queries the Vector Database for legal documents matching the provided query",
        "parameters": {
          "type": "object",
          "properties": {
            "Query": {
              "type": "string",
              "description": "The query to the Vector db"
            }
          },
          "required": ["Query"]
        }
      }
    }
  ]
}

but it keeps trying to call the tool again

zsueo

Z

zsueo•7/29/25, 7:16 PM

do i need to send the assistant querying the tool as part of the message history?

DrWayne

D

DrWayne•7/29/25, 7:25 PM

I'm having trouble with the https://api.cloudflare.com/client/v4/accounts/{accountId}/ai/models/search endpoint in-browser. The fetch triggers an OPTIONS request that fails with "400 Bad Request".
The GET request works just fine via curl on the command line, and the OPTIONS request curl returns
{"success":false,"errors":[{"code":9106,"message":"Missing X-Auth-Key, X-Auth-Email or Authorization headers"}]}{"success":false,"errors":[{"code":9106,"message":"Missing X-Auth-Key, X-Auth-Email or Authorization headers"}]}

Since the OPTIONS call is checking whether the endpoint allows those headers, it will never include them in the request. I'm betting it is just an overzealous config on needing the auth header/s missing an exception for the OPTIONS call. Or am I missing something here?

zsueo

Zzsueo do i need to send the assistant querying the tool as part of the message history...

Isaac McFadyen

I

Isaac McFadyen•7/29/25, 7:38 PM

Yes, both the tool result and the tool call need to be included as a message.

Isaac McFadyen

I

Isaac McFadyen•7/29/25, 7:38 PM

Also it looks like you have the messages array backwards. Lower in the array (higher index) is more recent.

Isaac McFadyen

I

Isaac McFadyen•7/29/25, 7:38 PM

I.e. system, then user, then assistant tool call, then tool result.

Isaac McFadyen

IIsaac McFadyen Also it looks like you have the messages array backwards. Lower in the array (hi...

zsueo

Z

zsueo•7/29/25, 7:38 PM

huh the example had it higher in the array as first, easier for me to manage

Isaac McFadyen

IIsaac McFadyen Yes, both the tool result and the tool call need to be included as a message.

zsueo

Z

zsueo•7/29/25, 7:39 PM

the input schema for this model doesnt have anything like that i guess the docs are a bit behind

zsueo

Zzsueo huh the example had it higher in the array as first, easier for me to manage

zsueo

Z

zsueo•7/29/25, 7:39 PM

https://github.com/craigsdennis/lightbulb-moment-tool-calling/blob/main/src/index.ts <- this is the example i was referencing

GitHub

lightbulb-moment-tool-calling/src/index.ts at main · craigsdennis/...

Workers AI Tool Calling Example. Contribute to craigsdennis/lightbulb-moment-tool-calling development by creating an account on GitHub.

zsueo

Zzsueo https://github.com/craigsdennis/lightbulb-moment-tool-calling/blob/main/src/inde...

Isaac McFadyen

I

Isaac McFadyen•7/29/25, 7:41 PM

Yeah, so the unshiftunshift adds the system message to the start of the array

zsueo

Z

zsueo•7/29/25, 7:42 PM

oh right myb i got it mixed up

Isaac McFadyen

IIsaac McFadyen Yes, both the tool result and the tool call need to be included as a message.

zsueo

Z

zsueo•7/29/25, 7:42 PM

is there a recommended schema for this?

zsueo

Z

zsueo•7/29/25, 7:42 PM

i cant find anything on cloudflares docs

Isaac McFadyen

I

Isaac McFadyen•7/29/25, 7:45 PM

I'm not 100% sure, but I expect it mirrors the OpenAI schema, so it would be an assistant message but with

tool_calls

tool_calls

instead of string

content

content

: https://platform.openai.com/docs/api-reference/chat/create

Isaac McFadyen

I

Isaac McFadyen•7/29/25, 7:45 PM

Worth a try, if it's wrong you'll get an error that might help troubleshoot further

zsueo

Z

zsueo•7/29/25, 7:46 PM

thanks so much!

Isaac McFadyen

IIsaac McFadyen I'm not 100% sure, but I expect it mirrors the OpenAI schema, so it would be an ...

zsueo

Z

zsueo•7/29/25, 9:02 PM

it worked!! just incase anyone wants to do something similar this is the schema i used

   {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "039920ecd55b4b5ba689d692933602cc",
          "function": {
            "name": "Query Vector Database",
            "arguments": "{\"Query\": \"joe\"}"
          },
          "type": "function"
        }
      ],
      "content": ""
    }

   {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "039920ecd55b4b5ba689d692933602cc",
          "function": {
            "name": "Query Vector Database",
            "arguments": "{\"Query\": \"joe\"}"
          },
          "type": "function"
        }
      ],
      "content": ""
    }

zsueo

Z

zsueo•7/30/25, 6:02 PM

hey im noticing that sometimes the ai does not call the tool properly leading to json just floating in the reply, what is the expected format the ai is supposed to output in so i can try and force that?

E

ed•7/30/25, 7:54 PM

when's the next models update? would love to have kimi-k2, GLM, etc

[6:27:34 PM] read_form_data: 25.065ms
[6:27:39 PM] transcribe: 4.907s
[6:27:40 PM] generate_fist_text_chunk: 763.861ms
[6:27:41 PM] generate_speech_chunk_0: 1.476s
[6:27:42 PM] generate_speech_chunk_1: 901.312ms


[6:28:02 PM] read_form_data: 19.01ms
[6:28:06 PM] transcribe: 3.367s
[6:28:06 PM] generate_fist_text_chunk: 612.575ms
[6:28:07 PM] generate_speech_chunk_0: 1.148s
[6:28:09 PM] generate_speech_chunk_1: 1.428s
[6:28:10 PM] generate_speech_chunk_2: 1.582s
[6:28:12 PM] generate_speech_chunk_3: 1.267s


[6:28:26 PM] read_form_data: 5.596ms
[6:28:27 PM] transcribe: 1.593s
[6:28:28 PM] generate_fist_text_chunk: 798.88ms
[6:28:29 PM] generate_speech_chunk_0: 1.294s
[6:28:30 PM] generate_speech_chunk_1: 698.474ms
[6:28:31 PM] generate_speech_chunk_2: 920.976ms

[6:27:34 PM] read_form_data: 25.065ms
[6:27:39 PM] transcribe: 4.907s
[6:27:40 PM] generate_fist_text_chunk: 763.861ms
[6:27:41 PM] generate_speech_chunk_0: 1.476s
[6:27:42 PM] generate_speech_chunk_1: 901.312ms


[6:28:02 PM] read_form_data: 19.01ms
[6:28:06 PM] transcribe: 3.367s
[6:28:06 PM] generate_fist_text_chunk: 612.575ms
[6:28:07 PM] generate_speech_chunk_0: 1.148s
[6:28:09 PM] generate_speech_chunk_1: 1.428s
[6:28:10 PM] generate_speech_chunk_2: 1.582s
[6:28:12 PM] generate_speech_chunk_3: 1.267s


[6:28:26 PM] read_form_data: 5.596ms
[6:28:27 PM] transcribe: 1.593s
[6:28:28 PM] generate_fist_text_chunk: 798.88ms
[6:28:29 PM] generate_speech_chunk_0: 1.294s
[6:28:30 PM] generate_speech_chunk_1: 698.474ms
[6:28:31 PM] generate_speech_chunk_2: 920.976ms

{
  "messages": [
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-04bad465677b4783b6797433a8deb05b",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-5d73b086fe964ae2900cdd527a659f1f",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-0140345ec2584ae0b8d4d6d1e3c6585e",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "user",
      "content": "hey can you fetch data relating to joe?"
    },
    {
      "role": "system",
      "content": "You are a database assistant, you will query the database if a user requests information from it and respond with a human readable output, please only call the tool once and respond each time after a tool call, all tool responses have the role tool"
    }
  ],
  "max_tokens": 1024,
  "raw": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "Query Vector Database",
        "description": "Queries the Vector Database for legal documents matching the provided query",
        "parameters": {
          "type": "object",
          "properties": {
            "Query": {
              "type": "string",
              "description": "The query to the Vector db"
            }
          },
          "required": ["Query"]
        }
      }
    }
  ]
}

{
  "messages": [
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-04bad465677b4783b6797433a8deb05b",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-5d73b086fe964ae2900cdd527a659f1f",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-0140345ec2584ae0b8d4d6d1e3c6585e",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "user",
      "content": "hey can you fetch data relating to joe?"
    },
    {
      "role": "system",
      "content": "You are a database assistant, you will query the database if a user requests information from it and respond with a human readable output, please only call the tool once and respond each time after a tool call, all tool responses have the role tool"
    }
  ],
  "max_tokens": 1024,
  "raw": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "Query Vector Database",
        "description": "Queries the Vector Database for legal documents matching the provided query",
        "parameters": {
          "type": "object",
          "properties": {
            "Query": {
              "type": "string",
              "description": "The query to the Vector db"
            }
          },
          "required": ["Query"]
        }
      }
    }
  ]
}

   {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "039920ecd55b4b5ba689d692933602cc",
          "function": {
            "name": "Query Vector Database",
            "arguments": "{\"Query\": \"joe\"}"
          },
          "type": "function"
        }
      ],
      "content": ""
    }

   {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "039920ecd55b4b5ba689d692933602cc",
          "function": {
            "name": "Query Vector Database",
            "arguments": "{\"Query\": \"joe\"}"
          },
          "type": "function"
        }
      ],
      "content": ""
    }