I understand with scaling

B

brylanOP•7/14/25, 4:04 PM

But from a consumer perspective this is kinda unacceptable no?

B

brylanOP•7/14/25, 4:04 PM

4 requests per minute and your seeing errors? I mean with any other service this would just be crazy

B

brylanOP•7/14/25, 4:15 PM

Being said I love cloudflare ai, but this does seem like this should be talked about internally or somthing

Bbrylan I understand with scaling

C

Chaika•7/14/25, 4:23 PM

Just saying in case it's worth noting: Green/champs are just volunteers and the AI team has said things in the past about needing to scale models, espec after launches/big increases: workers-ai, which is where I'm saying that from, but they did not comment on this exact issue/rise in errors (nor was there a status page). I track those capacity errors on my own (unrelated to the team) and I do see them come and go for various models, and it does seem to be around just scaling and such

B

brylanOP•7/14/25, 4:40 PM

ok thanks

R

razor•7/14/25, 8:57 PM

Hey, Is the @cf/openai/whisper-large-v3-turbo model workers only ?? do we also have an API endpoint for this like a CURL way of doing it ??

L

lachlan•7/15/25, 3:15 AM

is there a place we can request new open source models? https://developers.cloudflare.com/workers-ai/models/ I am wanting to use the new Kimi K2?

Cloudflare Docs

Models

N

nclevenger•7/15/25, 2:45 PM

is there a problem with AI.toMarkdown from HTML? I'm just getting raw HTML whenever passing html in:

export default {
  async fetch(req, env) {
    const response = await fetch('https://example.com')
    const blob = await response.blob()
    const markdown = await env.ai.toMarkdown({ name: 'example.com', blob })
    return Response.json(markdown )
  }
}

export default {
  async fetch(req, env) {
    const response = await fetch('https://example.com')
    const blob = await response.blob()
    const markdown = await env.ai.toMarkdown({ name: 'example.com', blob })
    return Response.json(markdown )
  }
}

export default {
  async fetch(req, env) {
    const response = await fetch('https://example.com')
    const blob = await response.blob()
    const markdown = await env.ai.toMarkdown({ name: 'example.com', blob })
    return Response.json(markdown )
  }
}

export default {
  async fetch(req, env) {
    const response = await fetch('https://example.com')
    const blob = await response.blob()
    const markdown = await env.ai.toMarkdown({ name: 'example.com', blob })
    return Response.json(markdown )
  }
}

returns:

{
  "name": "example.com",
  "mimeType": "text/html",
  "format": "markdown",
  "tokens": 314,
  "data": "<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset=\"utf-8\" />\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n    <style type=\"text/css\">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n"
}

{
  "name": "example.com",
  "mimeType": "text/html",
  "format": "markdown",
  "tokens": 314,
  "data": "<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset=\"utf-8\" />\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n    <style type=\"text/css\">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n"
}

{
  "name": "example.com",
  "mimeType": "text/html",
  "format": "markdown",
  "tokens": 314,
  "data": "<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset=\"utf-8\" />\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n    <style type=\"text/css\">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n"
}

{
  "name": "example.com",
  "mimeType": "text/html",
  "format": "markdown",
  "tokens": 314,
  "data": "<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset=\"utf-8\" />\n    <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n    <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n    <style type=\"text/css\">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n"
}

V

Vladimir_dev•7/16/25, 11:19 AM

Feature Request: Add MedGemma Models to Cloudflare Workers AI
Dear Cloudflare Workers AI Team,
I would like to request the addition of MedGemma models to the Workers AI platform. MedGemma is Google's specialized collection of medical AI models that would significantly enhance the platform's capabilities for healthcare applications.
Key Benefits:

Compact Size: MedGemma models are relatively small, making them well-suited for edge deployment
Medical Specialization: These models are specifically trained for medical use cases, filling a current gap in Workers AI's model offerings
Healthcare Applications: Would enable developers to build medical applications directly on Cloudflare's edge infrastructure
Cost-Effective: Smaller model sizes mean lower computational costs while maintaining medical domain expertise

Use Cases:

Medical chatbots and virtual assistants
Clinical decision support tools
Medical document analysis
Healthcare workflow automation

Reference:
MedGemma models are available in Google's model collection and have been designed with medical applications in mind.
Adding MedGemma would position Cloudflare Workers AI as a leading platform for healthcare AI applications, combining the benefits of edge computing with specialized medical AI capabilities.
Thank you for considering this request. I believe this addition would greatly benefit the developer community working on healthcare solutions.
Best regards, Vladimir

MedGemma Release - a google Collection

Original message was deleted

V

Vladimir_dev•7/16/25, 11:24 AM

Claude sonnet4)))
Simulated Pre-visit Intake Demo by google
https://huggingface.co/spaces/google/appoint-ready

Appoint Ready - MedGemma Demo - a Hugging Face Space by google

Llachlan is there a place we can request new open source models? https://developers.cloud...

V

vmg•7/16/25, 2:52 PM

seconded, kimi k2 would be awesome

T

trace•7/17/25, 10:24 AM

Hello cloudflare team, the links mentioned in this url for terms and license and more information is not working https://developers.cloudflare.com/workers-ai/models/stable-diffusion-v1-5-img2img/

Cloudflare Docs

stable-diffusion-v1-5-img2img

Stable Diffusion is a latent text-to-image diffusion model capable of generating photo-realistic images. Img2img generate a new image from an input image with Stable Diffusion.

T

trace•7/17/25, 12:13 PM

Tried dreamshaper-8-lcm model through workers AI and the quality is ~~absolutely terrible~~. Is there an issue happening at your end?

edit: need to lower the guidance value for better quality

N

NekoKenma•7/19/25, 8:27 PM

Hey guys! I'm trying to use workers AI with the openai compatible endpoint. While it's been great for text generation, I wanted to plug it into n8n as the model for the AI agent, since the endpoint is openai compatible, and the model I'm using (

@cf/meta/llama-4-scout-17b-16e-instruct

@cf/meta/llama-4-scout-17b-16e-instruct

) does support tool calls as per this page.

The problem is that it's not working in agents that have tools to call. Upon looking further into the problem, there was a thread in n8n's GitHub issues

#15862

#15862

that explored this exact issue. Apparently, the tool call input format in the response from the cf model was wrong, and hence langchain would pick it up as a normal response (more details in the github thread).

Would really appreciate some response on this. Thanks.

N

NekoKenma•7/20/25, 5:55 PM

Devs?

NNekoKenma Hey guys! I'm trying to use workers AI with the openai compatible endpoint. Whil...

C

creet•7/21/25, 6:46 AM

do you have the actual example? or do you have a specific question?

Ccreet do you have the actual example? or do you have a specific question?

N

NekoKenma•7/21/25, 11:35 AM

The last post of the gh thread documents exactly what I'm facing, basically the OpenAI compatible endpoint, when plugged as a model for the langchain agent with tools, does not work.

NNekoKenma The last post of the gh thread documents exactly what I'm facing, basically the ...

C

creet•7/22/25, 12:32 AM

i see, looking at the docs Workers AI doesnt natively support "Tool Calling" the way the openai SDK or others does. where tool calling in other SDKs parses the <tool_call> portion of the generated text for you. its pretty straightforward to do yourself tho, im not sure if/how you would do that in n8n but with code you would just have a method that wraps the llm generation, collects the text between the <tool_call> and </tool_call> tags, and then streams the remainder as the text response.

C

creet•7/22/25, 12:33 AM

oh nevermind it looks like the docs does say that they support openai tool calling

C

creet•7/22/25, 12:34 AM

okay yeah so it is broken but a workaround is like how i mentioned above

Ccreet okay yeah so it is broken but a workaround is like how i mentioned above

N

NekoKenma•7/22/25, 12:36 AM

Actually, I think it's not exactly broken - but can be a feature request maybe. You see, the openai compatible endpoint page clearly states its for text generation and embedding models only, so I suppose that doesn't include tool calling.

Thanks, ig I'll just have to do what you said, make a wrapper worker.

Cloudflare Docs

OpenAI compatible API endpoints

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions) and text embedding models (/v1/embeddings). This allows you to use the same code as you would for your OpenAI commands, but swap in Workers AI easily.

C

creet•7/22/25, 12:53 AM

https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/ this is what youre looking for, response.tool_calls is supposed to be the automatic tool call parsing

Cloudflare Docs

Traditional

This page shows how you can do traditional function calling, as defined by industry standards. Workers AI also offers embedded function calling, which is drastically easier than traditional function calling.

S

scotto•7/22/25, 12:57 AM

can we please add some of the qwen3 reranker models?

top in the leaderboard for all ranking tasks

S

scotto•7/22/25, 12:58 AM

also atm cloudflare dosent have any multilingual reranker

Nnclevenger is there a problem with AI.toMarkdown from HTML? I'm just getting raw HTML when...

L

ldenoue•7/23/25, 8:07 AM

@nclevenger I have the same issue. it works for PDFs, but not for HTML. Have you found an answer since?

E

Elliot Hesp•7/24/25, 10:32 AM

Am I reading the pricing correclty of workers AI - they ONLY charge for Neurons, not Neurons + tokens?

Cause the LLM model pricing table shows both, so me scanning it assumed that you paid for both

U

usualdev•7/24/25, 8:41 PM

For the AutoRAG:

1. How can I change the prompt to refer to the URL of the article instead of the file itself? For example, I converted my website into markdown, now the answer is citing the actual file in R2. Is there a way to cite the url itself?
2. Can I use a different model for generation instead of the default ones? such as GPT4o-mini?

J

Jehrome•7/25/25, 4:07 PM

Hi. I'm seeing an error in my logs >> AI analysis failed: AiError: 3040: Capacity temporarily exceeded, please try again.
Unsure what this means, no idea what my capacity actually is.. Could someone help? thanks

L

Laurence•7/25/25, 4:17 PM

https://github.com/cloudflare/cloudflare-docs/issues/23308 I am also hitting this error. Quick search on Google there's a number of people with the same problem. How do we get it "seen" and fixed?

English works perfectly fine, for example with the below request I get a 1 second .mp3
{
"prompt": "Hello, this is a test",
"lang": "en"
}

But if I do the same with French (which melotts & cloudflare docs says is supported)
{
"prompt": "Hello, this is a test",
"lang": "fr"
}

I get this back:
{
"httpCode": 404,
"internalCode": 8002,
"message": "AiError: AiError: Invalid input",
"name": "AiError",
"skipSentry": true,
"description": "Invalid input"
}

C

cosbgn•7/25/25, 8:29 PM

Does anyone has latency data for whisper via workers AI? I'm looking to figure out where workers are in this chart.

I'm working on a real-time agent and I need the fastest there is

Ccosbgn Does anyone has latency data for whisper via workers AI? I'm looking to figure o...

C

creet•7/25/25, 8:39 PM

you definitely dont need "fastest there is" for real time applications. per the figure out just posted those show inference times of 375-35x real time. truly you could have 1x and still build realtime applications if you did a good job of doing realtime inference and transcribe throughout the user talking

C

creet•7/25/25, 8:40 PM

the figure you shared is more applicable to batch processing where it would actually be useful to transcribe 375 seconds of audio in 1 second

C

cosbgn•7/25/25, 8:41 PM

I actually chunk it so I process only very short sentences

C

creet•7/25/25, 8:42 PM

yes exactly if you are processing chunks of speech as the user is talking you can still respond immediately with little to no delay

C

creet•7/25/25, 8:42 PM

but to answer your original question idk the latency of whisper with workers ai but its probably 'fast enough'

C

cosbgn•7/25/25, 8:42 PM

These are some stats of my setup:

[6:27:34 PM] read_form_data: 25.065ms
[6:27:39 PM] transcribe: 4.907s
[6:27:40 PM] generate_fist_text_chunk: 763.861ms
[6:27:41 PM] generate_speech_chunk_0: 1.476s
[6:27:42 PM] generate_speech_chunk_1: 901.312ms


[6:28:02 PM] read_form_data: 19.01ms
[6:28:06 PM] transcribe: 3.367s
[6:28:06 PM] generate_fist_text_chunk: 612.575ms
[6:28:07 PM] generate_speech_chunk_0: 1.148s
[6:28:09 PM] generate_speech_chunk_1: 1.428s
[6:28:10 PM] generate_speech_chunk_2: 1.582s
[6:28:12 PM] generate_speech_chunk_3: 1.267s


[6:28:26 PM] read_form_data: 5.596ms
[6:28:27 PM] transcribe: 1.593s
[6:28:28 PM] generate_fist_text_chunk: 798.88ms
[6:28:29 PM] generate_speech_chunk_0: 1.294s
[6:28:30 PM] generate_speech_chunk_1: 698.474ms
[6:28:31 PM] generate_speech_chunk_2: 920.976ms

[6:27:34 PM] read_form_data: 25.065ms
[6:27:39 PM] transcribe: 4.907s
[6:27:40 PM] generate_fist_text_chunk: 763.861ms
[6:27:41 PM] generate_speech_chunk_0: 1.476s
[6:27:42 PM] generate_speech_chunk_1: 901.312ms


[6:28:02 PM] read_form_data: 19.01ms
[6:28:06 PM] transcribe: 3.367s
[6:28:06 PM] generate_fist_text_chunk: 612.575ms
[6:28:07 PM] generate_speech_chunk_0: 1.148s
[6:28:09 PM] generate_speech_chunk_1: 1.428s
[6:28:10 PM] generate_speech_chunk_2: 1.582s
[6:28:12 PM] generate_speech_chunk_3: 1.267s


[6:28:26 PM] read_form_data: 5.596ms
[6:28:27 PM] transcribe: 1.593s
[6:28:28 PM] generate_fist_text_chunk: 798.88ms
[6:28:29 PM] generate_speech_chunk_0: 1.294s
[6:28:30 PM] generate_speech_chunk_1: 698.474ms
[6:28:31 PM] generate_speech_chunk_2: 920.976ms

[6:27:34 PM] read_form_data: 25.065ms
[6:27:39 PM] transcribe: 4.907s
[6:27:40 PM] generate_fist_text_chunk: 763.861ms
[6:27:41 PM] generate_speech_chunk_0: 1.476s
[6:27:42 PM] generate_speech_chunk_1: 901.312ms


[6:28:02 PM] read_form_data: 19.01ms
[6:28:06 PM] transcribe: 3.367s
[6:28:06 PM] generate_fist_text_chunk: 612.575ms
[6:28:07 PM] generate_speech_chunk_0: 1.148s
[6:28:09 PM] generate_speech_chunk_1: 1.428s
[6:28:10 PM] generate_speech_chunk_2: 1.582s
[6:28:12 PM] generate_speech_chunk_3: 1.267s


[6:28:26 PM] read_form_data: 5.596ms
[6:28:27 PM] transcribe: 1.593s
[6:28:28 PM] generate_fist_text_chunk: 798.88ms
[6:28:29 PM] generate_speech_chunk_0: 1.294s
[6:28:30 PM] generate_speech_chunk_1: 698.474ms
[6:28:31 PM] generate_speech_chunk_2: 920.976ms

[6:27:34 PM] read_form_data: 25.065ms
[6:27:39 PM] transcribe: 4.907s
[6:27:40 PM] generate_fist_text_chunk: 763.861ms
[6:27:41 PM] generate_speech_chunk_0: 1.476s
[6:27:42 PM] generate_speech_chunk_1: 901.312ms


[6:28:02 PM] read_form_data: 19.01ms
[6:28:06 PM] transcribe: 3.367s
[6:28:06 PM] generate_fist_text_chunk: 612.575ms
[6:28:07 PM] generate_speech_chunk_0: 1.148s
[6:28:09 PM] generate_speech_chunk_1: 1.428s
[6:28:10 PM] generate_speech_chunk_2: 1.582s
[6:28:12 PM] generate_speech_chunk_3: 1.267s


[6:28:26 PM] read_form_data: 5.596ms
[6:28:27 PM] transcribe: 1.593s
[6:28:28 PM] generate_fist_text_chunk: 798.88ms
[6:28:29 PM] generate_speech_chunk_0: 1.294s
[6:28:30 PM] generate_speech_chunk_1: 698.474ms
[6:28:31 PM] generate_speech_chunk_2: 920.976ms

C

cosbgn•7/25/25, 8:43 PM

transcribe is the one which takes the most. (These are using openai endpoints)

C

cosbgn•7/25/25, 8:44 PM

After

generate_speech_chunk_0

generate_speech_chunk_0

that is already sent back to the client, so the other chunks don't really matter

C

cosbgn•7/25/25, 8:44 PM

Ideally I would like to get transcribe to a few hundreds of milliseconds, not 5 seconds.

Ccosbgn These are some stats of my setup: ``` [6:27:34 PM] read_form_data: 25.065ms [6:...

I

Isaac McFadyen•7/25/25, 8:47 PM

How long is your audio?

C

cosbgn•7/25/25, 8:47 PM

a sentence

C

cosbgn•7/25/25, 8:47 PM

couple of seconds, like 10 seconds maybe

C

creet•7/26/25, 5:14 AM

Hmm yeah 2x transcription speed is pretty bad

C

creet•7/26/25, 5:14 AM

Make sure you’re using whisper v3 large turbo or tiny en

C

creet•7/26/25, 5:15 AM

Not the base whisper

Z

zsueo•7/29/25, 4:33 PM

hey im using llama-4-scout-17b-16e-instruct for tool calling and it seems to just get stuck in a call loop? like it just keeps calling the tool over and over

Z

zsueo•7/29/25, 6:54 PM

it seems to be doing this for both this and the hermes model

Z

zsueo•7/29/25, 6:54 PM

im sending in requests like this

{
  "messages": [
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-04bad465677b4783b6797433a8deb05b",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-5d73b086fe964ae2900cdd527a659f1f",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-0140345ec2584ae0b8d4d6d1e3c6585e",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "user",
      "content": "hey can you fetch data relating to joe?"
    },
    {
      "role": "system",
      "content": "You are a database assistant, you will query the database if a user requests information from it and respond with a human readable output, please only call the tool once and respond each time after a tool call, all tool responses have the role tool"
    }
  ],
  "max_tokens": 1024,
  "raw": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "Query Vector Database",
        "description": "Queries the Vector Database for legal documents matching the provided query",
        "parameters": {
          "type": "object",
          "properties": {
            "Query": {
              "type": "string",
              "description": "The query to the Vector db"
            }
          },
          "required": ["Query"]
        }
      }
    }
  ]
}

{
  "messages": [
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-04bad465677b4783b6797433a8deb05b",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-5d73b086fe964ae2900cdd527a659f1f",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-0140345ec2584ae0b8d4d6d1e3c6585e",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "user",
      "content": "hey can you fetch data relating to joe?"
    },
    {
      "role": "system",
      "content": "You are a database assistant, you will query the database if a user requests information from it and respond with a human readable output, please only call the tool once and respond each time after a tool call, all tool responses have the role tool"
    }
  ],
  "max_tokens": 1024,
  "raw": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "Query Vector Database",
        "description": "Queries the Vector Database for legal documents matching the provided query",
        "parameters": {
          "type": "object",
          "properties": {
            "Query": {
              "type": "string",
              "description": "The query to the Vector db"
            }
          },
          "required": ["Query"]
        }
      }
    }
  ]
}

{
  "messages": [
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-04bad465677b4783b6797433a8deb05b",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-5d73b086fe964ae2900cdd527a659f1f",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-0140345ec2584ae0b8d4d6d1e3c6585e",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "user",
      "content": "hey can you fetch data relating to joe?"
    },
    {
      "role": "system",
      "content": "You are a database assistant, you will query the database if a user requests information from it and respond with a human readable output, please only call the tool once and respond each time after a tool call, all tool responses have the role tool"
    }
  ],
  "max_tokens": 1024,
  "raw": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "Query Vector Database",
        "description": "Queries the Vector Database for legal documents matching the provided query",
        "parameters": {
          "type": "object",
          "properties": {
            "Query": {
              "type": "string",
              "description": "The query to the Vector db"
            }
          },
          "required": ["Query"]
        }
      }
    }
  ]
}

{
  "messages": [
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-04bad465677b4783b6797433a8deb05b",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-5d73b086fe964ae2900cdd527a659f1f",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "tool",
      "tool_call_id": "chatcmpl-tool-0140345ec2584ae0b8d4d6d1e3c6585e",
      "content": "Pretend you got databack this tool is still under construction"
    },
    {
      "role": "user",
      "content": "hey can you fetch data relating to joe?"
    },
    {
      "role": "system",
      "content": "You are a database assistant, you will query the database if a user requests information from it and respond with a human readable output, please only call the tool once and respond each time after a tool call, all tool responses have the role tool"
    }
  ],
  "max_tokens": 1024,
  "raw": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "Query Vector Database",
        "description": "Queries the Vector Database for legal documents matching the provided query",
        "parameters": {
          "type": "object",
          "properties": {
            "Query": {
              "type": "string",
              "description": "The query to the Vector db"
            }
          },
          "required": ["Query"]
        }
      }
    }
  ]
}

but it keeps trying to call the tool again

Z

zsueo•7/29/25, 7:16 PM

do i need to send the assistant querying the tool as part of the message history?

Similar Threads