Thank you very much for your attention, I appreciate that. So the issue was in the appearance of ro

PProman4713 🇵🇸Thank you very much for your attention, I appreciate that. So the issue was in ...

S

samjs•9/12/25, 2:40 PM

So the issue was in the appearance of role "tool" after role "user", not in the message object itself? Alright, I guess.

Supposedly, yes. Although it's not obvious to me what you would do differently. Searching for that error message comes up with similar github issues though: https://github.com/langchain-ai/langchain/issues/22928

Ssamjs > So the issue was in the appearance of role "tool" after role "user", not in th...

P

Proman4713 🇵🇸OP•9/12/25, 2:43 PM

I read the docs, and apparently it's pushing the

response.message

response.message

response.message

response.message string to the messages array, then after executing the tools adds the

{ role: "tool" }

{ role: "tool" }

{ role: "tool" }

{ role: "tool" } object(s)... But when the LLM chooses a tool to call it doesn't give a message in the first place

H

Himanshu•9/12/25, 6:01 PM

I've successfully integrated Aura and Nova models from Deepgram into OpenGPT. For anyone interested, you can try it here: https://opengpt.weblinq.dev/

For those interested in the implementation, you can check the source code: https://github.com/devhims/opengpt

P

Proman4713 🇵🇸OP•9/12/25, 8:32 PM

@samjs i did as they said and provided the assistant's response before running the tools. it didnt work:

[
  {
    "role": "system",
    "content": "My system prompt goes here."
  },
  {
      "role": "user",
      "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
      "role": "assistant",
      "content": "",
      "tool_calls": [{
          "name": "getClassRoster",
          "arguments": {
              "grade": 10,
              "classLetter": "A"
          }
      }]
  },
  {
    "role": "tool",
    "tool_call_id": "000000001",
    "name": "getClassRoster",
    "content": "{\"data\":[...TheResults]}"
  }
]

[
  {
    "role": "system",
    "content": "My system prompt goes here."
  },
  {
      "role": "user",
      "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
      "role": "assistant",
      "content": "",
      "tool_calls": [{
          "name": "getClassRoster",
          "arguments": {
              "grade": 10,
              "classLetter": "A"
          }
      }]
  },
  {
    "role": "tool",
    "tool_call_id": "000000001",
    "name": "getClassRoster",
    "content": "{\"data\":[...TheResults]}"
  }
]

[
  {
    "role": "system",
    "content": "My system prompt goes here."
  },
  {
      "role": "user",
      "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
      "role": "assistant",
      "content": "",
      "tool_calls": [{
          "name": "getClassRoster",
          "arguments": {
              "grade": 10,
              "classLetter": "A"
          }
      }]
  },
  {
    "role": "tool",
    "tool_call_id": "000000001",
    "name": "getClassRoster",
    "content": "{\"data\":[...TheResults]}"
  }
]

[
  {
    "role": "system",
    "content": "My system prompt goes here."
  },
  {
      "role": "user",
      "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
      "role": "assistant",
      "content": "",
      "tool_calls": [{
          "name": "getClassRoster",
          "arguments": {
              "grade": 10,
              "classLetter": "A"
          }
      }]
  },
  {
    "role": "tool",
    "tool_call_id": "000000001",
    "name": "getClassRoster",
    "content": "{\"data\":[...TheResults]}"
  }
]

and i get this error:

[wrangler:error] AiError: 3030: 3 validation errors for ValidatorIterator
0.id
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
0.function
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
0.type
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing

[wrangler:error] AiError: 3030: 3 validation errors for ValidatorIterator
0.id
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
0.function
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
0.type
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing

[wrangler:error] AiError: 3030: 3 validation errors for ValidatorIterator
0.id
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
0.function
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
0.type
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing

[wrangler:error] AiError: 3030: 3 validation errors for ValidatorIterator
0.id
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
0.function
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
0.type
  Field required [type=missing, input_value={'name': 'getClassRoster'...10, 'classLetter': 'A'}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing

the link didnt give me any new info and the error doesnt tell me where these fields are missing

P

Proman4713 🇵🇸OP•9/12/25, 10:08 PM

@samjs is it just me or is cloudflare's schema pretty much unrelated to what the model wants?

P

Proman4713 🇵🇸OP•9/12/25, 10:08 PM

I'm not even supposed to have "name" in there

P

Proman4713 🇵🇸OP•9/12/25, 10:08 PM

mistral's videos say they want name, but not tool_call_id

P

Proman4713 🇵🇸OP•9/12/25, 10:10 PM

and this shows that:

P

Proman4713 🇵🇸OP•9/12/25, 10:11 PM

P

Proman4713 🇵🇸OP•9/12/25, 10:14 PM

apparently, I'm guessing it's checking the old reply from the assistant:

{
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "name": "getClassRoster",
        "arguments": {
          "grade": 10,
          "classLetter": "A"
        }
      }
    ]
  },

{
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "name": "getClassRoster",
        "arguments": {
          "grade": 10,
          "classLetter": "A"
        }
      }
    ]
  },

{
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "name": "getClassRoster",
        "arguments": {
          "grade": 10,
          "classLetter": "A"
        }
      }
    ]
  },

{
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "name": "getClassRoster",
        "arguments": {
          "grade": 10,
          "classLetter": "A"
        }
      }
    ]
  },

it's asking for type, function, and id, but it didnt provide these things when it first gave this result!

P

Proman4713 🇵🇸OP•9/12/25, 10:15 PM

fine, i'm converting it myself

P

Proman4713 🇵🇸OP•9/12/25, 10:15 PM

just get this thing to work

P

Proman4713 🇵🇸OP•9/12/25, 10:16 PM

{
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "type": "function",
        "function": {
          "name": "getClassRoster",
          "parameters": {
            "grade": 10,
            "classLetter": "A"
          }
        },
        "id": "000000001"
      }
    ]
  },

{
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "type": "function",
        "function": {
          "name": "getClassRoster",
          "parameters": {
            "grade": 10,
            "classLetter": "A"
          }
        },
        "id": "000000001"
      }
    ]
  },

{
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "type": "function",
        "function": {
          "name": "getClassRoster",
          "parameters": {
            "grade": 10,
            "classLetter": "A"
          }
        },
        "id": "000000001"
      }
    ]
  },

{
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "type": "function",
        "function": {
          "name": "getClassRoster",
          "parameters": {
            "grade": 10,
            "classLetter": "A"
          }
        },
        "id": "000000001"
      }
    ]
  },

oh GOD this worked and now it wants me to turn parameters into arguments

P

Proman4713 🇵🇸OP•9/12/25, 10:16 PM

why is this??

P

Proman4713 🇵🇸OP•9/12/25, 10:18 PM

oh god and now we're back at internal server error 3043

P

Proman4713 🇵🇸OP•9/12/25, 10:18 PM

I'm continuing in workers-aiany news?

P

Proman4713 🇵🇸OP•9/12/25, 10:18 PM

I'm flooding too much

O

omgwot•9/13/25, 10:03 AM

hey all, i'm getting 400s when using

@cf/openai/gpt-oss-120b

@cf/openai/gpt-oss-120b

@cf/openai/gpt-oss-120b

@cf/openai/gpt-oss-120b. is this a known thing? using the completions API:

import { createClient as createOpenAIClient } from "./client";
import type { ChatCompletionMessageParam } from "openai/resources/chat/completions";

export async function callCloudflareAI(
  model: string,
  messages: ChatCompletionMessageParam[],
  options: {
    temperature?: number;
    max_tokens?: number;
  } = {}
): Promise<string> {
  const client = createOpenAIClient();

  const response = await client.chat.completions.create({
    model,
    messages,
    temperature: options.temperature,
    max_tokens: options.max_tokens
  });

  return response.choices[0]?.message?.content || "";
}

import { createClient as createOpenAIClient } from "./client";
import type { ChatCompletionMessageParam } from "openai/resources/chat/completions";

export async function callCloudflareAI(
  model: string,
  messages: ChatCompletionMessageParam[],
  options: {
    temperature?: number;
    max_tokens?: number;
  } = {}
): Promise<string> {
  const client = createOpenAIClient();

  const response = await client.chat.completions.create({
    model,
    messages,
    temperature: options.temperature,
    max_tokens: options.max_tokens
  });

  return response.choices[0]?.message?.content || "";
}

import { createClient as createOpenAIClient } from "./client";
import type { ChatCompletionMessageParam } from "openai/resources/chat/completions";

export async function callCloudflareAI(
  model: string,
  messages: ChatCompletionMessageParam[],
  options: {
    temperature?: number;
    max_tokens?: number;
  } = {}
): Promise<string> {
  const client = createOpenAIClient();

  const response = await client.chat.completions.create({
    model,
    messages,
    temperature: options.temperature,
    max_tokens: options.max_tokens
  });

  return response.choices[0]?.message?.content || "";
}

import { createClient as createOpenAIClient } from "./client";
import type { ChatCompletionMessageParam } from "openai/resources/chat/completions";

export async function callCloudflareAI(
  model: string,
  messages: ChatCompletionMessageParam[],
  options: {
    temperature?: number;
    max_tokens?: number;
  } = {}
): Promise<string> {
  const client = createOpenAIClient();

  const response = await client.chat.completions.create({
    model,
    messages,
    temperature: options.temperature,
    max_tokens: options.max_tokens
  });

  return response.choices[0]?.message?.content || "";
}

should I be using responses instead? thanks.

Oomgwot hey all, i'm getting 400s when using `@cf/openai/gpt-oss-120b`. is this a known ...

H

Himanshu•9/13/25, 10:41 AM

You're right, gpt-oss models support the responses API not the completions API. Try using that.

O

omgwot•9/13/25, 10:41 AM

feels like this should all just work seamlessly

HHimanshu I've successfully integrated Aura and Nova models from Deepgram into OpenGPT. F...

T

TuChutya•9/14/25, 4:06 AM

Wao

T

Tim•9/14/25, 10:14 PM

Hi, in relation to the deepgram tools Nova-3 and aura, are they wrappers connecting to deepgrams infrastructure? Or are you using the self-hosted platform and hosting it yourself?

TTim Hi, in relation to the deepgram tools Nova-3 and aura, are they wrappers connect...

I

Isaac McFadyen•9/14/25, 11:34 PM

Per the blog https://blog.cloudflare.com/workers-ai-partner-models/ it's on Cloudflare's own infra.

I

Isaac McFadyen•9/14/25, 11:35 PM

[...] partnered with Leonardo.Ai and Deepgram to bring their latest and greatest models to Workers AI, hosted on Cloudflare’s infrastructure

T

toadilywasted•9/16/25, 3:16 AM

What's the pricing for embeddinggemma-300m ? I can't find any on the page

R

robinsloan•9/16/25, 5:04 PM

I'm having the same issue exactly that @Object documented here, back in May 2025: workers-ai

Any ideas from the CF team?

H

Himanshu•9/16/25, 7:13 PM

I'm trying to use Cloudflare-hosted GPT OSS models with AI SDK v5. I integrated workers-ai-provider, but it doesn't seem to support these models, as I'm getting empty responses. @Sunil Pai any input?

HHimanshu I'm trying to use Cloudflare-hosted GPT OSS models with AI SDK v5. I integrated ...

I

Isaac McFadyen•9/16/25, 7:45 PM

Please do not ping people.

S

SuperHelpflare•9/16/25, 7:45 PM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

A

ajgeiss0702•9/17/25, 3:22 AM

:/

Aajgeiss0702 :/

S

samjs•9/17/25, 4:19 PM

Hey @ajgeiss0702 . Which gpt model is that? And can you share your account ID with me (by DM) and I can look into this for you.

Ssamjs Hey @ajgeiss0702 . Which gpt model is that? And can you share your account ID wi...

A

ajgeiss0702•9/17/25, 5:01 PM

its the 20b. my account id is f55b85c8a963663b11036975203c63c0

A

ajgeiss0702•9/17/25, 6:05 PM

ive been watching the ai gateway, and most errors are either capacity or something like

AiError: AiError: unexpected tokens remaining in message header: [\"……………...]???…….……...…..\", \"………...…ż…………..………………….………………………..\", \"...??…\",

AiError: AiError: unexpected tokens remaining in message header: [\"……………...]???…….……...…..\", \"………...…ż…………..………………….………………………..\", \"...??…\",

AiError: AiError: unexpected tokens remaining in message header: [\"……………...]???…….……...…..\", \"………...…ż…………..………………….………………………..\", \"...??…\",

AiError: AiError: unexpected tokens remaining in message header: [\"……………...]???…….……...…..\", \"………...…ż…………..………………….………………………..\", \"...??…\",

.... and it goes on for a while until it looks like some of the strings are words from thoughts?

A

ajgeiss0702•9/17/25, 6:06 PM

my retry logic automatically tries the 120b model if the 20b model fails after 8 retries, and that works first time every time ive seen

Aajgeiss0702 :/

.

.r20•9/18/25, 2:41 AM

yeah im running into the same issue as well im getting an absolute ton of 429 exceeded capacity errors when i try to use workersai

.

.r20•9/18/25, 2:44 AM

not sure if this is temporary since i only started using it today but its been like this all day

..r20 yeah im running into the same issue as well im getting an absolute ton of 429 ex...

A

ajgeiss0702•9/18/25, 2:45 AM

usually bge-m3 is fine for me, but I have seen a few errors with it today. probably just temporary

A

ajgeiss0702•9/18/25, 2:45 AM

capcity errors are pretty common on workers ai, especially with what i assume are the more popular models

.

.r20•9/18/25, 2:46 AM

ah i assume due to autoscaling serverless

V

Vikash•9/18/25, 5:32 AM

Getting too many errors: AiError: 3040: Capacity temporarily exceeded, please try again.

const embeddings: any = await env.AI.run("@cf/baai/bge-m3", {
    text: [prompt],
  });

const embeddings: any = await env.AI.run("@cf/baai/bge-m3", {
    text: [prompt],
  });

const embeddings: any = await env.AI.run("@cf/baai/bge-m3", {
    text: [prompt],
  });

const embeddings: any = await env.AI.run("@cf/baai/bge-m3", {
    text: [prompt],
  });

..r20 yeah im running into the same issue as well im getting an absolute ton of 429 ex...

V

Vikash•9/18/25, 5:42 AM

Hey, looks like you are using a gateway, how can I change my code to also use a gateway instead the direct reqeust?

const embeddings: any = await env.AI.run("@cf/baai/bge-m3", {
text: [text],
});

F

fortytwo•9/18/25, 6:15 AM

Can a folder within a bucket be used as a RAG source? Or do I need to create separate buckets for independent rag data?

VVikash Hey, looks like you are using a gateway, how can I change my code to also use a ...

.

.r20•9/18/25, 6:22 AM

https://developers.cloudflare.com/ai-gateway/usage/providers/workersai/

Cloudflare Docs

Workers AI

Use AI Gateway for analytics, caching, and security on requests to Workers AI. Workers AI integrates seamlessly with AI Gateway, allowing you to execute AI inference via API requests or through an environment binding for Workers scripts. The binding simplifies the process by routing requests through your AI Gateway with minimal setup.

L

Lami•9/18/25, 10:29 AM

@cf/google/embeddinggemma-300m

@cf/google/embeddinggemma-300m

@cf/google/embeddinggemma-300m

@cf/google/embeddinggemma-300m Does anyone know the price of this model?

VVikash Getting too many errors: AiError: 3040: Capacity temporarily exceeded, please t...

S

samjs•9/18/25, 3:26 PM

Hey @Vikash and others. Apologies for the 3040 errors you all are seeing. We're aware of these issues (we get alerted when they happen) and we're working to (a) provision more capacity for those models, and (b) implement some additional measures to keep capacity available even when we're getting flooded with requests. Appreciate your patience and thanks for flagging these issues!

Ffortytwo Can a folder within a bucket be used as a RAG source? Or do I need to create sep...

S

samjs•9/18/25, 6:15 PM

I believe we only support using the entire bucket. #autorag channel might know more about this however!

Ssamjs I believe we only support using the entire bucket. #autorag channel might know ...

F

fortytwo•9/18/25, 6:23 PM

Ah I missed that space. Thanks!

R

Razmjoo•9/18/25, 6:36 PM

Hi, is there any outages with AI Workers? I am getting a lot of errors on text-embeddings workers (using bge-m3) cc: @samjs

{
  "errors": [
    {
      "message": "AiError: AiError: model temporarily unavailable (58da3f75-5ff1-46ba-a45e-a59d4ada1c33)",
      "code": 9000
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

{
  "errors": [
    {
      "message": "AiError: AiError: model temporarily unavailable (58da3f75-5ff1-46ba-a45e-a59d4ada1c33)",
      "code": 9000
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

{
  "errors": [
    {
      "message": "AiError: AiError: model temporarily unavailable (58da3f75-5ff1-46ba-a45e-a59d4ada1c33)",
      "code": 9000
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

{
  "errors": [
    {
      "message": "AiError: AiError: model temporarily unavailable (58da3f75-5ff1-46ba-a45e-a59d4ada1c33)",
      "code": 9000
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

Ssamjs Hey @Vikash and others. Apologies for the 3040 errors you all are seeing. We're ...

I

Isaac McFadyen•9/18/25, 6:38 PM

@Razmjoo see this message

Thank you very much for your attention, I appreciate that. So the issue was in the appearance of ro

Similar Threads