any news?

S

samjs•9/9/25, 3:01 PM

Back in the office today and looking into this!

S

samjs•9/9/25, 3:09 PM

Are you able to share the input with me so I can test it directly?

Ssamjs Are you able to share the input with me so I can test it directly?

P

Proman4713 🇵🇸OP•9/9/25, 4:57 PM

alright

P

Proman4713 🇵🇸OP•9/9/25, 4:59 PM

const messages = [ { role: "user", content: "How are you?" } ];

const tools = AITools;

const result = await AI.run("@cf/meta/llama-4-scout-17b-16e-instruct", {
    messages,
    tools,
    max_tokens: 512
});

const messages = [ { role: "user", content: "How are you?" } ];

const tools = AITools;

const result = await AI.run("@cf/meta/llama-4-scout-17b-16e-instruct", {
    messages,
    tools,
    max_tokens: 512
});

const messages = [ { role: "user", content: "How are you?" } ];

const tools = AITools;

const result = await AI.run("@cf/meta/llama-4-scout-17b-16e-instruct", {
    messages,
    tools,
    max_tokens: 512
});

const messages = [ { role: "user", content: "How are you?" } ];

const tools = AITools;

const result = await AI.run("@cf/meta/llama-4-scout-17b-16e-instruct", {
    messages,
    tools,
    max_tokens: 512
});

/**
 * 
 * @type {import("@cloudflare/workers-types").Ai_Cf_Meta_Llama_4_Messages["tools"]}
 */
export const AITools = [
    {
        name: "getClassRoster",
        description: "Returns all student objects for that class",
        parameters: {
            type: "object",
            properties: {
                grade: {
                    type: "number",
                    description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                },
                classLetter: {
                    type: "string",
                    description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                }
            },
            required: ["grade", "classLetter"]
        }
    }
]

/**
 * 
 * @type {import("@cloudflare/workers-types").Ai_Cf_Meta_Llama_4_Messages["tools"]}
 */
export const AITools = [
    {
        name: "getClassRoster",
        description: "Returns all student objects for that class",
        parameters: {
            type: "object",
            properties: {
                grade: {
                    type: "number",
                    description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                },
                classLetter: {
                    type: "string",
                    description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                }
            },
            required: ["grade", "classLetter"]
        }
    }
]

/**
 * 
 * @type {import("@cloudflare/workers-types").Ai_Cf_Meta_Llama_4_Messages["tools"]}
 */
export const AITools = [
    {
        name: "getClassRoster",
        description: "Returns all student objects for that class",
        parameters: {
            type: "object",
            properties: {
                grade: {
                    type: "number",
                    description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                },
                classLetter: {
                    type: "string",
                    description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                }
            },
            required: ["grade", "classLetter"]
        }
    }
]

/**
 * 
 * @type {import("@cloudflare/workers-types").Ai_Cf_Meta_Llama_4_Messages["tools"]}
 */
export const AITools = [
    {
        name: "getClassRoster",
        description: "Returns all student objects for that class",
        parameters: {
            type: "object",
            properties: {
                grade: {
                    type: "number",
                    description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                },
                classLetter: {
                    type: "string",
                    description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                }
            },
            required: ["grade", "classLetter"]
        }
    }
]

P

Proman4713 🇵🇸OP•9/9/25, 4:59 PM

@samjs there you go

S

samjs•9/9/25, 5:00 PM

perfect, thank you!

Ssamjs perfect, thank you!

P

Proman4713 🇵🇸OP•9/9/25, 5:00 PM

You're welcome... Let me know if you need anything else

S

samjs•9/10/25, 3:33 PM

Hey @Proman4713

. Sorry for the delay in chasing it down. As far as I can tell, the issue is in our type schemas. If I modify your AITools constant to:

export const AITools: Ai_Cf_Meta_Llama_4_Messages["tools"] = [
    {
        type: "function",
        function: {
            name: "getClassRoster",
            description: "Returns all student objects for that class",
            parameters: {
                type: "object",
                properties: {
                    grade: {
                        type: "number",
                        description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                    },
                    classLetter: {
                        type: "string",
                        description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                    }
                },
                required: ["grade", "classLetter"]
            }
        }
    }
]

export const AITools: Ai_Cf_Meta_Llama_4_Messages["tools"] = [
    {
        type: "function",
        function: {
            name: "getClassRoster",
            description: "Returns all student objects for that class",
            parameters: {
                type: "object",
                properties: {
                    grade: {
                        type: "number",
                        description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                    },
                    classLetter: {
                        type: "string",
                        description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                    }
                },
                required: ["grade", "classLetter"]
            }
        }
    }
]

export const AITools: Ai_Cf_Meta_Llama_4_Messages["tools"] = [
    {
        type: "function",
        function: {
            name: "getClassRoster",
            description: "Returns all student objects for that class",
            parameters: {
                type: "object",
                properties: {
                    grade: {
                        type: "number",
                        description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                    },
                    classLetter: {
                        type: "string",
                        description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                    }
                },
                required: ["grade", "classLetter"]
            }
        }
    }
]

export const AITools: Ai_Cf_Meta_Llama_4_Messages["tools"] = [
    {
        type: "function",
        function: {
            name: "getClassRoster",
            description: "Returns all student objects for that class",
            parameters: {
                type: "object",
                properties: {
                    grade: {
                        type: "number",
                        description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                    },
                    classLetter: {
                        type: "string",
                        description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                    }
                },
                required: ["grade", "classLetter"]
            }
        }
    }
]

it seems to work for me. Can you try that out?

I'll file an issue internally about the incorrect schema.

S

samjs•9/10/25, 3:42 PM

That's also why the error was so bad :/ typically our schema validation will catch these problems sooner and return nicer errors.

Ssamjs Hey @Proman4713 🇵🇸 . Sorry for the delay in chasing it down. As far as I can t...

P

Proman4713 🇵🇸OP•9/10/25, 7:09 PM

I'll try it out in a few minutes, sorry, I was offline this morning and didn't get your notification

Ssamjs That's also why the error was so bad :/ typically our schema validation will cat...

P

Proman4713 🇵🇸OP•9/10/25, 7:11 PM

Aha, well, thanks for your attention and effort! Let me know if the error gets fixed... I'll try your alternate schema in a minute

P

Proman4713 🇵🇸OP•9/10/25, 7:35 PM

@samjs this didn't give any errors! I just had to use toolCall.function.name rather than toolCall.name!

P

Proman4713 🇵🇸OP•9/10/25, 7:38 PM

[
  {
    "role": "user",
    "content": "Who are the students of class 10/A"
  },
  {
    "role": "tool",
    "name": "getClassRoster",
    "content": "[{\"englishName\":\"Aaser Mohammed Ahmed\",\"arabicName\":\"آسر محمد أحمد\",\"gender\":\"Male\",\"religion\":\"Muslim\",\"studentId\":\"0123456789\"}]"
  },
  {
    "role": "assistant",
    "content": "The students of class 10/A are:\n\n* Aaser Mohammed Ahmed \n\nWould you like to know more about their attendance?"
  }
]

[
  {
    "role": "user",
    "content": "Who are the students of class 10/A"
  },
  {
    "role": "tool",
    "name": "getClassRoster",
    "content": "[{\"englishName\":\"Aaser Mohammed Ahmed\",\"arabicName\":\"آسر محمد أحمد\",\"gender\":\"Male\",\"religion\":\"Muslim\",\"studentId\":\"0123456789\"}]"
  },
  {
    "role": "assistant",
    "content": "The students of class 10/A are:\n\n* Aaser Mohammed Ahmed \n\nWould you like to know more about their attendance?"
  }
]

[
  {
    "role": "user",
    "content": "Who are the students of class 10/A"
  },
  {
    "role": "tool",
    "name": "getClassRoster",
    "content": "[{\"englishName\":\"Aaser Mohammed Ahmed\",\"arabicName\":\"آسر محمد أحمد\",\"gender\":\"Male\",\"religion\":\"Muslim\",\"studentId\":\"0123456789\"}]"
  },
  {
    "role": "assistant",
    "content": "The students of class 10/A are:\n\n* Aaser Mohammed Ahmed \n\nWould you like to know more about their attendance?"
  }
]

[
  {
    "role": "user",
    "content": "Who are the students of class 10/A"
  },
  {
    "role": "tool",
    "name": "getClassRoster",
    "content": "[{\"englishName\":\"Aaser Mohammed Ahmed\",\"arabicName\":\"آسر محمد أحمد\",\"gender\":\"Male\",\"religion\":\"Muslim\",\"studentId\":\"0123456789\"}]"
  },
  {
    "role": "assistant",
    "content": "The students of class 10/A are:\n\n* Aaser Mohammed Ahmed \n\nWould you like to know more about their attendance?"
  }
]

thank you for your help!

P

Proman4713 🇵🇸OP•9/11/25, 9:58 AM

uhh

P

Proman4713 🇵🇸OP•9/11/25, 9:58 AM

sorry for disturbing you again, but I'm seeing this:

export const AITools = [{
    type: "function",
    function: {
        name: "getClassRoster",
        description: "Returns all student objects for that class. If you get students with 'Student {x}' English names, then use their Arabic names.",
        parameters: {
            type: "object",
            properties: {
                grade: {
                    type: "number",
                    description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                },
                classLetter: {
                    type: "string",
                    description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                }
            },
            required: ["grade", "classLetter"]
        }
    }
}, {
    type: "function",
    function: {
        name: "getAttendanceForDate",
        description: "Returns attendance for all periods on a given day for all classes. You would need the required class roster(s) first to know which students are in which classes.",
        parameters: {
            type: "object",
            properties: {
                date: {
                    type: "string",
                    description: "The date to get attendance for. Must be in the format DD/MM/YYYY."
                }
            },
            required: ["date"]
        }
    }
}]

export const AITools = [{
    type: "function",
    function: {
        name: "getClassRoster",
        description: "Returns all student objects for that class. If you get students with 'Student {x}' English names, then use their Arabic names.",
        parameters: {
            type: "object",
            properties: {
                grade: {
                    type: "number",
                    description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                },
                classLetter: {
                    type: "string",
                    description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                }
            },
            required: ["grade", "classLetter"]
        }
    }
}, {
    type: "function",
    function: {
        name: "getAttendanceForDate",
        description: "Returns attendance for all periods on a given day for all classes. You would need the required class roster(s) first to know which students are in which classes.",
        parameters: {
            type: "object",
            properties: {
                date: {
                    type: "string",
                    description: "The date to get attendance for. Must be in the format DD/MM/YYYY."
                }
            },
            required: ["date"]
        }
    }
}]

export const AITools = [{
    type: "function",
    function: {
        name: "getClassRoster",
        description: "Returns all student objects for that class. If you get students with 'Student {x}' English names, then use their Arabic names.",
        parameters: {
            type: "object",
            properties: {
                grade: {
                    type: "number",
                    description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                },
                classLetter: {
                    type: "string",
                    description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                }
            },
            required: ["grade", "classLetter"]
        }
    }
}, {
    type: "function",
    function: {
        name: "getAttendanceForDate",
        description: "Returns attendance for all periods on a given day for all classes. You would need the required class roster(s) first to know which students are in which classes.",
        parameters: {
            type: "object",
            properties: {
                date: {
                    type: "string",
                    description: "The date to get attendance for. Must be in the format DD/MM/YYYY."
                }
            },
            required: ["date"]
        }
    }
}]

export const AITools = [{
    type: "function",
    function: {
        name: "getClassRoster",
        description: "Returns all student objects for that class. If you get students with 'Student {x}' English names, then use their Arabic names.",
        parameters: {
            type: "object",
            properties: {
                grade: {
                    type: "number",
                    description: "The grade (e.g. 10) of the needed class. Between 1 and 12, inclusive."
                },
                classLetter: {
                    type: "string",
                    description: "The letter (e.g. A) of the needed class. Between A and Z, inclusive."
                }
            },
            required: ["grade", "classLetter"]
        }
    }
}, {
    type: "function",
    function: {
        name: "getAttendanceForDate",
        description: "Returns attendance for all periods on a given day for all classes. You would need the required class roster(s) first to know which students are in which classes.",
        parameters: {
            type: "object",
            properties: {
                date: {
                    type: "string",
                    description: "The date to get attendance for. Must be in the format DD/MM/YYYY."
                }
            },
            required: ["date"]
        }
    }
}]

the AI will return the getClassRoster function in tool_calls fine, but if it needs the getAttendanceForDate function, it doesnt return anything in tool_calls and simply says what it needs to do in text

P

Proman4713 🇵🇸OP•9/11/25, 9:59 AM

like this

P

Proman4713 🇵🇸OP•9/11/25, 9:59 AM

Initial AI result: {
  "response": "To answer your question, I need to follow these steps:\n\n1. Get the roster for class 10/A.\n2. Get attendance for class 10/A on the 1st of September, 2025.\n3. Get attendance for class 10/A on the 31st of August, 2025.\n\nFirst, I will get the roster for class 10/A. \n{\"name\": \"getClassRoster\", \"parameters\": {\"grade\": 10, \"classLetter\": \"A\"}}\n\nThen, I will get attendance for the 1st of September, 2025. \n{\"name\": \"getAttendanceForDate\", \"parameters\": {\"date\": \"01/09/2025\"}}\n\nLastly, I will get attendance for the 31st of August, 2025. \n{\"name\": \"getAttendanceForDate\", \"parameters\": {\"date\": \"31/08/2025\"}}",
  "tool_calls": [],
  "usage": {
    "prompt_tokens": 646,
    "completion_tokens": 191,
    "total_tokens": 837
  }
}

Initial AI result: {
  "response": "To answer your question, I need to follow these steps:\n\n1. Get the roster for class 10/A.\n2. Get attendance for class 10/A on the 1st of September, 2025.\n3. Get attendance for class 10/A on the 31st of August, 2025.\n\nFirst, I will get the roster for class 10/A. \n{\"name\": \"getClassRoster\", \"parameters\": {\"grade\": 10, \"classLetter\": \"A\"}}\n\nThen, I will get attendance for the 1st of September, 2025. \n{\"name\": \"getAttendanceForDate\", \"parameters\": {\"date\": \"01/09/2025\"}}\n\nLastly, I will get attendance for the 31st of August, 2025. \n{\"name\": \"getAttendanceForDate\", \"parameters\": {\"date\": \"31/08/2025\"}}",
  "tool_calls": [],
  "usage": {
    "prompt_tokens": 646,
    "completion_tokens": 191,
    "total_tokens": 837
  }
}

Initial AI result: {
  "response": "To answer your question, I need to follow these steps:\n\n1. Get the roster for class 10/A.\n2. Get attendance for class 10/A on the 1st of September, 2025.\n3. Get attendance for class 10/A on the 31st of August, 2025.\n\nFirst, I will get the roster for class 10/A. \n{\"name\": \"getClassRoster\", \"parameters\": {\"grade\": 10, \"classLetter\": \"A\"}}\n\nThen, I will get attendance for the 1st of September, 2025. \n{\"name\": \"getAttendanceForDate\", \"parameters\": {\"date\": \"01/09/2025\"}}\n\nLastly, I will get attendance for the 31st of August, 2025. \n{\"name\": \"getAttendanceForDate\", \"parameters\": {\"date\": \"31/08/2025\"}}",
  "tool_calls": [],
  "usage": {
    "prompt_tokens": 646,
    "completion_tokens": 191,
    "total_tokens": 837
  }
}

Initial AI result: {
  "response": "To answer your question, I need to follow these steps:\n\n1. Get the roster for class 10/A.\n2. Get attendance for class 10/A on the 1st of September, 2025.\n3. Get attendance for class 10/A on the 31st of August, 2025.\n\nFirst, I will get the roster for class 10/A. \n{\"name\": \"getClassRoster\", \"parameters\": {\"grade\": 10, \"classLetter\": \"A\"}}\n\nThen, I will get attendance for the 1st of September, 2025. \n{\"name\": \"getAttendanceForDate\", \"parameters\": {\"date\": \"01/09/2025\"}}\n\nLastly, I will get attendance for the 31st of August, 2025. \n{\"name\": \"getAttendanceForDate\", \"parameters\": {\"date\": \"31/08/2025\"}}",
  "tool_calls": [],
  "usage": {
    "prompt_tokens": 646,
    "completion_tokens": 191,
    "total_tokens": 837
  }
}

and it doesnt even try to run tool calls

P

Proman4713 🇵🇸OP•9/11/25, 10:05 AM

https://github.com/ggml-org/llama.cpp/issues/14697 nevermind, it seems to be an issue with the LLM itself

GitHub

Eval bug: Regression: Tool calls still returned in content field as...

Name and Version Environment ./llama-server --version version: 5894 (494c589) built with Apple clang version 17.0.0 (clang-1700.0.13.5) for arm64-apple-darwin24.5.0 Server Configuration export LLAM...

P

Proman4713 🇵🇸OP•9/11/25, 10:29 AM

uhhh i found other models that work with function calling

P

Proman4713 🇵🇸OP•9/11/25, 10:29 AM

will they use the same schema?

P

Proman4713 🇵🇸OP•9/11/25, 10:36 AM

they do, it worked, thanks

S

samjs•9/11/25, 1:28 PM

yep, exactly

Ssamjs yep, exactly

P

Proman4713 🇵🇸OP•9/11/25, 1:49 PM

yeah, and it feels so fricking fun to see the AI respond based on data from my application

P

Proman4713 🇵🇸OP•9/12/25, 10:20 PM

[
  {
    "role": "system",
    "content": "System prompt goes here"
  },
  {
    "role": "user",
    "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "type": "function",
        "function": {
          "name": "getClassRoster",
          "arguments": "{\"grade\":10,\"classLetter\":\"A\"}"
        },
        "id": "000000001"
      }
    ]
  },
  {
    "role": "tool",
    "name": "getClassRoster",
    "content": "{ ...stuffHere }"
  }
]

[
  {
    "role": "system",
    "content": "System prompt goes here"
  },
  {
    "role": "user",
    "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "type": "function",
        "function": {
          "name": "getClassRoster",
          "arguments": "{\"grade\":10,\"classLetter\":\"A\"}"
        },
        "id": "000000001"
      }
    ]
  },
  {
    "role": "tool",
    "name": "getClassRoster",
    "content": "{ ...stuffHere }"
  }
]

[
  {
    "role": "system",
    "content": "System prompt goes here"
  },
  {
    "role": "user",
    "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "type": "function",
        "function": {
          "name": "getClassRoster",
          "arguments": "{\"grade\":10,\"classLetter\":\"A\"}"
        },
        "id": "000000001"
      }
    ]
  },
  {
    "role": "tool",
    "name": "getClassRoster",
    "content": "{ ...stuffHere }"
  }
]

[
  {
    "role": "system",
    "content": "System prompt goes here"
  },
  {
    "role": "user",
    "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "type": "function",
        "function": {
          "name": "getClassRoster",
          "arguments": "{\"grade\":10,\"classLetter\":\"A\"}"
        },
        "id": "000000001"
      }
    ]
  },
  {
    "role": "tool",
    "name": "getClassRoster",
    "content": "{ ...stuffHere }"
  }
]

I manipulated the initial result myself, and YET

P

Proman4713 🇵🇸OP•9/12/25, 10:20 PM

[wrangler:error] AiError: 3043: Internal server error
    at Ai._parseError (cloudflare-internal:ai-api:212:24)
    at async Ai.run (cloudflare-internal:ai-api:190:19)
    at async runWorker (file:///home/proman4713/JavaScript%20Projects/MawjoodAPI/src/index.js:50:20)
    at async jsonError (file:///home/proman4713/.nvm/versions/node/v22.13.1/lib/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)
    at async drainBody (file:///home/proman4713/.nvm/versions/node/v22.13.1/lib/node_modules/wrangler/templates/middleware/middleware-ensure-req-body-drained.ts:5:10)

[wrangler:error] AiError: 3043: Internal server error
    at Ai._parseError (cloudflare-internal:ai-api:212:24)
    at async Ai.run (cloudflare-internal:ai-api:190:19)
    at async runWorker (file:///home/proman4713/JavaScript%20Projects/MawjoodAPI/src/index.js:50:20)
    at async jsonError (file:///home/proman4713/.nvm/versions/node/v22.13.1/lib/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)
    at async drainBody (file:///home/proman4713/.nvm/versions/node/v22.13.1/lib/node_modules/wrangler/templates/middleware/middleware-ensure-req-body-drained.ts:5:10)

[wrangler:error] AiError: 3043: Internal server error
    at Ai._parseError (cloudflare-internal:ai-api:212:24)
    at async Ai.run (cloudflare-internal:ai-api:190:19)
    at async runWorker (file:///home/proman4713/JavaScript%20Projects/MawjoodAPI/src/index.js:50:20)
    at async jsonError (file:///home/proman4713/.nvm/versions/node/v22.13.1/lib/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)
    at async drainBody (file:///home/proman4713/.nvm/versions/node/v22.13.1/lib/node_modules/wrangler/templates/middleware/middleware-ensure-req-body-drained.ts:5:10)

[wrangler:error] AiError: 3043: Internal server error
    at Ai._parseError (cloudflare-internal:ai-api:212:24)
    at async Ai.run (cloudflare-internal:ai-api:190:19)
    at async runWorker (file:///home/proman4713/JavaScript%20Projects/MawjoodAPI/src/index.js:50:20)
    at async jsonError (file:///home/proman4713/.nvm/versions/node/v22.13.1/lib/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)
    at async drainBody (file:///home/proman4713/.nvm/versions/node/v22.13.1/lib/node_modules/wrangler/templates/middleware/middleware-ensure-req-body-drained.ts:5:10)

P

Proman4713 🇵🇸OP•9/12/25, 10:21 PM

beginning of this issue: workers-ai

S

samjs•9/12/25, 11:25 PM

Can you share the full code?

Ssamjs Can you share the full code?

P

Proman4713 🇵🇸OP•9/13/25, 7:45 AM

alright

P

Proman4713 🇵🇸OP•9/13/25, 7:48 AM

message.js2.33KB

AITools.js8.17KB

P

Proman4713 🇵🇸OP•9/13/25, 7:50 AM

@samjs there you go, for context:
- createRouteDefinition is just a utility function I wrote to create a sort of "file-based" routing
- I'm using JS, not TS, hence the imports from @cloudflare/workers-types in JSDoc
- the AI type is provided by the parent createRouteDefinition function, so I do have the proper autocomplete... And I frequently ctrl+click my AI.run call to check my types

P

Proman4713 🇵🇸OP•9/13/25, 6:39 PM

@samjs uhhh, any leads?

S

samjs•9/13/25, 7:13 PM

Hey @Proman4713

im away from my computer at the moment. I'll try and get back to you in the next couple of days

Ssamjs Hey @Proman4713 🇵🇸 im away from my computer at the moment. I'll try and get b...

P

Proman4713 🇵🇸OP•9/13/25, 8:09 PM

Alright, thank you very much for your determination!

S

samjs•9/15/25, 6:24 PM

Hey @Proman4713

. I've been doing some testing with your code -- thank you for sharing.

I think there's a slightly weird incompatibility between mistral and vllm here. We're looking into some things we could do for this, like some vllm templates.

In the meantime though, I think you can address this by constructing messages like this:

messages: [
  {
    "role": "system",
    "content": "sys prompt goes here"
  },
  {
    "role": "user",
    "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
    "role": "assistant",
    "content": "{\"name\":\"getClassRoster\",\"arguments\":{\"grade\":10,\"classLetter\":\"A\"}}"
  },
  {
    "role": "tool",
    "content": "{\"data\":[{\"englishName\":\"bob\",\"arabicName\":\"ابراهيم محمد ابراهيم الدسوقي\",\"studentId\":123}]}",
    "tool_call_id": "000000001"
  }
]

messages: [
  {
    "role": "system",
    "content": "sys prompt goes here"
  },
  {
    "role": "user",
    "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
    "role": "assistant",
    "content": "{\"name\":\"getClassRoster\",\"arguments\":{\"grade\":10,\"classLetter\":\"A\"}}"
  },
  {
    "role": "tool",
    "content": "{\"data\":[{\"englishName\":\"bob\",\"arabicName\":\"ابراهيم محمد ابراهيم الدسوقي\",\"studentId\":123}]}",
    "tool_call_id": "000000001"
  }
]

messages: [
  {
    "role": "system",
    "content": "sys prompt goes here"
  },
  {
    "role": "user",
    "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
    "role": "assistant",
    "content": "{\"name\":\"getClassRoster\",\"arguments\":{\"grade\":10,\"classLetter\":\"A\"}}"
  },
  {
    "role": "tool",
    "content": "{\"data\":[{\"englishName\":\"bob\",\"arabicName\":\"ابراهيم محمد ابراهيم الدسوقي\",\"studentId\":123}]}",
    "tool_call_id": "000000001"
  }
]

messages: [
  {
    "role": "system",
    "content": "sys prompt goes here"
  },
  {
    "role": "user",
    "content": "What's ابراهيم محمد ابراهيم الدسوقي's attendance percentage this year in class 10/A?"
  },
  {
    "role": "assistant",
    "content": "{\"name\":\"getClassRoster\",\"arguments\":{\"grade\":10,\"classLetter\":\"A\"}}"
  },
  {
    "role": "tool",
    "content": "{\"data\":[{\"englishName\":\"bob\",\"arabicName\":\"ابراهيم محمد ابراهيم الدسوقي\",\"studentId\":123}]}",
    "tool_call_id": "000000001"
  }
]

i.e.

- construct the assistant message by just taking the function params as a json string
- construct the tool call with a dummy tool call ID

If the tool call also includes response text, you could consider also inserting an assistant message for it.

How does that sound?

Ssamjs Hey @Proman4713 🇵🇸 . I've been doing some testing with your code -- thank you ...

P

Proman4713 🇵🇸OP•9/15/25, 6:27 PM

Alright, I'll do that...

I'd like to tell you that I literally had this thread open, waiting for your response... And I'd also like to thank you again for diving into this and helping me

S

samjs•9/15/25, 6:28 PM

you're welcome. Apologies for the rough experience here. I'm accumulating examples like these and hoping to get some consistency to our sdks/apis/tests etc.

Ssamjs you're welcome. Apologies for the rough experience here. I'm accumulating exampl...

P

Proman4713 🇵🇸OP•9/15/25, 6:29 PM

Alright, I will try this code tonight and let you know about the results

P

Proman4713 🇵🇸OP•9/15/25, 7:23 PM

okay @samjs it worked but I just discovered that the model apparently does not stack tool calls, it calls the first tool, then waits for the response, then calls the next one... is this the correct behaviour or can I change it? I wouldn't mind accounting for that as long as it works, though

S

samjs•9/15/25, 7:30 PM

yup, that's the expected behaviour for most tool-calling models

S

samjs•9/15/25, 7:30 PM

typically you'll want to call them in a loop and break out of the loop once there are no more tool calls

Ssamjs typically you'll want to call them in a loop and break out of the loop once ther...

P

Proman4713 🇵🇸OP•9/15/25, 7:33 PM

alright, then thank you for making it work!

S

samjs•9/15/25, 7:37 PM

Just a heads up: I've seen some of these models start to struggle for longer chains of tool calling. It's not an exact art at the moment unfortunately. I think the gpt-oss tools are a little better (they had a lot more training for tool calling).

Ssamjs Just a heads up: I've seen some of these models start to struggle for longer cha...

P

Proman4713 🇵🇸OP•9/15/25, 7:56 PM

I just went to the models page on the cloudflare docs and Function calling didn't show any GPTs

S

samjs•9/15/25, 11:51 PM

Yeah, we have some docs to update still. It's not supported via

runWithTools

runWithTools

, but if you call it with Responses API format, you will get back tool calls. e.g. similar to this: https://github.com/cloudflare/sandbox-sdk/blob/main/examples/code-interpreter/src/index.ts#L26-L37

GitHub

sandbox-sdk/examples/code-interpreter/src/index.ts at main · cloud...

Run sandboxed code environments on Cloudflare's edge network - cloudflare/sandbox-sdk

S

samjs•9/16/25, 3:37 PM

e.g.

import { Hono } from "hono";

export const TOOLS = [
    {
        type: "function",
        name: "get_weather",
        description: "Get the weather for a given location",
        parameters: {
            type: "object",
            properties: {
                location: {
                    type: "string",
                    description: "The location to get the weather for",
                },
            },
            required: ["location"],
            additionalProperties: false,
        },
        output: '{"weather":"sunny"}',
    }
]

// GPT OSS router
const gptOss = new Hono<{ Bindings: Env }>();

gptOss.get('/get', async (c) => {
    const response = await c.env.AI.run("@cf/openai/gpt-oss-20b" as any, { input: "what is the weather like in London?", tools: TOOLS });
    return c.json(response);
});

export default gptOss;

import { Hono } from "hono";

export const TOOLS = [
    {
        type: "function",
        name: "get_weather",
        description: "Get the weather for a given location",
        parameters: {
            type: "object",
            properties: {
                location: {
                    type: "string",
                    description: "The location to get the weather for",
                },
            },
            required: ["location"],
            additionalProperties: false,
        },
        output: '{"weather":"sunny"}',
    }
]

// GPT OSS router
const gptOss = new Hono<{ Bindings: Env }>();

gptOss.get('/get', async (c) => {
    const response = await c.env.AI.run("@cf/openai/gpt-oss-20b" as any, { input: "what is the weather like in London?", tools: TOOLS });
    return c.json(response);
});

export default gptOss;

import { Hono } from "hono";

export const TOOLS = [
    {
        type: "function",
        name: "get_weather",
        description: "Get the weather for a given location",
        parameters: {
            type: "object",
            properties: {
                location: {
                    type: "string",
                    description: "The location to get the weather for",
                },
            },
            required: ["location"],
            additionalProperties: false,
        },
        output: '{"weather":"sunny"}',
    }
]

// GPT OSS router
const gptOss = new Hono<{ Bindings: Env }>();

gptOss.get('/get', async (c) => {
    const response = await c.env.AI.run("@cf/openai/gpt-oss-20b" as any, { input: "what is the weather like in London?", tools: TOOLS });
    return c.json(response);
});

export default gptOss;

import { Hono } from "hono";

export const TOOLS = [
    {
        type: "function",
        name: "get_weather",
        description: "Get the weather for a given location",
        parameters: {
            type: "object",
            properties: {
                location: {
                    type: "string",
                    description: "The location to get the weather for",
                },
            },
            required: ["location"],
            additionalProperties: false,
        },
        output: '{"weather":"sunny"}',
    }
]

// GPT OSS router
const gptOss = new Hono<{ Bindings: Env }>();

gptOss.get('/get', async (c) => {
    const response = await c.env.AI.run("@cf/openai/gpt-oss-20b" as any, { input: "what is the weather like in London?", tools: TOOLS });
    return c.json(response);
});

export default gptOss;

Ssamjs e.g. ``` import { Hono } from "hono"; export const TOOLS = [ { ty...

P

Proman4713 🇵🇸OP•9/18/25, 10:01 PM

I was just gonna resort to recursively running the AI until it doesn't call anymore tools

P

Proman4713 🇵🇸OP•9/18/25, 10:01 PM

but it seems like there isnt much difference in the schema

P

Proman4713 🇵🇸OP•9/18/25, 10:02 PM

are there any other differences I should know of?

P

Proman4713 🇵🇸OP•9/18/25, 10:02 PM

So that I don't go down another rabbit hole?

S

samjs•9/18/25, 10:31 PM

Hmm, nothing that jumps to mind

Similar Threads

Similar Threads

Similar Threads