Has there been a change to the `hermes-2-pro-mistral-7b` model. I have a very basic app based on CF'

Has there been a change to the

hermes-2-pro-mistral-7b

hermes-2-pro-mistral-7b

hermes-2-pro-mistral-7b

hermes-2-pro-mistral-7b model. I have a very basic app based on CF's function calling tutorial. I made the app about a month ago and has been working fine since. But over the past few days it has stopped working and it appears that the model has changed the format of its response. Here is how it is responding now:

User Prompt: show me the color of an avocado?
AI result is: {
  response: '<tool_call>\n' +
    "{'arguments': {'r': 69, 'g': 139, 'b': 69}, 'name': 'switchLightColor'}\n" +
    '</tool_call>\n' +
    '\n' +
    'I have chosen a color that represents an avocado. The color of an avocado is typically a pale green with a hint of creaminess. To represent this, I have selected a color with a higher green value (139) and balanced amounts of red (69) and blue (69) values. This creates a color that is reminiscent of the natural hue of an avocado, bringing the essence of the fruit into the room.',
  usage: { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 }
}

User Prompt: show me the color of an avocado?
AI result is: {
  response: '<tool_call>\n' +
    "{'arguments': {'r': 69, 'g': 139, 'b': 69}, 'name': 'switchLightColor'}\n" +
    '</tool_call>\n' +
    '\n' +
    'I have chosen a color that represents an avocado. The color of an avocado is typically a pale green with a hint of creaminess. To represent this, I have selected a color with a higher green value (139) and balanced amounts of red (69) and blue (69) values. This creates a color that is reminiscent of the natural hue of an avocado, bringing the essence of the fruit into the room.',
  usage: { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 }
}

User Prompt: show me the color of an avocado?
AI result is: {
  response: '<tool_call>\n' +
    "{'arguments': {'r': 69, 'g': 139, 'b': 69}, 'name': 'switchLightColor'}\n" +
    '</tool_call>\n' +
    '\n' +
    'I have chosen a color that represents an avocado. The color of an avocado is typically a pale green with a hint of creaminess. To represent this, I have selected a color with a higher green value (139) and balanced amounts of red (69) and blue (69) values. This creates a color that is reminiscent of the natural hue of an avocado, bringing the essence of the fruit into the room.',
  usage: { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 }
}

User Prompt: show me the color of an avocado?
AI result is: {
  response: '<tool_call>\n' +
    "{'arguments': {'r': 69, 'g': 139, 'b': 69}, 'name': 'switchLightColor'}\n" +
    '</tool_call>\n' +
    '\n' +
    'I have chosen a color that represents an avocado. The color of an avocado is typically a pale green with a hint of creaminess. To represent this, I have selected a color with a higher green value (139) and balanced amounts of red (69) and blue (69) values. This creates a color that is reminiscent of the natural hue of an avocado, bringing the essence of the fruit into the room.',
  usage: { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 }
}

The tool call is being returned within the response and is correctly tagged with

<tool_call>

<tool_call>

<tool_call>

<tool_call> but the response no longer returns a

tool_calls

tool_calls

tool_calls

tool_calls prop, so my code isn't calling anything it only shows the malformed response.

blindChickenOP•3/16/25, 6:11 PM

If I replace

hermes-2-pro-mistral-7b

hermes-2-pro-mistral-7b

with @cf/meta/llama-3.3-70b-instruct-fp8-fast@cf/meta/llama-3.3-70b-instruct-fp8-fast then it works fine. Something must have changed with the Mistral model

kalepail•3/16/25, 7:58 PM

I'm trying to use the @cf/meta/llama-guard-3-8b@cf/meta/llama-guard-3-8b. What's the proper way to tweak this to modify the UNSAFE CONTENT CATEGORIESUNSAFE CONTENT CATEGORIES? A lot of sexually explict stuff is getting through when the prompts aren't very long. I want a really robust NSFW content filter but asis this model isn't working well

BblindChicken Has there been a change to the `hermes-2-pro-mistral-7b` model. I have a very ba...

James•3/16/25, 8:26 PM

cc @michelle possible breaking change or incident with tool calling for this model?

James•3/16/25, 8:44 PM

@Isaac | AS213339 confirmed the regression and has raised an escalation with the team. Thanks for the report.

JJames @Isaac | AS213339 confirmed the regression and has raised an escalation with the...

Isaac McFadyen•3/16/25, 8:45 PM

^ Oh yes, thanks for mentioning; forgot to say that here.

Houston CF•3/16/25, 9:58 PM

@blindChicken @Isaac | AS213339 @James Are we seeing the results we expected now? We have deployed a fix for the " tool calling using hermes-2-pro-mistral-7b model" issue.

James•3/16/25, 9:59 PM

Looks good now, thank you!

JJames Looks good now, thank you!

blindChickenOP•3/17/25, 1:17 AM

We just tested it on our end and it appears to be working correctly. Thanks!

markvaldez•3/17/25, 6:11 PM

Hi, I don't see any way to rotate my REST bearer AI token..

markvaldez•3/17/25, 6:14 PM

I didn't create a token, it was created for me on the dashboard when I went to the AI Worker page

markvaldez•3/17/25, 6:39 PM

I applied as an Engineer at Cloudflare this past month, but I don't think I got past the resume screen. I live around the corner from the SF offices and have been there for a hackathon and lunch-and-learn HONO event the past 2 weeks. I can fix this issue: https://github.com/valdezm/dash.cloudflare.com/issues/1

GitHub

Cannot rotate REST AI Bearer token · Issue #1 · valdezm/dash.cloudf...

Problem A REST Bearer token was created for me and does not show up in my Account API Tokens using these instructions: https://developers.cloudflare.com/workers-ai/get-started/rest-api/#1-get-api-t...

markvaldez•3/17/25, 6:40 PM

https://tenor.com/view/bills-ugh-gif-9811803164280522639

Tenor

markvaldez•3/17/25, 6:41 PM

Send me a DM and I will happliy get to work

markvaldez•3/17/25, 6:43 PM

Also, you all are awesome

Alex Patterson•3/20/25, 3:30 AM

Seems like all the models on Workers AI have such small Context WIndows, is there any that will be added that come closer to Google Gemini?

si(dot).xth•3/20/25, 11:30 AM

my pillow fixed my issue:

raw data for simplicity and efficiency in handling large images, this method may lead to more API calls if the image is re-generated each time it's accessed (Caching handles that last part.)

base64 encoding to minimize API calls over performance (Easier to implement)

Raw data + cache is the only way to go

Original: Blob not supported for workers ai? URL.CreateObjectURL() is not implemented

AAlex Patterson Seems like all the models on Workers AI have such small Context WIndows, is ther...

Silvan•3/20/25, 12:01 PM

Having the same issue here. Trying to create a story is fairly hard this way but some models do have bigger context windows but many of the "good ones" are less than 10k

JMPA•3/20/25, 9:14 PM

hello, is inpaiting ai workers working currently?

JMPA•3/20/25, 9:15 PM

i tried 2 models and they dont works.

JMPA•3/20/25, 9:16 PM

stable-diffusion-v1-5-inpainting
API response status: 400 body: {"errors":[{"message":"AiError: Bad input: Error: Type mismatch of '/mask', 'array' not in 'string'","code":5006}],"success":false,"result":{},"messages":[]}
API error: 400 - {"errors":[{"message":"AiError: Bad input: Error: Type mismatch of '/mask', 'array' not in 'string'","code":5006}],"success":false,"result":{},"messages":[]}
Response with status 400 in 1363 ms.

JMPA•3/20/25, 10:19 PM

anyone there?

James•3/20/25, 10:27 PM

Is this something that worked previously and now doesn't? Can you provide any code to reproduce?

JJMPA hello, is inpaiting ai workers working currently?

Raylight•3/22/25, 6:59 AM

As James implied, it would be easier to help if you have a snippet showing the call to the model, but in general: The docs show the same set of parameters (e.g. imageimage, maskmask) for several models, but the only model that does inpainting is @cf/runwayml/stable-diffusion-v1-5-inpainting@cf/runwayml/stable-diffusion-v1-5-inpainting.
Make sure to supply both imageimage and maskmask when calling it. The format is mask: [...new Uint8Array(await maskImage.arrayBuffer())]mask: [...new Uint8Array(await maskImage.arrayBuffer())], where maskImagemaskImage could be the result of e.g. await fetch(/* mask image url */)await fetch(/* mask image url */).

Silvan•3/23/25, 11:09 AM

I really hope we get some “better” text generation models on here soon. i tried every single one for story creation and nothing came even remotely close to claude atm.

ochrons•3/23/25, 3:37 PM

I'm using @cf/openai/whisper-large-v3-turbo@cf/openai/whisper-large-v3-turbo to convert audio into text. It works fine when recording on Pixel 9 in AAC format, but fails when recording on an older Oneplus 7T with AAC (I'm using Flutter to build the app). Using an AMR WB format works fine, though. Also all the AAC formats work fine when given to Google Gemini Flash/Pro models

ochrons•3/23/25, 3:38 PM

I tried both MPEG-4 AAC Low complexityMPEG-4 AAC Low complexity and MPEG-4 High Efficiency AACMPEG-4 High Efficiency AAC formats with identical results

ochrons•3/23/25, 3:54 PM

ffprobeffprobe says this about the one recorded on Oneplus,

[aac @ 0x560e66ce38c0] Number of scalefactor bands in group (46) exceeds limit (43).

[aac @ 0x560e66ce38c0] Number of scalefactor bands in group (46) exceeds limit (43).

while there are no errors with the AAC coming from pixel 9

0xCC•3/25/25, 4:17 AM

Hi the Daily Usage Limits for ai workder is not reset today
(my account 4631e4e4e57a2e43b16a0a5a1490d7c3 )

Original message was deleted

DaniFoldi•3/25/25, 8:35 AM

Argh I thought I got them all, thanks

nuno•3/25/25, 10:40 AM

Hey guys, I'm trying to generate embeddings using the @cf/baai/bge-m3@cf/baai/bge-m3 model but I keep getting the error below. Previously I was using the @cf/baai/bge-base-en-v1.5@cf/baai/bge-base-en-v1.5 model and everything was just fine.

err : AiError: 5006: Error: required properties at '/' are 'contexts'

err : AiError: 5006: Error: required properties at '/' are 'contexts'

Does anyone have any idea why is this happening? Any guidance here is much appreciated.

/dev/null•3/27/25, 9:51 AM

the cost of flux schnell is 172 neurons per image when I set with 1 step.

cyrilis•3/27/25, 2:22 PM

the deepseek model: @cf/deepseek-ai/deepseek-r1-distill-qwen-32b@cf/deepseek-ai/deepseek-r1-distill-qwen-32b

response sometime got empty response:

data: {"response":""}

data: {"response":""}

data: {"response":"","usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1}}

data: {"response":""}

data: {"response":""}

data: {"response":"","usage":{"prompt_tokens":0,"completion_tokens":1,"total_tokens":1}}

and sometimes it works all right, but missing the opening <think> tag. any others rushed into same problem?

Ccyrilis the deepseek model: `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` response s...

cyrilis•3/27/25, 2:24 PM

Or maybe it's just because I enabled smart placement? and a few AI models are not ready in some region ?

Ccyrilis the deepseek model: `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` response s...

Isaac McFadyen•3/27/25, 3:21 PM

missing the opening <think> tag

This is due to a quirk in the prompt template, which forces the <think> tag in order to require the model to think through it's response, but which results in this behavior

Isaac McFadyen•3/27/25, 3:21 PM

@michelle ^ been reported many times here, might want to fix it eventually

Isaac McFadyen•3/27/25, 3:22 PM

Not sure if it was always the case or whether it was a change but regardless, it's different from how other providers generally do it

Pytato•3/28/25, 6:58 AM

I think I have found a bug with Cloudflare's Worker Ai

I'm basically making a super simple Stable Diffusion wrapper with Pages and Worker Ai. While making the UI and testing it, I probably made 50 - 80 50 - 80 images with it. But my Daily Neurons Limits is still 0 0 (as in nothing used)

So, I checked the Pricing and found no mention of Stable Diffusion's neuron pricing.

Could it be they forgot to consider these model to add to the daily usage limit?
OR is 8080 images too low to be show in the dashboard?

Pytato•3/28/25, 7:03 AM

Oh... I'm stupid

mr.niko.la•3/28/25, 2:56 PM

@cf/openai/whisper-large-v3-turbo@cf/openai/whisper-large-v3-turbo

is there no sample worker code for this for something basic like r2 audio file to transcribe ?

I am using a worker to send it to 3rd party api to transcribe using queue

But I would like use worker ai whisper turbo as a failover. Without over engineering or breaking my webapp.

Do I just implement a new queue instead of a separate worker?

Perhaps eventually cf workflow but for now I don’t want to break my current webapp by reengineering anything too much.

Just want to do backup for the rare times 3rd api fails.

Mistic92•3/29/25, 5:34 PM

I'm getting 3001 error when trying to use Whisper :/

{"errors":[{"message":"AiError: AiError: Unknown internal error (8e2b7fc6-c9be-4abd-b5da-82def70cb5be)","code":3001}],"success":false,"result":{},"messages":[]}%

{"errors":[{"message":"AiError: AiError: Unknown internal error (8e2b7fc6-c9be-4abd-b5da-82def70cb5be)","code":3001}],"success":false,"result":{},"messages":[]}%

Jaka•3/30/25, 11:25 AM

I need help with pricing for creating embendings, storing them in vectorize for almost 280k md files about Romanian legislation. i have them in R2. i want to create a rag that is capable of answering very complex questions about Romanian legislation. i did some estimations uisng chatgpt but i hope it is wrong( almost 8k was the estimation for embedings).

Brandon Studio•3/31/25, 8:18 AM

Currently stream APIs (including official API and OpenAI compatible API) do not properly handle tool_call response; the tool call content is embedded in the responseresponse or

content

content

field.
Could the stream API provide the same behavior as the non-stream API? For example, "response“: null, "tool_calls": [...]"response“: null, "tool_calls": [...]

Brandon Studio•3/31/25, 8:19 AM

Also, OpenAI compatible API only returns

null

null

in non-stream mode

vivsam•3/31/25, 8:38 AM

It'd be nice if you guys add Zero-Shot-Text-Classification models like https://huggingface.co/facebook/bart-large-mnli

facebook/bart-large-mnli · Hugging Face

sku•3/31/25, 5:46 PM

I've tried to make requests to several different models today over the REST API and I've only gotten this response. Are there any known issues?

{
    "result": null,
    "success": false,
    "errors": [
        {
            "code": 7010,
            "message": "Service unavailable"
        }
    ],
    "messages": []
}

{
    "result": null,
    "success": false,
    "errors": [
        {
            "code": 7010,
            "message": "Service unavailable"
        }
    ],
    "messages": []
}

BBrandon Studio Currently stream APIs (including official API and OpenAI compatible API) do not ...

Isaac McFadyen•3/31/25, 8:28 PM

Is this for AI Gateway or Workers AI?

Isaac McFadyen•3/31/25, 8:28 PM

And what models does this occur on?

nclevenger•3/31/25, 9:49 PM

hey - this new model is amazing ... it's SOTA on SWE Bench for a 32b model, scoring 37% which is SHOCKING for a model this small ... would love to see this on Workers AI https://huggingface.co/all-hands/openhands-lm-32b-v0.1

all-hands/openhands-lm-32b-v0.1 · Hugging Face

IIsaac McFadyen Is this for AI Gateway or Workers AI?

Brandon Studio•4/1/25, 2:09 AM

Workers AI, behaviors with and without AI Gateway are same

Has there been a change to the `hermes-2-pro-mistral-7b` model. I have a very basic app based on CF'

Similar Threads