ah! default `max_tokens` for the result is only 256. bumping it up fixed it. there is no indication

Llfdepombo ah! default `max_tokens` for the result is only 256. bumping it up fixed it. the...

J

James•3/7/25, 5:07 PM

Yeah this was an undocumented breaking change. Previously the defaults weren't enforced but they have been for a short while now. We're trying to get a changelog entry or something about this so folks are aware of the change.

A

ajgeiss0702•3/7/25, 11:52 PM

it looks like

@cf/deepseek-ai/deepseek-r1-distill-qwen-32b

@cf/deepseek-ai/deepseek-r1-distill-qwen-32b

is missing the starting

<think>

<think>

tag in responses

G

Gyurmatag•3/8/25, 11:42 AM

Hi guys!
Should't we add a new channel inside AI+ML for the Agents SDK?

K

kian•3/8/25, 4:55 PM

#agents

K

kian•3/8/25, 4:55 PM

There is already one

R

Rubi•3/9/25, 10:24 AM

why cloudflare deepseek qwen is not compatible like Deepseek official cloud?

Aajgeiss0702 it looks like `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` is missing the star...

R

Rubi•3/9/25, 1:29 PM

yeah why this happen

L

Love•3/10/25, 10:23 PM

Hey, I'm completely new to Cloudflare use but I'm setting up a worker for parsing JSON to my discord bot but im having difficulties setting up the worker and "connecting it" to my docker runtime

R

Rubi•3/11/25, 8:04 AM

you can now use Cloudflare R1 Distill Qwen with browser-use

https://github.com/browser-use/browser-use

GitHub

GitHub - browser-use/browser-use: Make websites accessible for AI a...

Make websites accessible for AI agents. Contribute to browser-use/browser-use development by creating an account on GitHub.

M

Muhammadreza Haghiri•3/11/25, 8:58 AM

I am using FLUX Schnell like this:

curl --location --request POST 'https://api.cloudflare.com/client/v4/accounts/REDACTED/ai/run/@cf/black-forest-labs/flux-1-schnell' \
--header 'Authorization: Bearer REDACTED' \
--header 'Content-Type: application/json' \
--header 'Cookie: __cf_bm=V5sLTJx2Y7d6s6bClk.umX36FohdsTMoXamYf1pW1f8-1741683179-1.0.1.1-rGuk5rHKjPNKMCBtB45ZW51MZeJUyqVGXeW4hr0o0m7F7JFsT315BeJAbXd8LKCOuApweso_4kAYQf4SfM6R60cd.ZtvxRyn6o3qLISOqS4' \
--data-raw '{
    "prompt" : "A serene tropical beach at sunset, a cozy wooden cottage with smoke rising from the chimney, tall swaying palm trees, calm ocean waves reflecting golden hues of sunlight, painted in a dreamy watercolor style with soft, vibrant colors and intricate details.", 
    "steps" : 8,
    "width" :1024, 
    "height" : 1024, 
    "seed" : 93617911245
}'

curl --location --request POST 'https://api.cloudflare.com/client/v4/accounts/REDACTED/ai/run/@cf/black-forest-labs/flux-1-schnell' \
--header 'Authorization: Bearer REDACTED' \
--header 'Content-Type: application/json' \
--header 'Cookie: __cf_bm=V5sLTJx2Y7d6s6bClk.umX36FohdsTMoXamYf1pW1f8-1741683179-1.0.1.1-rGuk5rHKjPNKMCBtB45ZW51MZeJUyqVGXeW4hr0o0m7F7JFsT315BeJAbXd8LKCOuApweso_4kAYQf4SfM6R60cd.ZtvxRyn6o3qLISOqS4' \
--data-raw '{
    "prompt" : "A serene tropical beach at sunset, a cozy wooden cottage with smoke rising from the chimney, tall swaying palm trees, calm ocean waves reflecting golden hues of sunlight, painted in a dreamy watercolor style with soft, vibrant colors and intricate details.", 
    "steps" : 8,
    "width" :1024, 
    "height" : 1024, 
    "seed" : 93617911245
}'

curl --location --request POST 'https://api.cloudflare.com/client/v4/accounts/REDACTED/ai/run/@cf/black-forest-labs/flux-1-schnell' \
--header 'Authorization: Bearer REDACTED' \
--header 'Content-Type: application/json' \
--header 'Cookie: __cf_bm=V5sLTJx2Y7d6s6bClk.umX36FohdsTMoXamYf1pW1f8-1741683179-1.0.1.1-rGuk5rHKjPNKMCBtB45ZW51MZeJUyqVGXeW4hr0o0m7F7JFsT315BeJAbXd8LKCOuApweso_4kAYQf4SfM6R60cd.ZtvxRyn6o3qLISOqS4' \
--data-raw '{
    "prompt" : "A serene tropical beach at sunset, a cozy wooden cottage with smoke rising from the chimney, tall swaying palm trees, calm ocean waves reflecting golden hues of sunlight, painted in a dreamy watercolor style with soft, vibrant colors and intricate details.", 
    "steps" : 8,
    "width" :1024, 
    "height" : 1024, 
    "seed" : 93617911245
}'

curl --location --request POST 'https://api.cloudflare.com/client/v4/accounts/REDACTED/ai/run/@cf/black-forest-labs/flux-1-schnell' \
--header 'Authorization: Bearer REDACTED' \
--header 'Content-Type: application/json' \
--header 'Cookie: __cf_bm=V5sLTJx2Y7d6s6bClk.umX36FohdsTMoXamYf1pW1f8-1741683179-1.0.1.1-rGuk5rHKjPNKMCBtB45ZW51MZeJUyqVGXeW4hr0o0m7F7JFsT315BeJAbXd8LKCOuApweso_4kAYQf4SfM6R60cd.ZtvxRyn6o3qLISOqS4' \
--data-raw '{
    "prompt" : "A serene tropical beach at sunset, a cozy wooden cottage with smoke rising from the chimney, tall swaying palm trees, calm ocean waves reflecting golden hues of sunlight, painted in a dreamy watercolor style with soft, vibrant colors and intricate details.", 
    "steps" : 8,
    "width" :1024, 
    "height" : 1024, 
    "seed" : 93617911245
}'

And I get this error:

{
    "errors": [
        {
            "message": "AiError: AiError: Invalid input",
            "code": 5006
        }
    ],
    "success": false,
    "result": {},
    "messages": []
}

{
    "errors": [
        {
            "message": "AiError: AiError: Invalid input",
            "code": 5006
        }
    ],
    "success": false,
    "result": {},
    "messages": []
}

{
    "errors": [
        {
            "message": "AiError: AiError: Invalid input",
            "code": 5006
        }
    ],
    "success": false,
    "result": {},
    "messages": []
}

{
    "errors": [
        {
            "message": "AiError: AiError: Invalid input",
            "code": 5006
        }
    ],
    "success": false,
    "result": {},
    "messages": []
}

What causes this? What modification should I make to my code?

M

Muhammadreza Haghiri•3/11/25, 9:49 AM

I looked at the docs and there is no error code for 5006.

M

Muhammadreza Haghiri•3/11/25, 2:50 PM

It works once and doesn't for a 100 times.

C

Chaika•3/11/25, 3:13 PM

Thanks, repro'd and raising

CChaika Thanks, repro'd and raising

M

Muhammadreza Haghiri•3/11/25, 3:53 PM

Was it for my problem?

MMuhammadreza Haghiri Was it for my problem?

C

Chaika•3/11/25, 3:59 PM

Yes. CF acknowledged the report, hopefully fix soon, seems to have been erroring for a bit..

CChaika Yes. CF acknowledged the report, hopefully fix soon, seems to have been erroring...

M

Muhammadreza Haghiri•3/11/25, 4:12 PM

Thanks

D

dan•3/11/25, 6:08 PM

are there any STT models on workers ai that support live streaming?

Ddan are there any STT models on workers ai that support live streaming?

I

Isaac McFadyen•3/11/25, 6:16 PM

No, and a lot of the new models being developed don't stream at all any more, actually. For example, Whisper/Nvidia Parakeet/Nvidia Canary don't support streaming, and instead recommend applications implement their own streaming by sending a request every <n> seconds while continuing recording (e.g. using multiple threads).

MMuhammadreza Haghiri I am using FLUX Schnell like this: ``` curl --location --request POST 'https:/...

A

ale•3/11/25, 6:26 PM

hi @Muhammadreza Haghiri, could you please try this request once again? The team has deployed a fix

D

DennisMotta•3/12/25, 12:51 AM

why language auto-detection is not supported on whisper-large-v3-turbo?

D

DennisMotta•3/12/25, 12:51 AM

by default it transcribes to English, I would like to keep the original language, similar to what @cf/openai/whisper does

M

Max•3/12/25, 9:12 AM

Hi. is this the correct channel to ask about workers-mcp? if so: what is the ETA on

Remote MCP support: Connect to MCP servers directly without requiring a local proxy

Remote MCP support: Connect to MCP servers directly without requiring a local proxy

@geleen

D

dmaretskyi•3/12/25, 2:18 PM

Does

deepseek-r1-distill-qwen-32b

deepseek-r1-distill-qwen-32b

support tools calls in stream mode? Is there a code example?

Aale hi @Muhammadreza Haghiri, could you please try this request once again? The team...

M

Muhammadreza Haghiri•3/12/25, 7:33 PM

Hi. It was okay today.

LLiam Every single day :/ The fail rate at this point is just 99%

C

certified dirtiboi•3/12/25, 10:24 PM

Also seeing this error. I've seen other posts saying it's a network issue at the workers-ai level, but I have yet to see a resolution or RCA

Is the solution to just add a fallback retry?

errors: [
{
message: 'AiError: AiError: Capacity temporarily exceeded, please try again.',
code: 3040
}
]

Aajgeiss0702 it looks like `@cf/deepseek-ai/deepseek-r1-distill-qwen-32b` is missing the star...

A

ajgeiss0702•3/13/25, 12:35 AM

bump

Ddmaretskyi Does `deepseek-r1-distill-qwen-32b` support tools calls in stream mode? Is there...

D

dmaretskyi•3/13/25, 5:27 PM

Here's the snipped I'm running;

const response = await env.AI.run('@cf/deepseek-ai/deepseek-r1-distill-qwen-32b', {
    messages: [
      {
        role: 'user',
        content: 'Make a cup of earl grey tea. Use the tool provided.',
      },
    ],
    tools: [
      {
        name: 'make_tea',
        description: 'Make a cup of tea',
        parameters: {
          type: 'object',
          properties: {
            tea_type: {
              type: 'string',
              description: 'The type of tea to make',
              enum: ['earl grey', 'chamomile', 'peppermint'],
            },
          },
        },
      },
    ],
  } satisfies BaseAiTextGeneration['inputs']);

  log.info('generateCloudflare', { response });

const response = await env.AI.run('@cf/deepseek-ai/deepseek-r1-distill-qwen-32b', {
    messages: [
      {
        role: 'user',
        content: 'Make a cup of earl grey tea. Use the tool provided.',
      },
    ],
    tools: [
      {
        name: 'make_tea',
        description: 'Make a cup of tea',
        parameters: {
          type: 'object',
          properties: {
            tea_type: {
              type: 'string',
              description: 'The type of tea to make',
              enum: ['earl grey', 'chamomile', 'peppermint'],
            },
          },
        },
      },
    ],
  } satisfies BaseAiTextGeneration['inputs']);

  log.info('generateCloudflare', { response });

const response = await env.AI.run('@cf/deepseek-ai/deepseek-r1-distill-qwen-32b', {
    messages: [
      {
        role: 'user',
        content: 'Make a cup of earl grey tea. Use the tool provided.',
      },
    ],
    tools: [
      {
        name: 'make_tea',
        description: 'Make a cup of tea',
        parameters: {
          type: 'object',
          properties: {
            tea_type: {
              type: 'string',
              description: 'The type of tea to make',
              enum: ['earl grey', 'chamomile', 'peppermint'],
            },
          },
        },
      },
    ],
  } satisfies BaseAiTextGeneration['inputs']);

  log.info('generateCloudflare', { response });

const response = await env.AI.run('@cf/deepseek-ai/deepseek-r1-distill-qwen-32b', {
    messages: [
      {
        role: 'user',
        content: 'Make a cup of earl grey tea. Use the tool provided.',
      },
    ],
    tools: [
      {
        name: 'make_tea',
        description: 'Make a cup of tea',
        parameters: {
          type: 'object',
          properties: {
            tea_type: {
              type: 'string',
              description: 'The type of tea to make',
              enum: ['earl grey', 'chamomile', 'peppermint'],
            },
          },
        },
      },
    ],
  } satisfies BaseAiTextGeneration['inputs']);

  log.info('generateCloudflare', { response });

 {
  response: {
    response: "Okay, so I need to make a cup of Earl Grey tea using the tools provided. Let me think about what I know about making tea and what tools I might have access to. First, I know that Earl Grey is a black tea with a bergamot flavor, so it's usually served hot and might be drunk with milk or just on its own.\n" +
      '\n' +
      "I should start by figuring out what tools I need. The user mentioned to use the tool provided, but didn't specify what it is. So I'll assume I have a teapot, a kettle for boiling water, a teaspoon for measuring tea, a cup, maybe a strainer, and perhaps a milk pitcher if I want to add milk. I should also consider whether the tools are electric or manual, but I'll go with basic assumptions.\n" +
      '\n' +
      "First step: Boil water. I'll need to fill the kettle with fresh water and bring it to a boil. The temperature for black tea is usually around 90-100°C, but I think boiling water is fine. Wait, no, sometimes it's better to let the water cool a bit after boiling to prevent over-extracting the tea. But Earl Grey is a black tea, so maybe it's okay to use boiling water.\n" +
      '\n',
    usage: { prompt_tokens: 20, completion_tokens: 256, total_tokens: 276 }
  }
}

 {
  response: {
    response: "Okay, so I need to make a cup of Earl Grey tea using the tools provided. Let me think about what I know about making tea and what tools I might have access to. First, I know that Earl Grey is a black tea with a bergamot flavor, so it's usually served hot and might be drunk with milk or just on its own.\n" +
      '\n' +
      "I should start by figuring out what tools I need. The user mentioned to use the tool provided, but didn't specify what it is. So I'll assume I have a teapot, a kettle for boiling water, a teaspoon for measuring tea, a cup, maybe a strainer, and perhaps a milk pitcher if I want to add milk. I should also consider whether the tools are electric or manual, but I'll go with basic assumptions.\n" +
      '\n' +
      "First step: Boil water. I'll need to fill the kettle with fresh water and bring it to a boil. The temperature for black tea is usually around 90-100°C, but I think boiling water is fine. Wait, no, sometimes it's better to let the water cool a bit after boiling to prevent over-extracting the tea. But Earl Grey is a black tea, so maybe it's okay to use boiling water.\n" +
      '\n',
    usage: { prompt_tokens: 20, completion_tokens: 256, total_tokens: 276 }
  }
}

 {
  response: {
    response: "Okay, so I need to make a cup of Earl Grey tea using the tools provided. Let me think about what I know about making tea and what tools I might have access to. First, I know that Earl Grey is a black tea with a bergamot flavor, so it's usually served hot and might be drunk with milk or just on its own.\n" +
      '\n' +
      "I should start by figuring out what tools I need. The user mentioned to use the tool provided, but didn't specify what it is. So I'll assume I have a teapot, a kettle for boiling water, a teaspoon for measuring tea, a cup, maybe a strainer, and perhaps a milk pitcher if I want to add milk. I should also consider whether the tools are electric or manual, but I'll go with basic assumptions.\n" +
      '\n' +
      "First step: Boil water. I'll need to fill the kettle with fresh water and bring it to a boil. The temperature for black tea is usually around 90-100°C, but I think boiling water is fine. Wait, no, sometimes it's better to let the water cool a bit after boiling to prevent over-extracting the tea. But Earl Grey is a black tea, so maybe it's okay to use boiling water.\n" +
      '\n',
    usage: { prompt_tokens: 20, completion_tokens: 256, total_tokens: 276 }
  }
}

 {
  response: {
    response: "Okay, so I need to make a cup of Earl Grey tea using the tools provided. Let me think about what I know about making tea and what tools I might have access to. First, I know that Earl Grey is a black tea with a bergamot flavor, so it's usually served hot and might be drunk with milk or just on its own.\n" +
      '\n' +
      "I should start by figuring out what tools I need. The user mentioned to use the tool provided, but didn't specify what it is. So I'll assume I have a teapot, a kettle for boiling water, a teaspoon for measuring tea, a cup, maybe a strainer, and perhaps a milk pitcher if I want to add milk. I should also consider whether the tools are electric or manual, but I'll go with basic assumptions.\n" +
      '\n' +
      "First step: Boil water. I'll need to fill the kettle with fresh water and bring it to a boil. The temperature for black tea is usually around 90-100°C, but I think boiling water is fine. Wait, no, sometimes it's better to let the water cool a bit after boiling to prevent over-extracting the tea. But Earl Grey is a black tea, so maybe it's okay to use boiling water.\n" +
      '\n',
    usage: { prompt_tokens: 20, completion_tokens: 256, total_tokens: 276 }
  }
}

Ddmaretskyi Here's the snipped I'm running; ```ts const response = await env.AI.run('@cf/dee...

I

Isaac McFadyen•3/13/25, 6:05 PM

Notice how the completion_tokens are 256; the model wasn't actually done, but got cut off because max_tokens defaults to 256 as of a few weeks ago. You'll want to specify a higher maxTokens in your request in order to allow the model to finish it's thought.

J

JoaoCupido•3/13/25, 6:34 PM

Speaking of max_tokens, I can't seem to change max_tokens with the messages property in any chat model, and I have no clue why. I have done this via URL (api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-3-8b-instruct).

IIsaac McFadyen Notice how the completion_tokens are 256; the model wasn't actually done, but go...

D

dmaretskyi•3/13/25, 7:06 PM

hi, thanks for the suggestion! I've increased the token count to 4096 and this is the response I get now:

(info) generateCloudflare {
  stream: {
    response: 'Okay, so I need to make a cup of Earl Grey tea using the tools provided. Hmm, let me think about what tools I might have. Usually, when making tea, you need a kettle, a teapot, a teacup, tea bags or loose tea, hot water, maybe a strainer, and sugar or milk if you want it. \n' +
      '\n' +
      "Wait, the user said to use the tool provided, but didn't specify what that tool is. Maybe it's just a regular teapot? Or maybe a tea infuser? I'm not sure. Well, I guess I'll go with the basics.\n" +
      '\n' +
      "First, I should probably boil some water. If I have a kettle, I can fill it with water and turn it on. While the water is boiling, I can prepare the teapot. If I'm using loose tea, I'll need a tea infuser or a strainer to prevent the leaves from getting into the cup. \n" +
      '\n' +
      "Once the water is boiling, I should let it cool a bit because Earl Grey tea is best when brewed with water that's just below boiling point, maybe around 90-95 degrees Celsius. If I pour boiling water directly on the tea, it might make it too strong or bitter. \n" +
      '\n' +
      "So, I'll pour a little bit of hot water into the teapot to warm it up and then discard that water. Then, I'll add the tea leaves or the tea bag. If I'm using loose tea, I'll put the infuser in the teapot. For a standard cup, maybe about a teaspoon of loose tea or one tea bag. \n" +
      '\n' +
      "Next, I'll pour the hot water over the tea. I'll make sure to cover the teapot to keep the heat in. The steeping time is important. Earl Grey usually takes about 3-5 minutes. If I steep it too long, it might become too strong.\n" +
      '\n' +
      "After steeping, I'll pour the tea into the cup. If I used an infuser, I'll remove it now. I can add sugar or honey if I want it sweet, or maybe a splash of milk. Some people like it with lemon, so that's an option too.\n" +
      '\n' +
      "Wait, did I forget anything? Oh, maybe I should consider the type of cup. If it's a ceramic cup, that's good for retaining heat. Glass might be nice if I want to see the tea. Also, maybe I should clean the teapot and tools afterwards to keep them ready for next time.\n" +
      '\n' +
      "I think that's the basic process. Let me summarize the steps to make sure I didn't miss anything.\n" +
      '</think>\n' +
      '\n' +
      'To make a cup of Earl Grey tea, follow these organized steps:\n' +
      '\n' +
      '1. **Boil Water**: Fill a kettle with water and bring it to a boil. Allow the water to cool slightly to around 90-95°C (194-203°F) to avoid bitterness.\n' +
      '\n' +
      '2. **Prepare the Teapot**: Warm the teapot by pouring a small amount of hot water into it, then discard the water.\n' +
      '\n' +
      '3. **Add Tea**: Place 1 teaspoon of loose Earl Grey tea leaves into a tea infuser or add one Earl Grey tea bag to the teapot.\n' +
      '\n' +
      '4. **Steep the Tea**: Pour the hot water over the tea and cover the teapot. Steep for 3-5 minutes to achieve the desired strength.\n' +
      '\n' +
      '5. **Serve**: Carefully pour the tea into a cup. Remove the infuser or tea bag.\n' +
      '\n' +
      '6. **Customize**: Add sugar, honey, milk, or lemon as preferred.\n' +
      '\n' +
      '7. **Clean Up**: Rinse the teapot and infuser to keep them ready for future use.\n' +
      '\n' +
      'Enjoy your Earl Grey tea!',
    usage: { prompt_tokens: 20, completion_tokens: 761, total_tokens: 781 }
  }
}

(info) generateCloudflare {
  stream: {
    response: 'Okay, so I need to make a cup of Earl Grey tea using the tools provided. Hmm, let me think about what tools I might have. Usually, when making tea, you need a kettle, a teapot, a teacup, tea bags or loose tea, hot water, maybe a strainer, and sugar or milk if you want it. \n' +
      '\n' +
      "Wait, the user said to use the tool provided, but didn't specify what that tool is. Maybe it's just a regular teapot? Or maybe a tea infuser? I'm not sure. Well, I guess I'll go with the basics.\n" +
      '\n' +
      "First, I should probably boil some water. If I have a kettle, I can fill it with water and turn it on. While the water is boiling, I can prepare the teapot. If I'm using loose tea, I'll need a tea infuser or a strainer to prevent the leaves from getting into the cup. \n" +
      '\n' +
      "Once the water is boiling, I should let it cool a bit because Earl Grey tea is best when brewed with water that's just below boiling point, maybe around 90-95 degrees Celsius. If I pour boiling water directly on the tea, it might make it too strong or bitter. \n" +
      '\n' +
      "So, I'll pour a little bit of hot water into the teapot to warm it up and then discard that water. Then, I'll add the tea leaves or the tea bag. If I'm using loose tea, I'll put the infuser in the teapot. For a standard cup, maybe about a teaspoon of loose tea or one tea bag. \n" +
      '\n' +
      "Next, I'll pour the hot water over the tea. I'll make sure to cover the teapot to keep the heat in. The steeping time is important. Earl Grey usually takes about 3-5 minutes. If I steep it too long, it might become too strong.\n" +
      '\n' +
      "After steeping, I'll pour the tea into the cup. If I used an infuser, I'll remove it now. I can add sugar or honey if I want it sweet, or maybe a splash of milk. Some people like it with lemon, so that's an option too.\n" +
      '\n' +
      "Wait, did I forget anything? Oh, maybe I should consider the type of cup. If it's a ceramic cup, that's good for retaining heat. Glass might be nice if I want to see the tea. Also, maybe I should clean the teapot and tools afterwards to keep them ready for next time.\n" +
      '\n' +
      "I think that's the basic process. Let me summarize the steps to make sure I didn't miss anything.\n" +
      '</think>\n' +
      '\n' +
      'To make a cup of Earl Grey tea, follow these organized steps:\n' +
      '\n' +
      '1. **Boil Water**: Fill a kettle with water and bring it to a boil. Allow the water to cool slightly to around 90-95°C (194-203°F) to avoid bitterness.\n' +
      '\n' +
      '2. **Prepare the Teapot**: Warm the teapot by pouring a small amount of hot water into it, then discard the water.\n' +
      '\n' +
      '3. **Add Tea**: Place 1 teaspoon of loose Earl Grey tea leaves into a tea infuser or add one Earl Grey tea bag to the teapot.\n' +
      '\n' +
      '4. **Steep the Tea**: Pour the hot water over the tea and cover the teapot. Steep for 3-5 minutes to achieve the desired strength.\n' +
      '\n' +
      '5. **Serve**: Carefully pour the tea into a cup. Remove the infuser or tea bag.\n' +
      '\n' +
      '6. **Customize**: Add sugar, honey, milk, or lemon as preferred.\n' +
      '\n' +
      '7. **Clean Up**: Rinse the teapot and infuser to keep them ready for future use.\n' +
      '\n' +
      'Enjoy your Earl Grey tea!',
    usage: { prompt_tokens: 20, completion_tokens: 761, total_tokens: 781 }
  }
}

(info) generateCloudflare {
  stream: {
    response: 'Okay, so I need to make a cup of Earl Grey tea using the tools provided. Hmm, let me think about what tools I might have. Usually, when making tea, you need a kettle, a teapot, a teacup, tea bags or loose tea, hot water, maybe a strainer, and sugar or milk if you want it. \n' +
      '\n' +
      "Wait, the user said to use the tool provided, but didn't specify what that tool is. Maybe it's just a regular teapot? Or maybe a tea infuser? I'm not sure. Well, I guess I'll go with the basics.\n" +
      '\n' +
      "First, I should probably boil some water. If I have a kettle, I can fill it with water and turn it on. While the water is boiling, I can prepare the teapot. If I'm using loose tea, I'll need a tea infuser or a strainer to prevent the leaves from getting into the cup. \n" +
      '\n' +
      "Once the water is boiling, I should let it cool a bit because Earl Grey tea is best when brewed with water that's just below boiling point, maybe around 90-95 degrees Celsius. If I pour boiling water directly on the tea, it might make it too strong or bitter. \n" +
      '\n' +
      "So, I'll pour a little bit of hot water into the teapot to warm it up and then discard that water. Then, I'll add the tea leaves or the tea bag. If I'm using loose tea, I'll put the infuser in the teapot. For a standard cup, maybe about a teaspoon of loose tea or one tea bag. \n" +
      '\n' +
      "Next, I'll pour the hot water over the tea. I'll make sure to cover the teapot to keep the heat in. The steeping time is important. Earl Grey usually takes about 3-5 minutes. If I steep it too long, it might become too strong.\n" +
      '\n' +
      "After steeping, I'll pour the tea into the cup. If I used an infuser, I'll remove it now. I can add sugar or honey if I want it sweet, or maybe a splash of milk. Some people like it with lemon, so that's an option too.\n" +
      '\n' +
      "Wait, did I forget anything? Oh, maybe I should consider the type of cup. If it's a ceramic cup, that's good for retaining heat. Glass might be nice if I want to see the tea. Also, maybe I should clean the teapot and tools afterwards to keep them ready for next time.\n" +
      '\n' +
      "I think that's the basic process. Let me summarize the steps to make sure I didn't miss anything.\n" +
      '</think>\n' +
      '\n' +
      'To make a cup of Earl Grey tea, follow these organized steps:\n' +
      '\n' +
      '1. **Boil Water**: Fill a kettle with water and bring it to a boil. Allow the water to cool slightly to around 90-95°C (194-203°F) to avoid bitterness.\n' +
      '\n' +
      '2. **Prepare the Teapot**: Warm the teapot by pouring a small amount of hot water into it, then discard the water.\n' +
      '\n' +
      '3. **Add Tea**: Place 1 teaspoon of loose Earl Grey tea leaves into a tea infuser or add one Earl Grey tea bag to the teapot.\n' +
      '\n' +
      '4. **Steep the Tea**: Pour the hot water over the tea and cover the teapot. Steep for 3-5 minutes to achieve the desired strength.\n' +
      '\n' +
      '5. **Serve**: Carefully pour the tea into a cup. Remove the infuser or tea bag.\n' +
      '\n' +
      '6. **Customize**: Add sugar, honey, milk, or lemon as preferred.\n' +
      '\n' +
      '7. **Clean Up**: Rinse the teapot and infuser to keep them ready for future use.\n' +
      '\n' +
      'Enjoy your Earl Grey tea!',
    usage: { prompt_tokens: 20, completion_tokens: 761, total_tokens: 781 }
  }
}

(info) generateCloudflare {
  stream: {
    response: 'Okay, so I need to make a cup of Earl Grey tea using the tools provided. Hmm, let me think about what tools I might have. Usually, when making tea, you need a kettle, a teapot, a teacup, tea bags or loose tea, hot water, maybe a strainer, and sugar or milk if you want it. \n' +
      '\n' +
      "Wait, the user said to use the tool provided, but didn't specify what that tool is. Maybe it's just a regular teapot? Or maybe a tea infuser? I'm not sure. Well, I guess I'll go with the basics.\n" +
      '\n' +
      "First, I should probably boil some water. If I have a kettle, I can fill it with water and turn it on. While the water is boiling, I can prepare the teapot. If I'm using loose tea, I'll need a tea infuser or a strainer to prevent the leaves from getting into the cup. \n" +
      '\n' +
      "Once the water is boiling, I should let it cool a bit because Earl Grey tea is best when brewed with water that's just below boiling point, maybe around 90-95 degrees Celsius. If I pour boiling water directly on the tea, it might make it too strong or bitter. \n" +
      '\n' +
      "So, I'll pour a little bit of hot water into the teapot to warm it up and then discard that water. Then, I'll add the tea leaves or the tea bag. If I'm using loose tea, I'll put the infuser in the teapot. For a standard cup, maybe about a teaspoon of loose tea or one tea bag. \n" +
      '\n' +
      "Next, I'll pour the hot water over the tea. I'll make sure to cover the teapot to keep the heat in. The steeping time is important. Earl Grey usually takes about 3-5 minutes. If I steep it too long, it might become too strong.\n" +
      '\n' +
      "After steeping, I'll pour the tea into the cup. If I used an infuser, I'll remove it now. I can add sugar or honey if I want it sweet, or maybe a splash of milk. Some people like it with lemon, so that's an option too.\n" +
      '\n' +
      "Wait, did I forget anything? Oh, maybe I should consider the type of cup. If it's a ceramic cup, that's good for retaining heat. Glass might be nice if I want to see the tea. Also, maybe I should clean the teapot and tools afterwards to keep them ready for next time.\n" +
      '\n' +
      "I think that's the basic process. Let me summarize the steps to make sure I didn't miss anything.\n" +
      '</think>\n' +
      '\n' +
      'To make a cup of Earl Grey tea, follow these organized steps:\n' +
      '\n' +
      '1. **Boil Water**: Fill a kettle with water and bring it to a boil. Allow the water to cool slightly to around 90-95°C (194-203°F) to avoid bitterness.\n' +
      '\n' +
      '2. **Prepare the Teapot**: Warm the teapot by pouring a small amount of hot water into it, then discard the water.\n' +
      '\n' +
      '3. **Add Tea**: Place 1 teaspoon of loose Earl Grey tea leaves into a tea infuser or add one Earl Grey tea bag to the teapot.\n' +
      '\n' +
      '4. **Steep the Tea**: Pour the hot water over the tea and cover the teapot. Steep for 3-5 minutes to achieve the desired strength.\n' +
      '\n' +
      '5. **Serve**: Carefully pour the tea into a cup. Remove the infuser or tea bag.\n' +
      '\n' +
      '6. **Customize**: Add sugar, honey, milk, or lemon as preferred.\n' +
      '\n' +
      '7. **Clean Up**: Rinse the teapot and infuser to keep them ready for future use.\n' +
      '\n' +
      'Enjoy your Earl Grey tea!',
    usage: { prompt_tokens: 20, completion_tokens: 761, total_tokens: 781 }
  }
}

D

dmaretskyi•3/13/25, 7:07 PM

I noticed that the opening <think> tag is missing

D

dmaretskyi•3/13/25, 7:07 PM

but also the model doesn't seem to recognize that it has access to a tool

I

Isaac McFadyen•3/13/25, 7:13 PM

Hmm yeah. Interesting.

I

Isaac McFadyen•3/13/25, 7:14 PM

Not sure about the latter but I'm guessing the leading <think> tag is missing because it's part of the prompt used by Workers AI (this is what tools like llama.cpp do in order to force the model to start it's response with thinking).

I

Isaac McFadyen•3/13/25, 7:14 PM

That said, for compatibility it should be manually added by Workers AI if it's part of the prompt.

I

Isaac McFadyen•3/13/25, 7:14 PM

CC @michelle ^ not 100% sure that's what's happening, but since multiple users are running into this it might be worth a bug fix

IIsaac McFadyen CC @michelle ^ not 100% sure that's what's happening, but since multiple users a...

I

Isaac McFadyen•3/14/25, 5:16 AM

Per https://huggingface.co/blog/open-r1/update-3

Incidentally, this is also what DeepSeek does with their distilled models

So yes, guessing that the <think> tag is just being prefilled and Workers AI isn't adding it back into the response.

K

karenr•3/14/25, 5:47 AM

quick question. from the cloudflare workers ai, what's the best model for image compression?

Kkarenr quick question. from the cloudflare workers ai, what's the best model for image ...

T

Torianna•3/14/25, 10:50 AM

You wouldn't want to use AI to compress, you'd want to use something like WebP, AVIF, etc

T

Torianna•3/14/25, 10:50 AM

Those repackage your image into a smaller file, while keeping about the same quality

T

Torianna•3/14/25, 10:52 AM

Now if you want to compress inside Workers, that's a little bit more tricky, but you can look up Cloudflare Images. You get free (I think) image transformations so long as you don't store images there

TTorianna You wouldn't want to use AI to compress, you'd want to use something like WebP, ...

K

karenr•3/14/25, 10:57 AM

Yeah, this is what I actually did. was wondering if using AI for compression is a better choice. Thanks @Torianna

T

Torianna•3/14/25, 10:57 AM

I don't think there's any public models

T

Torianna•3/14/25, 10:58 AM

I know Google is working on one, but that's internal

K

karenr•3/14/25, 11:01 AM

Google - oh really?
yeah.. current models are multimodal AI model with both vision and language capabilities not specific to compression

TTorianna I don't think there's any public models

I

Isaac McFadyen•3/14/25, 3:24 PM

Nvidia has one and it's teeeechnically public but not really? i.e. it's integrated into their new Neural Shader stuff but I don't believe the model is open-weight.

ah! default `max_tokens` for the result is only 256. bumping it up fixed it. there is no indication

Similar Threads