0.000015$ per token, does that mean per word? - Cloudflare Developers

© 2026 Hedgehog Software, LLC

More

Communities Docs About Terms Privacy

James

J

James•1/6/25, 12:03 AM

Hopefully we’ll get some clarification soon - this was meant to happen November 1st but didn’t

James

J

James•1/6/25, 12:03 AM

I don’t know

bat ✞

B

bat ✞OP•1/6/25, 12:03 AM

Oh alright

bat ✞

B

bat ✞OP•1/6/25, 12:03 AM

I just need to know before we start paying

James

J

James•1/6/25, 12:04 AM

Sorry I can’t be of more help. We’re (the community) all just as confused as you

bat ✞

B

bat ✞OP•1/6/25, 12:04 AM

As we don't wanna go around spending too much, though 1 token per word seems more reasonable

James

JJames Sorry I can’t be of more help. We’re (the community) all just as confused as you

bat ✞

B

bat ✞OP•1/6/25, 12:04 AM

Yeah it's alright thank you alot

bat ✞

Bbat ✞0.000015$ per token, does that mean per word?

Chaika

C

Chaika•1/6/25, 12:31 AM

This is a more generic AI/LLM question, tokens are their sort of base building blocks, not always matching up with a single word: https://www.johno.com/tokens-vs-words

Chaika

CChaika This is a more generic AI/LLM question, tokens are their sort of base building b...

bat ✞

B

bat ✞OP•1/6/25, 12:33 AM

Thank you!
But for example, if I used only latin and symbols, how much would a token be?

Chaika

C

Chaika•1/6/25, 12:33 AM

Depends on the model you use/it's tokenizer

Chaika

C

Chaika•1/6/25, 12:35 AM

For example, OpenAI's tokenizers are like this: https://platform.openai.com/tokenizer, cool visulization to play around with

Chaika

C

Chaika•1/6/25, 12:36 AM

one for a model that is on Worker's AI, Lllama 3's https://lunary.ai/llama3-tokenizer
Mistral and Gemma's linked on that one too

Chaika

CChaika one for a model that is on Worker's AI, Lllama 3's https://lunary.ai/llama3-toke...

bat ✞

B

bat ✞OP•1/6/25, 12:55 AM

What about mixtral's?

bat ✞

Bbat ✞What about mixtral's?

Isaac McFadyen

I

Isaac McFadyen•1/6/25, 1:29 AM

Mixtral uses the Mistral tokenizer.

Isaac McFadyen

I

Isaac McFadyen•1/6/25, 1:30 AM

So it'll be the same as the Mistral button on that page.

Isaac McFadyen

IIsaac McFadyen Mixtral uses the Mistral tokenizer.

bat ✞

B

bat ✞OP•1/6/25, 1:31 AM

Oh alright thanks

kyle

K

kyle•1/7/25, 6:26 PM

I noticed that llama 3.1-8B is out of beta as of the past month or so, but I don't see any reference to this in the Workers AI changelog. imo a billing-impacting change like this needs to be mentioned in the changelog

Cloudflare Docs

Changelog · Cloudflare Workers AI docs

Review recent changes to Cloudflare Workers AI.

blindChicken

B

blindChicken•1/7/25, 9:03 PM

I don't see any reranking models in the models offered by CF. Is there any plan to add one or more in the near future? BAAI embedding models are offered, it would be great to also offer their reranking model: https://huggingface.co/BAAI/bge-reranker-base.

Chaika

CChaika hmm guessing it's still cooking? It throws ` "message": "5006: must be st...

zabatonni

Z

zabatonni•1/8/25, 6:27 PM

not working for me, what it should be? I tried uint8, base64 but nothing seems to work

LordQ

L

LordQ•1/9/25, 10:21 AM

Hi, do you know how to get more than only 5 labels and scores as the result?

Destiny

D

Destiny•1/10/25, 9:31 AM

does setting width and height to 2048 result in a 500 error?

Destiny

D

Destiny•1/10/25, 9:31 AM

i am on a free plan

Destiny

D

Destiny•1/10/25, 9:37 AM

also when i set the width to 1600 and height to 900. i seem to get this error

AiError: AiError: Invalid or incomplete input for the model: model returned: Invalid input

AiError: AiError: Invalid or incomplete input for the model: model returned: Invalid input

ella

E

ella•1/10/25, 1:33 PM

i have cf enterprise. is there anyway to upload my own mode lto workers ai?

ella

E

ella•1/10/25, 1:44 PM

bruh

Destiny

DDestiny also when i set the width to 1600 and height to 900. i seem to get this error `...

Raylight

R

Raylight•1/10/25, 3:20 PM

Make sure width and height are multiples of 8, e.g. try 1600 x 896 instead.

Korcan

KKorcan `InferenceUpstreamError: undefined: undefined` ``` at Ai._parseError (cloudflare...

Chris Jayden

C

Chris Jayden•1/11/25, 12:39 PM

Did you ever figure this error out? I'm also getting it locally when trying to work with AI...

My dev command is:

wrangler dev --port 8787  --experimental-vectorize-bind-to-prod --inspector-port 9229

wrangler dev --port 8787  --experimental-vectorize-bind-to-prod --inspector-port 9229

user

U

user•1/12/25, 2:13 AM

Can you not pass previous messages to AI when using LoRA? Every time I try, the message content is empty.

user

U

user•1/12/25, 2:13 AM

It doesnt return any errors, just nothing.

user

U

user•1/12/25, 2:15 AM

also, i keep constantly getting error

when using LoRA

Aleksandar

AAleksandar Hello. Was this issue with `InferenceUpstreamError: undefined: undefined` fixed?...

flashblaze

F

flashblaze•1/15/25, 2:07 AM

Were you able to fix this? I have a similar code using Hono

app.post(
    '/get-alt',
    zValidator('form', getAltSchema),
    async (c) => {
        const { image } = c.req.valid('form')

        try {
            let imageData: ArrayBuffer;
            if (image instanceof File) {
                imageData = await image.arrayBuffer();
            } else {
                const response = await fetch(image.toString());
                if (!response.ok) {
                    return c.json({ error: 'Failed to fetch image from URL' }, 400);
                }
                imageData = await response.arrayBuffer();
            }

            const imageBytes = new Uint8Array(imageData);

            try {
                const response = await c.env.AI.run('@cf/meta/llama-3.2-11b-vision-instruct', {
                    prompt: 'Create a concise alt text for this image. Use simple language and keep it under 160 characters.',
                    image: imageBytes
                });

                return c.json(response);
            } catch (error) {
                console.error('AI processing error:', error);
                return c.json({ error: 'Failed to process image with AI model' }, 500);
            }
        } catch (error) {
            console.error('Image processing error:', error);
            return c.json({ error: 'Failed to process image data' }, 500);
        }
    }
)

app.post(
    '/get-alt',
    zValidator('form', getAltSchema),
    async (c) => {
        const { image } = c.req.valid('form')

        try {
            let imageData: ArrayBuffer;
            if (image instanceof File) {
                imageData = await image.arrayBuffer();
            } else {
                const response = await fetch(image.toString());
                if (!response.ok) {
                    return c.json({ error: 'Failed to fetch image from URL' }, 400);
                }
                imageData = await response.arrayBuffer();
            }

            const imageBytes = new Uint8Array(imageData);

            try {
                const response = await c.env.AI.run('@cf/meta/llama-3.2-11b-vision-instruct', {
                    prompt: 'Create a concise alt text for this image. Use simple language and keep it under 160 characters.',
                    image: imageBytes
                });

                return c.json(response);
            } catch (error) {
                console.error('AI processing error:', error);
                return c.json({ error: 'Failed to process image with AI model' }, 500);
            }
        } catch (error) {
            console.error('Image processing error:', error);
            return c.json({ error: 'Failed to process image data' }, 500);
        }
    }
)

And each time I'm sending an image, getting this error. No matter what size I send.

AI processing error: InferenceUpstreamError: undefined: undefined

      at Ai._parseError (cloudflare-internal:ai-api:81:20)
      at async Ai.run (cloudflare-internal:ai-api:61:23)
      at null.<

AI processing error: InferenceUpstreamError: undefined: undefined

      at Ai._parseError (cloudflare-internal:ai-api:81:20)
      at async Ai.run (cloudflare-internal:ai-api:61:23)
      at null.<

kai

K

kai•1/15/25, 7:32 AM

Is it possible to run our own model using workers ai or we can only use the available open source models?

flashblaze

Fflashblaze Were you able to fix this? I have a similar code using Hono ```ts app.post( ...

lord5et

L

lord5et•1/15/25, 8:34 AM

Any ETA on fixing this? I have same issue with following hono code:

app.get("/transcribe", async (c: Context) => {
  const res = await fetch(
    "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/samples/cpp/windows/console/samples/enrollment_audio_katie.wav"
  );
  const blob = await res.arrayBuffer();

  const input = {
    audio: Buffer.from(blob).toString("base64"),
  };
  console.log("AI");
  console.log(c.env);
  console.log(c.env.AI);
  const response = await c.env.AI.run(
    "@cf/openai/whisper",
    input
  );

  return Response.json({ input: { audio: [] }, response });
});

app.get("/transcribe", async (c: Context) => {
  const res = await fetch(
    "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/samples/cpp/windows/console/samples/enrollment_audio_katie.wav"
  );
  const blob = await res.arrayBuffer();

  const input = {
    audio: Buffer.from(blob).toString("base64"),
  };
  console.log("AI");
  console.log(c.env);
  console.log(c.env.AI);
  const response = await c.env.AI.run(
    "@cf/openai/whisper",
    input
  );

  return Response.json({ input: { audio: [] }, response });
});

it worked fine yesterday morning

lord5et

L

lord5et•1/15/25, 8:58 AM

What is the file size limit when it comes to @cf/openai/whisper model?

flashblaze

Fflashblaze Were you able to fix this? I have a similar code using Hono ```ts app.post( ...

Raylight

R

Raylight•1/16/25, 9:01 AM

Try changing

image: imageBytes

image: imageBytes

to

image: [...imageBytes]

image: [...imageBytes]

.

lord5et

Llord5et What is the file size limit when it comes to @cf/openai/whisper model?

Raylight

R

Raylight•1/16/25, 9:41 AM

The file size limit was roughly 2.6MB last time I tested, but it depends on the content and whether or not it's supplied as a JS array. See workers-ai. If it's possible to use base64, the limit should be higher. I haven't tested it, though.

flashblaze

Fflashblaze Were you able to fix this? I have a similar code using Hono ```ts app.post( ...

dmaretskyi

D

dmaretskyi•1/16/25, 10:55 AM

seeing the same error with stable-diffusion model

flashblaze

Fflashblaze Were you able to fix this? I have a similar code using Hono ```ts app.post( ...

Siddanth Reddy

S

Siddanth Reddy•1/16/25, 2:05 PM

facing same error with @cf/meta/llama-3.1-8b-instruct-awq

✘ [ERROR] Error in /chat handler: InferenceUpstreamError: undefined: undefined

at Ai._parseError (cloudflare-internal:ai-api:76:20)
at async Ai.run (cloudflare-internal:ai-api:57:23)
at async _ContextManager.analyzeQuery

Siddanth Reddy

S

Siddanth Reddy•1/16/25, 2:29 PM

and same issue with : @cf/mistral/mistral-7b-instruct-v0.1

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined

Siddanth Reddy

S

Siddanth Reddy•1/16/25, 2:30 PM

To force the models to respond, I sent the same message multiple times (around four attempts), which temporarily resolved the issue. However, the problem appears again after a few prompts

flashblaze

F

flashblaze•1/16/25, 10:38 PM

I'm surprised CF has still not addressed this issue. No reply on the community post as well
https://community.cloudflare.com/t/x-error-inferenceupstreamerror-undefined-undefined/758340/2

drdiffie

D

drdiffie•1/17/25, 5:54 AM

Hey guys new here. Is there any documentation or threads somewhere detailing who decide which new open source models get listed for deployment? Can we suggest models or only via the custom enterprise form?

flashblaze

Fflashblaze I'm surprised CF has still not addressed this issue. No reply on the community p...

Siddanth Reddy

S

Siddanth Reddy•1/17/25, 11:40 AM

Yes, the error is still occurring

/ sssssslboom

/

/ sssssslboom•1/17/25, 6:06 PM

Hi, nice to meet you.

/ sssssslboom

/

/ sssssslboom•1/17/25, 6:06 PM

@andrew_nyr might help me?

/ sssssslboom

// sssssslboom @andrew_nyr might help me?

Idle

I

Idle•1/17/25, 6:10 PM

?pings

SuperHelpflare

S

SuperHelpflare•1/17/25, 6:10 PM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

SimonBeno

S

SimonBeno•1/19/25, 10:35 PM

Hello people! New to Workers AI

what are the limits for bge-large embedding models?

In our app, we need to be able to get embeddings for 1 - 2M tokens of text per minute (peak), are the limits high enough?

Super Morris

S

Super Morris•1/21/25, 1:28 PM

When will DeepSeek-R1 be available?

SimonBeno

SSimonBeno Hello people! New to Workers AI what are the limits for bge-large embedding mod...

Bob

B

Bob•1/21/25, 7:37 PM

as per the docs, it has a limit of 1500 req/min. For 2M token, you will need chunk size of 1.3k tokens (or ~1k words).
ref: https://developers.cloudflare.com/workers-ai/platform/limits/
they also might increase the limit for you

app.post(
    '/get-alt',
    zValidator('form', getAltSchema),
    async (c) => {
        const { image } = c.req.valid('form')

        try {
            let imageData: ArrayBuffer;
            if (image instanceof File) {
                imageData = await image.arrayBuffer();
            } else {
                const response = await fetch(image.toString());
                if (!response.ok) {
                    return c.json({ error: 'Failed to fetch image from URL' }, 400);
                }
                imageData = await response.arrayBuffer();
            }

            const imageBytes = new Uint8Array(imageData);

            try {
                const response = await c.env.AI.run('@cf/meta/llama-3.2-11b-vision-instruct', {
                    prompt: 'Create a concise alt text for this image. Use simple language and keep it under 160 characters.',
                    image: imageBytes
                });

                return c.json(response);
            } catch (error) {
                console.error('AI processing error:', error);
                return c.json({ error: 'Failed to process image with AI model' }, 500);
            }
        } catch (error) {
            console.error('Image processing error:', error);
            return c.json({ error: 'Failed to process image data' }, 500);
        }
    }
)

app.post(
    '/get-alt',
    zValidator('form', getAltSchema),
    async (c) => {
        const { image } = c.req.valid('form')

        try {
            let imageData: ArrayBuffer;
            if (image instanceof File) {
                imageData = await image.arrayBuffer();
            } else {
                const response = await fetch(image.toString());
                if (!response.ok) {
                    return c.json({ error: 'Failed to fetch image from URL' }, 400);
                }
                imageData = await response.arrayBuffer();
            }

            const imageBytes = new Uint8Array(imageData);

            try {
                const response = await c.env.AI.run('@cf/meta/llama-3.2-11b-vision-instruct', {
                    prompt: 'Create a concise alt text for this image. Use simple language and keep it under 160 characters.',
                    image: imageBytes
                });

                return c.json(response);
            } catch (error) {
                console.error('AI processing error:', error);
                return c.json({ error: 'Failed to process image with AI model' }, 500);
            }
        } catch (error) {
            console.error('Image processing error:', error);
            return c.json({ error: 'Failed to process image data' }, 500);
        }
    }
)

app.get("/transcribe", async (c: Context) => {
  const res = await fetch(
    "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/samples/cpp/windows/console/samples/enrollment_audio_katie.wav"
  );
  const blob = await res.arrayBuffer();

  const input = {
    audio: Buffer.from(blob).toString("base64"),
  };
  console.log("AI");
  console.log(c.env);
  console.log(c.env.AI);
  const response = await c.env.AI.run(
    "@cf/openai/whisper",
    input
  );

  return Response.json({ input: { audio: [] }, response });
});

app.get("/transcribe", async (c: Context) => {
  const res = await fetch(
    "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/samples/cpp/windows/console/samples/enrollment_audio_katie.wav"
  );
  const blob = await res.arrayBuffer();

  const input = {
    audio: Buffer.from(blob).toString("base64"),
  };
  console.log("AI");
  console.log(c.env);
  console.log(c.env.AI);
  const response = await c.env.AI.run(
    "@cf/openai/whisper",
    input
  );

  return Response.json({ input: { audio: [] }, response });
});