also when i set the width to 1600 and height to 900. i seem to get this error `AiError: AiError: In

E

ella•1/10/25, 1:33 PM

i have cf enterprise. is there anyway to upload my own mode lto workers ai?

E

ella•1/10/25, 1:44 PM

bruh

DDestiny also when i set the width to 1600 and height to 900. i seem to get this error `...

R

Raylight•1/10/25, 3:20 PM

Make sure width and height are multiples of 8, e.g. try 1600 x 896 instead.

KKorcan `InferenceUpstreamError: undefined: undefined` ``` at Ai._parseError (cloudflare...

C

Chris Jayden•1/11/25, 12:39 PM

Did you ever figure this error out? I'm also getting it locally when trying to work with AI...

My dev command is:

wrangler dev --port 8787  --experimental-vectorize-bind-to-prod --inspector-port 9229

wrangler dev --port 8787  --experimental-vectorize-bind-to-prod --inspector-port 9229

wrangler dev --port 8787  --experimental-vectorize-bind-to-prod --inspector-port 9229

wrangler dev --port 8787  --experimental-vectorize-bind-to-prod --inspector-port 9229

U

user•1/12/25, 2:13 AM

Can you not pass previous messages to AI when using LoRA? Every time I try, the message content is empty.

U

user•1/12/25, 2:13 AM

It doesnt return any errors, just nothing.

U

user•1/12/25, 2:15 AM

also, i keep constantly getting error

3028

3028 when using LoRA

AAleksandar Hello. Was this issue with `InferenceUpstreamError: undefined: undefined` fixed?...

F

flashblaze•1/15/25, 2:07 AM

Were you able to fix this? I have a similar code using Hono

app.post(
    '/get-alt',
    zValidator('form', getAltSchema),
    async (c) => {
        const { image } = c.req.valid('form')

        try {
            let imageData: ArrayBuffer;
            if (image instanceof File) {
                imageData = await image.arrayBuffer();
            } else {
                const response = await fetch(image.toString());
                if (!response.ok) {
                    return c.json({ error: 'Failed to fetch image from URL' }, 400);
                }
                imageData = await response.arrayBuffer();
            }

            const imageBytes = new Uint8Array(imageData);

            try {
                const response = await c.env.AI.run('@cf/meta/llama-3.2-11b-vision-instruct', {
                    prompt: 'Create a concise alt text for this image. Use simple language and keep it under 160 characters.',
                    image: imageBytes
                });

                return c.json(response);
            } catch (error) {
                console.error('AI processing error:', error);
                return c.json({ error: 'Failed to process image with AI model' }, 500);
            }
        } catch (error) {
            console.error('Image processing error:', error);
            return c.json({ error: 'Failed to process image data' }, 500);
        }
    }
)

app.post(
    '/get-alt',
    zValidator('form', getAltSchema),
    async (c) => {
        const { image } = c.req.valid('form')

        try {
            let imageData: ArrayBuffer;
            if (image instanceof File) {
                imageData = await image.arrayBuffer();
            } else {
                const response = await fetch(image.toString());
                if (!response.ok) {
                    return c.json({ error: 'Failed to fetch image from URL' }, 400);
                }
                imageData = await response.arrayBuffer();
            }

            const imageBytes = new Uint8Array(imageData);

            try {
                const response = await c.env.AI.run('@cf/meta/llama-3.2-11b-vision-instruct', {
                    prompt: 'Create a concise alt text for this image. Use simple language and keep it under 160 characters.',
                    image: imageBytes
                });

                return c.json(response);
            } catch (error) {
                console.error('AI processing error:', error);
                return c.json({ error: 'Failed to process image with AI model' }, 500);
            }
        } catch (error) {
            console.error('Image processing error:', error);
            return c.json({ error: 'Failed to process image data' }, 500);
        }
    }
)

app.post(
    '/get-alt',
    zValidator('form', getAltSchema),
    async (c) => {
        const { image } = c.req.valid('form')

        try {
            let imageData: ArrayBuffer;
            if (image instanceof File) {
                imageData = await image.arrayBuffer();
            } else {
                const response = await fetch(image.toString());
                if (!response.ok) {
                    return c.json({ error: 'Failed to fetch image from URL' }, 400);
                }
                imageData = await response.arrayBuffer();
            }

            const imageBytes = new Uint8Array(imageData);

            try {
                const response = await c.env.AI.run('@cf/meta/llama-3.2-11b-vision-instruct', {
                    prompt: 'Create a concise alt text for this image. Use simple language and keep it under 160 characters.',
                    image: imageBytes
                });

                return c.json(response);
            } catch (error) {
                console.error('AI processing error:', error);
                return c.json({ error: 'Failed to process image with AI model' }, 500);
            }
        } catch (error) {
            console.error('Image processing error:', error);
            return c.json({ error: 'Failed to process image data' }, 500);
        }
    }
)

app.post(
    '/get-alt',
    zValidator('form', getAltSchema),
    async (c) => {
        const { image } = c.req.valid('form')

        try {
            let imageData: ArrayBuffer;
            if (image instanceof File) {
                imageData = await image.arrayBuffer();
            } else {
                const response = await fetch(image.toString());
                if (!response.ok) {
                    return c.json({ error: 'Failed to fetch image from URL' }, 400);
                }
                imageData = await response.arrayBuffer();
            }

            const imageBytes = new Uint8Array(imageData);

            try {
                const response = await c.env.AI.run('@cf/meta/llama-3.2-11b-vision-instruct', {
                    prompt: 'Create a concise alt text for this image. Use simple language and keep it under 160 characters.',
                    image: imageBytes
                });

                return c.json(response);
            } catch (error) {
                console.error('AI processing error:', error);
                return c.json({ error: 'Failed to process image with AI model' }, 500);
            }
        } catch (error) {
            console.error('Image processing error:', error);
            return c.json({ error: 'Failed to process image data' }, 500);
        }
    }
)

And each time I'm sending an image, getting this error. No matter what size I send.

AI processing error: InferenceUpstreamError: undefined: undefined

      at Ai._parseError (cloudflare-internal:ai-api:81:20)
      at async Ai.run (cloudflare-internal:ai-api:61:23)
      at null.<

AI processing error: InferenceUpstreamError: undefined: undefined

      at Ai._parseError (cloudflare-internal:ai-api:81:20)
      at async Ai.run (cloudflare-internal:ai-api:61:23)
      at null.<

AI processing error: InferenceUpstreamError: undefined: undefined

      at Ai._parseError (cloudflare-internal:ai-api:81:20)
      at async Ai.run (cloudflare-internal:ai-api:61:23)
      at null.<

AI processing error: InferenceUpstreamError: undefined: undefined

      at Ai._parseError (cloudflare-internal:ai-api:81:20)
      at async Ai.run (cloudflare-internal:ai-api:61:23)
      at null.<

K

kai•1/15/25, 7:32 AM

Is it possible to run our own model using workers ai or we can only use the available open source models?

Fflashblaze Were you able to fix this? I have a similar code using Hono ```ts app.post( ...

L

lord5et•1/15/25, 8:34 AM

Any ETA on fixing this? I have same issue with following hono code:

app.get("/transcribe", async (c: Context) => {
  const res = await fetch(
    "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/samples/cpp/windows/console/samples/enrollment_audio_katie.wav"
  );
  const blob = await res.arrayBuffer();

  const input = {
    audio: Buffer.from(blob).toString("base64"),
  };
  console.log("AI");
  console.log(c.env);
  console.log(c.env.AI);
  const response = await c.env.AI.run(
    "@cf/openai/whisper",
    input
  );

  return Response.json({ input: { audio: [] }, response });
});

app.get("/transcribe", async (c: Context) => {
  const res = await fetch(
    "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/samples/cpp/windows/console/samples/enrollment_audio_katie.wav"
  );
  const blob = await res.arrayBuffer();

  const input = {
    audio: Buffer.from(blob).toString("base64"),
  };
  console.log("AI");
  console.log(c.env);
  console.log(c.env.AI);
  const response = await c.env.AI.run(
    "@cf/openai/whisper",
    input
  );

  return Response.json({ input: { audio: [] }, response });
});

app.get("/transcribe", async (c: Context) => {
  const res = await fetch(
    "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/samples/cpp/windows/console/samples/enrollment_audio_katie.wav"
  );
  const blob = await res.arrayBuffer();

  const input = {
    audio: Buffer.from(blob).toString("base64"),
  };
  console.log("AI");
  console.log(c.env);
  console.log(c.env.AI);
  const response = await c.env.AI.run(
    "@cf/openai/whisper",
    input
  );

  return Response.json({ input: { audio: [] }, response });
});

app.get("/transcribe", async (c: Context) => {
  const res = await fetch(
    "https://github.com/Azure-Samples/cognitive-services-speech-sdk/raw/master/samples/cpp/windows/console/samples/enrollment_audio_katie.wav"
  );
  const blob = await res.arrayBuffer();

  const input = {
    audio: Buffer.from(blob).toString("base64"),
  };
  console.log("AI");
  console.log(c.env);
  console.log(c.env.AI);
  const response = await c.env.AI.run(
    "@cf/openai/whisper",
    input
  );

  return Response.json({ input: { audio: [] }, response });
});

it worked fine yesterday morning

L

lord5et•1/15/25, 8:58 AM

What is the file size limit when it comes to @cf/openai/whisper model?

Fflashblaze Were you able to fix this? I have a similar code using Hono ```ts app.post( ...

R

Raylight•1/16/25, 9:01 AM

Try changing

image: imageBytes

image: imageBytes

image: imageBytes

image: imageBytes to

image: [...imageBytes]

image: [...imageBytes]

image: [...imageBytes]

image: [...imageBytes].

Llord5et What is the file size limit when it comes to @cf/openai/whisper model?

R

Raylight•1/16/25, 9:41 AM

The file size limit was roughly 2.6MB last time I tested, but it depends on the content and whether or not it's supplied as a JS array. See workers-ai. If it's possible to use base64, the limit should be higher. I haven't tested it, though.

Fflashblaze Were you able to fix this? I have a similar code using Hono ```ts app.post( ...

D

dmaretskyi•1/16/25, 10:55 AM

seeing the same error with stable-diffusion model

Fflashblaze Were you able to fix this? I have a similar code using Hono ```ts app.post( ...

S

Siddanth Reddy•1/16/25, 2:05 PM

facing same error with @cf/meta/llama-3.1-8b-instruct-awq

✘ [ERROR] Error in /chat handler: InferenceUpstreamError: undefined: undefined

at Ai._parseError (cloudflare-internal:ai-api:76:20)
at async Ai.run (cloudflare-internal:ai-api:57:23)
at async _ContextManager.analyzeQuery

S

Siddanth Reddy•1/16/25, 2:29 PM

and same issue with : @cf/mistral/mistral-7b-instruct-v0.1

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined

S

Siddanth Reddy•1/16/25, 2:30 PM

To force the models to respond, I sent the same message multiple times (around four attempts), which temporarily resolved the issue. However, the problem appears again after a few prompts

F

flashblaze•1/16/25, 10:38 PM

I'm surprised CF has still not addressed this issue. No reply on the community post as well
https://community.cloudflare.com/t/x-error-inferenceupstreamerror-undefined-undefined/758340/2

D

drdiffie•1/17/25, 5:54 AM

Hey guys new here. Is there any documentation or threads somewhere detailing who decide which new open source models get listed for deployment? Can we suggest models or only via the custom enterprise form?

Fflashblaze I'm surprised CF has still not addressed this issue. No reply on the community p...

S

Siddanth Reddy•1/17/25, 11:40 AM

Yes, the error is still occurring

/

/ sssssslboom•1/17/25, 6:06 PM

Hi, nice to meet you.

/

/ sssssslboom•1/17/25, 6:06 PM

@andrew_nyr might help me?

// sssssslboom @andrew_nyr might help me?

I

Idle•1/17/25, 6:10 PM

?pings

S

SuperHelpflare•1/17/25, 6:10 PM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

S

SimonBeno•1/19/25, 10:35 PM

Hello people! New to Workers AI

what are the limits for bge-large embedding models?

In our app, we need to be able to get embeddings for 1 - 2M tokens of text per minute (peak), are the limits high enough?

S

Super Morris•1/21/25, 1:28 PM

When will DeepSeek-R1 be available?

SSimonBeno Hello people! New to Workers AI what are the limits for bge-large embedding mod...

B

Bob•1/21/25, 7:37 PM

as per the docs, it has a limit of 1500 req/min. For 2M token, you will need chunk size of 1.3k tokens (or ~1k words).
ref: https://developers.cloudflare.com/workers-ai/platform/limits/
they also might increase the limit for you

SSuper Morris When will DeepSeek-R1 be available? 😄

A

Advany•1/21/25, 7:41 PM

I was going to ask the same thing...

SSiddanth Reddy and same issue with : @cf/mistral/mistral-7b-instruct-v0.1 ```InferenceUpstreamE...

S

Siddanth Reddy•1/22/25, 1:07 PM

anyone found any solution related to this error?

BBob as per the docs, it has a limit of 1500 req/min. For 2M token, you will need chu...

S

SimonBeno•1/22/25, 5:55 PM

Thanks, I meant more the limit of tokens per minute. I couldn't find that. Is there a TPM limit for bge-large?

SSiddanth Reddy anyone found any solution related to this error?

R

Raylight•1/23/25, 2:03 PM

No solution, unless you count running

npx wrangler dev --remote

npx wrangler dev --remote

npx wrangler dev --remote

npx wrangler dev --remote where I thus far haven't been able to reproduce it.

As for the issue, I tried to get to the bottom of it, and it looks like only Cloudflare can solve it. It's seemingly random and when checking the debug logs (with

WRANGLER_LOG_SANITIZE=false

WRANGLER_LOG_SANITIZE=false

WRANGLER_LOG_SANITIZE=false

WRANGLER_LOG_SANITIZE=false) I noticed that the error coincides with a 401 unauthorized response from upstream.

Ai._parseError

Ai._parseError

Ai._parseError

Ai._parseError is unable to interpret the format of the error response, leading to "unknown: unknown". (Looks like there's a recent commit that would fix this specific part.)

C

crazyjack12•1/23/25, 3:49 PM

Possible to get DeepSeek-R1?

G

garff•1/23/25, 4:44 PM

Hi, we are thinking about using Worker AI for inference and I'm trying out the different models in the playground. I'm wondering if the inference speed corresponds to what i shown on the screen? Would each request take that long for it to finish? What are the inference speed? How many tokens can I expect to get back in what amount of time? Currenctly I'm testing llama 3.3 70b instruct but I'm open to other models that handles multilingual cases.

Fflashblaze I'm surprised CF has still not addressed this issue. No reply on the community p...

F

flashblaze•1/24/25, 7:49 AM

I'm appalled at the lack of communication from CF
Still nothing from them

R

rob•1/24/25, 2:26 PM

the PM for workers AI is on it .. I'm sure

A

akazwz•1/26/25, 9:30 AM

will Workers AI add more model support function call ？

A

akazwz•1/26/25, 9:30 AM

for now， just one model support that

Aakazwz will Workers AI add more model support function call ？

A

ANBU•1/26/25, 8:16 PM

Official only one model supports function call
But I’ve had good experience with llama models as well (I’m using ai gateway tho)
Only limitations is that i wasn’t able to make it do parallel function calling yet

S

scotto•1/26/25, 8:51 PM

how to ask for limit increase? like RPM

B

Bob•1/27/25, 11:02 AM

Hello, I am using the "bge-large-en-v1.5" model from cloudflare and same model running in local that I got from huggingface via SentenceTransformer.
For same string I am getting different vectors between cloudflare hosted model and local (screenshots - cloudflare ai in postman vs local with https://huggingface.co/BAAI/bge-large-en-v1.5)

Shouldn't these be exactly same?
One possible reason can be that cloudflare is using a older or different version than the hugging face one.
Also, what is the strategy from cloudflare to support these models? If I use the model for my current data set, I should have access to the model for a very long time to query too.