Hello. Was this issue with `InferenceUpstreamError: undefined: undefined` fixed? I'm getting the sam

Hello. Was this issue with

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined fixed? I'm getting the same error using

@cf/microsoft/resnet-50

@cf/microsoft/resnet-50

@cf/microsoft/resnet-50

@cf/microsoft/resnet-50 with a PNG and this code https://gist.github.com/net-tech/0eb5e0caa05c6be931ab373cbb35f360

Gist

Code + error on CF workers AI

Code + error on CF workers AI. GitHub Gist: instantly share code, notes, and snippets.

SSeanShan can anyone help me with this since our service relies on this

Isaac McFadyen•8/8/24, 5:42 PM

Workers AI/Cloudflare won't have any control of this, if it did happen (not sure if it will) it would be due to the underlying model which is Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-incIntel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc and how it's been trained.

Alex Potemkin•8/10/24, 10:46 AM

Hey, I wonder how I can be notified when specific AI models on AI workers will be out of beta?

AAleksandar Hello. Was this issue with `InferenceUpstreamError: undefined: undefined` fixed?...

AleksandarOP•8/10/24, 11:53 AM

Can anyone help with this

anywhichway•8/10/24, 2:14 PM

I have a really basic AI request:

export async function onRequest(context) {
    const input = { prompt: "What is the origin of the phrase Hello, World" }
    const answer = await context.env.AI.run('@cf/meta/llama-3.1-8b-instruct', input);
    return Response.json(answer);
}

export async function onRequest(context) {
    const input = { prompt: "What is the origin of the phrase Hello, World" }
    const answer = await context.env.AI.run('@cf/meta/llama-3.1-8b-instruct', input);
    return Response.json(answer);
}

The function gets called (I can put a debugger break in dev tools ). The AI run never seems to return and I ultimately get the error below. I have authorized wrangler to access my account and confirmed the AI is bound in wrangler.toml:

A ReadableStream branch was created but never consumed. Such branches can be created, for instance, by calling the tee() method on a ReadableStream, or by calling the clone() method on a Request or Response object. If a branch is created but never consumed, it can force the runtime to buffer the entire body of the stream in memory, which may cause the Worker to exceed its memory limit and be terminated. To avoid this, ensure that all branches created are consumed.

A ReadableStream branch was created but never consumed. Such branches can be created, for instance, by calling the tee() method on a ReadableStream, or by calling the clone() method on a Request or Response object. If a branch is created but never consumed, it can force the runtime to buffer the entire body of the stream in memory, which may cause the Worker to exceed its memory limit and be terminated. To avoid this, ensure that all branches created are consumed.

Muhammadreza Haghiri•8/10/24, 5:21 PM

Hey guys, what is up?
I'm curious, is there a way to request to add a model to the list? I'm just looking for opportunities for my models to be added to public lists/tools such as workers AI.

thipperz•8/10/24, 7:21 PM

A good Image Embedding model someday...? Would fit so nicely with Vectorize wouldn't it?

AAleksandar Can anyone help with this

Raylight•8/11/24, 9:20 AM

I can't reproduce "undefined: undefined" but haven't tried running locally. That said, make sure that the image is smaller than 2.5-3 MB. (JSON.stringify(inputs).lengthJSON.stringify(inputs).length needs to be less than 9.5 MB, to be precise.)

JCBahn•8/11/24, 9:32 AM

When I make a query in a non-English language, the response comes out broken. Has anyone else experienced this? This doesn't happen when I use llama3.1 with ollama. Please improve support for non-English queries. Especially for Llama 3.1.

RRaylight I can't reproduce "undefined: undefined" but haven't tried running locally. That...

AleksandarOP•8/11/24, 12:12 PM

that was happening locally. ill check the image size and get back to you

shellsort•8/11/24, 12:34 PM

Heyhey!
I was wondering if/when the documentation for this endpoint will be updated:
https://developers.cloudflare.com/api/operations/workers-ai-post-run-model

The responses are not correct.
Successful responses return more than result, they also return a success, errors and messages code where I can only assume that the errors and messages content is strings (thou it might very well be status codes of some sort) and the result contains a model-specific response.

For example for the M2M100-1.2B translation model it is translated_text (which is somewhat documented in the api schema definition of the worker: https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/)
For unsuccessful responses thou, the result is really weird. On wrong input I got this for example:
{
"result": {
"translated_text": "ERROR: invalid source_lang (English)"
},
"success": true,
"errors": [],
"messages": []
}

which is, very confusing and odd to use. It says success, even thou there's an error. The errors section is empty. The messages section is empty and the message itsself is in the model result. :/

It is also unclear which input values for this model are allowed.
I found this (https://github.com/cloudflare/cloudflare-docs/blob/production/content/workers-ai/models/m2m100-1.2b.md), but it doesnt seem to list all languages.
I also found this (https://github.com/ikergarcia1996/Easy-Translate/blob/main/supported_languages.md#supported-languages-m2m100), but the cloudflare version only seems to work with flatcase conventions and the "ld" abbreviations. It's also unclear if these abbreviations follow ISO 639, some other standard or if they are arbitrary.

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

GitHub

Easy-Translate/supported_languages.md at main · ikergarcia1996/Easy...

Easy-Translate is a script for translating large text files with a SINGLE COMMAND. Easy-Translate is designed to be as easy as possible for beginners and as seamlesscustomizable and as possible fo...

Treetop•8/11/24, 4:27 PM

Does anyone know the pricing for whisper-tiny-en?

Devon•8/11/24, 10:03 PM

is there any support for dialects in the translation AI model? Standard locale codes don't seem to be valid. Things like Brazilian Portuguese or the Hong Kong or Tawainese variants of Chinese.

Sshellsort Heyhey! I was wondering if/when the documentation for this endpoint will be upda...

shellsort•8/12/24, 12:53 AM

There's a list of supported languages in here somewhere

Devon•8/12/24, 12:28 PM

Oh I hadn't seen your comment previously. Yeah I couldn't find a list. It does seem like, contrary to most of the docs, that it only takes lowercase two char codes. Not wanting to look a gift horse in the mouth, but have the ability to translate Portugal Portugese and Brazil Portugese separately would be very nice (along with the variants of zh, etc... )

shellsort•8/12/24, 2:21 PM

Full names work for me.

shellsort•8/12/24, 2:21 PM

lowercase

Devon•8/12/24, 2:57 PM

ah, I've been using the two letter codes successfully. but no "pt_BR" or "zh_TW" etc...

kian•8/12/24, 3:26 PM

https://developers.cloudflare.com/turnstile/reference/supported-languages/#supported-languages

kian•8/12/24, 3:26 PM

zh-twzh-tw and pt-brpt-br

Devon•8/12/24, 3:30 PM

Doesn't work for the Translation AI: "ERROR: invalid target_lang (zh-tw)"

kian•8/12/24, 3:46 PM

Oops, I thought I was in the Turnstile channel.

kian•8/12/24, 3:46 PM

In that case, all of the languages supported by the model should be on https://huggingface.co/facebook/m2m100_1.2B#languages-covered which indeed doesn't cover dialects.

Devon•8/12/24, 3:47 PM

ah, thank you very much!

Bbradyjoslin Recommend adding the function calling pill visualization to llama-3.1-8b-instruc...

Craig Dennis•8/12/24, 9:35 PM

We're still working out the kinks before we add the pill, but thank you for the specific and thoughtful callout!

anirudh•8/13/24, 10:37 AM

Which version of Whisper is Workers AI using? V2 large or V3 large?

Aanirudh Which version of Whisper is Workers AI using? V2 large or V3 large?

Isaac McFadyen•8/13/24, 12:58 PM

Neither. It's base as far as I know.

Isaac McFadyen•8/13/24, 12:58 PM

Reference: https://canary.discord.com/channels/595317990191398933/1105477009964027914/1225461479730446427

Vero•8/13/24, 6:08 PM

Hey everyone, in case you haven’t seen it, we recently published a video on How to use Cloudflare AI models and inference in Python with Jupyter Notebooks

https://youtu.be/CHfKeFakGAI

YouTubeCloudflare Developers

How to use Cloudflare AI models and inference in Python with Jupyte...

Cloudflare Workers AI provides a ton of AI models and inference capabilities. In this video, we will explore how to make use of Cloudflare’s AI model catalog using a Python Jupyter Notebook. And remember, inference is free for all models in beta!

Create an account on Cloudflare today for free: https://dash.cloudflare.com/sign-up

Useful links m...

Uncle Sams Tech LLC•8/14/24, 12:09 AM

What is the best AI model for OCR kind of capability. Currently "image to text" only gives you summary, not the exact extracted text.

Uncle Sams Tech LLC•8/14/24, 12:10 AM

or is there some other way like custom model maybe?

Uncle Sams Tech LLC•8/14/24, 12:21 AM

I am doing a email router -> worker -> image extraction, so the AI is able to read the attachment from email, but its not a usable data

IIsaac McFadyen Neither. It's base as far as I know.

mr.niko.la•8/14/24, 2:50 AM

Too bad I need to self hosting whisperX

AleksandarOP•8/14/24, 10:47 AM

Is there any place to watch out for when beta models go out of beta

orangebrain•8/14/24, 1:35 PM

Hi everyone,
I'm currently working with the Cloudflare-hosted BAAI model (@cf/baai/bge-base-en-v1.5) for generating vector embeddings in a Hono application. However, I'm encountering an issue where the AI service throws an 3001: Unknown internal error.
Here's a brief overview of what I'm doing:
I send a request to the AI model with a text array containing a single string.
const { data } = await c.env.AI.run('@cf/baai/bge-base-en-v1.5', { text: [text] });
The error occurs at this point, and the response from the AI service contains the 3001 with the message "Unknown internal error."
I've double-checked the input data, and everything seems to be in order. However, I'm still unsure what's causing this issue.
Has anyone else encountered this error, or does anyone have insights into what might be causing it? Any guidance on how to resolve this would be greatly appreciated.

PS: This error seems to occur infrequently and intermittently. Most of requests to the embedding model successfully complete.

scotto•8/14/24, 5:40 PM

I believe pricing shoud be reviewed as it's order of magnitude more expensive than any compettipl

Owl Reddy•8/15/24, 1:37 AM

Is there a way to measure how many input and output tokens(or neurons) are being used when using workers AI? For context, other models return token usage details along with response via API but workers AI just seems to return just the response.

vin•8/15/24, 12:01 PM

Is there any plan for supporting FLUX.1 text-to-image on Workers AI?

Xinyue Zhang•8/16/24, 10:21 AM

Is there any plan to launch the QWen2 model

AAleksandar Is there any place to watch out for when beta models go out of beta

AleksandarOP•8/16/24, 4:24 PM

Beyondo•8/17/24, 5:12 PM

I believe the text-generation models in Cloudflare AI should at least return the number of tokens counted in the prompt (input) and also generated (output), so we can track the end user's usage and properly rate-limit for future requests like other AI providers.

captain•8/17/24, 9:27 PM

will flux.1 be added to cloudflare ai libraries...?

rob•8/18/24, 1:04 PM

I don't think so , I thought flux only
Partnered with few providers , possibly only certain companies can monetize it ? . maybe the "open" flux dev model but even that the license says no commercialize so idk

ToonyGen•8/18/24, 7:39 PM

where can i find information about how much does it cost to generate one image with stable diffusion?

ToonyGen•8/18/24, 7:40 PM

It only says about neuron which seems to only related to LLM space.

TToonyGen It only says about neuron which seems to only related to LLM space.

Isaac McFadyen•8/18/24, 7:42 PM

All models on Workers AI use neurons, it's not an LLM thing it's just a standardized unit to measure the computational cost of the model.

Hello. Was this issue with `InferenceUpstreamError: undefined: undefined` fixed? I'm getting the sam

Similar Threads

Similar Threads

Similar Threads