I can't reproduce "undefined: undefined" but haven't tried running locally. That said, make sure tha

I can't reproduce "undefined: undefined" but haven't tried running locally. That said, make sure that the image is smaller than 2.5-3 MB. (

JSON.stringify(inputs).length

JSON.stringify(inputs).length

JSON.stringify(inputs).length

JSON.stringify(inputs).length needs to be less than 9.5 MB, to be precise.)

JCBahn•8/11/24, 9:32 AM

When I make a query in a non-English language, the response comes out broken. Has anyone else experienced this? This doesn't happen when I use llama3.1 with ollama. Please improve support for non-English queries. Especially for Llama 3.1.

RRaylight I can't reproduce "undefined: undefined" but haven't tried running locally. That...

Aleksandar•8/11/24, 12:12 PM

that was happening locally. ill check the image size and get back to you

shellsort•8/11/24, 12:34 PM

Heyhey!
I was wondering if/when the documentation for this endpoint will be updated:
https://developers.cloudflare.com/api/operations/workers-ai-post-run-model

The responses are not correct.
Successful responses return more than result, they also return a success, errors and messages code where I can only assume that the errors and messages content is strings (thou it might very well be status codes of some sort) and the result contains a model-specific response.

For example for the M2M100-1.2B translation model it is translated_text (which is somewhat documented in the api schema definition of the worker: https://developers.cloudflare.com/workers-ai/models/m2m100-1.2b/)
For unsuccessful responses thou, the result is really weird. On wrong input I got this for example:
{
"result": {
"translated_text": "ERROR: invalid source_lang (English)"
},
"success": true,
"errors": [],
"messages": []
}

which is, very confusing and odd to use. It says success, even thou there's an error. The errors section is empty. The messages section is empty and the message itsself is in the model result. :/

It is also unclear which input values for this model are allowed.
I found this (https://github.com/cloudflare/cloudflare-docs/blob/production/content/workers-ai/models/m2m100-1.2b.md), but it doesnt seem to list all languages.
I also found this (https://github.com/ikergarcia1996/Easy-Translate/blob/main/supported_languages.md#supported-languages-m2m100), but the cloudflare version only seems to work with flatcase conventions and the "ld" abbreviations. It's also unclear if these abbreviations follow ISO 639, some other standard or if they are arbitrary.

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

GitHub

Easy-Translate/supported_languages.md at main · ikergarcia1996/Easy...

Easy-Translate is a script for translating large text files with a SINGLE COMMAND. Easy-Translate is designed to be as easy as possible for beginners and as seamlesscustomizable and as possible fo...

Treetop•8/11/24, 4:27 PM

Does anyone know the pricing for whisper-tiny-en?

Devon•8/11/24, 10:03 PM

is there any support for dialects in the translation AI model? Standard locale codes don't seem to be valid. Things like Brazilian Portuguese or the Hong Kong or Tawainese variants of Chinese.

Sshellsort Heyhey! I was wondering if/when the documentation for this endpoint will be upda...

shellsort•8/12/24, 12:53 AM

There's a list of supported languages in here somewhere

Devon•8/12/24, 12:28 PM

Oh I hadn't seen your comment previously. Yeah I couldn't find a list. It does seem like, contrary to most of the docs, that it only takes lowercase two char codes. Not wanting to look a gift horse in the mouth, but have the ability to translate Portugal Portugese and Brazil Portugese separately would be very nice (along with the variants of zh, etc... )

shellsort•8/12/24, 2:21 PM

Full names work for me.

shellsort•8/12/24, 2:21 PM

lowercase

Devon•8/12/24, 2:57 PM

ah, I've been using the two letter codes successfully. but no "pt_BR" or "zh_TW" etc...

kian•8/12/24, 3:26 PM

https://developers.cloudflare.com/turnstile/reference/supported-languages/#supported-languages

kian•8/12/24, 3:26 PM

zh-twzh-tw and pt-brpt-br

Devon•8/12/24, 3:30 PM

Doesn't work for the Translation AI: "ERROR: invalid target_lang (zh-tw)"

kian•8/12/24, 3:46 PM

Oops, I thought I was in the Turnstile channel.

kian•8/12/24, 3:46 PM

In that case, all of the languages supported by the model should be on https://huggingface.co/facebook/m2m100_1.2B#languages-covered which indeed doesn't cover dialects.

Devon•8/12/24, 3:47 PM

ah, thank you very much!

Bbradyjoslin Recommend adding the function calling pill visualization to llama-3.1-8b-instruc...

Craig Dennis•8/12/24, 9:35 PM

We're still working out the kinks before we add the pill, but thank you for the specific and thoughtful callout!

anirudh•8/13/24, 10:37 AM

Which version of Whisper is Workers AI using? V2 large or V3 large?

Aanirudh Which version of Whisper is Workers AI using? V2 large or V3 large?

Isaac McFadyen•8/13/24, 12:58 PM

Neither. It's base as far as I know.

Isaac McFadyen•8/13/24, 12:58 PM

Reference: https://canary.discord.com/channels/595317990191398933/1105477009964027914/1225461479730446427

Vero•8/13/24, 6:08 PM

Hey everyone, in case you haven’t seen it, we recently published a video on How to use Cloudflare AI models and inference in Python with Jupyter Notebooks

https://youtu.be/CHfKeFakGAI

YouTubeCloudflare Developers

How to use Cloudflare AI models and inference in Python with Jupyte...

Cloudflare Workers AI provides a ton of AI models and inference capabilities. In this video, we will explore how to make use of Cloudflare’s AI model catalog using a Python Jupyter Notebook. And remember, inference is free for all models in beta!

Create an account on Cloudflare today for free: https://dash.cloudflare.com/sign-up

Useful links m...

Uncle Sams Tech LLC•8/14/24, 12:09 AM

What is the best AI model for OCR kind of capability. Currently "image to text" only gives you summary, not the exact extracted text.

Uncle Sams Tech LLC•8/14/24, 12:10 AM

or is there some other way like custom model maybe?

Uncle Sams Tech LLC•8/14/24, 12:21 AM

I am doing a email router -> worker -> image extraction, so the AI is able to read the attachment from email, but its not a usable data

IIsaac McFadyen Neither. It's base as far as I know.

mr.niko.la•8/14/24, 2:50 AM

Too bad I need to self hosting whisperX

Aleksandar•8/14/24, 10:47 AM

Is there any place to watch out for when beta models go out of beta

orangebrain•8/14/24, 1:35 PM

Hi everyone,
I'm currently working with the Cloudflare-hosted BAAI model (@cf/baai/bge-base-en-v1.5) for generating vector embeddings in a Hono application. However, I'm encountering an issue where the AI service throws an 3001: Unknown internal error.
Here's a brief overview of what I'm doing:
I send a request to the AI model with a text array containing a single string.
const { data } = await c.env.AI.run('@cf/baai/bge-base-en-v1.5', { text: [text] });
The error occurs at this point, and the response from the AI service contains the 3001 with the message "Unknown internal error."
I've double-checked the input data, and everything seems to be in order. However, I'm still unsure what's causing this issue.
Has anyone else encountered this error, or does anyone have insights into what might be causing it? Any guidance on how to resolve this would be greatly appreciated.

PS: This error seems to occur infrequently and intermittently. Most of requests to the embedding model successfully complete.

scotto•8/14/24, 5:40 PM

I believe pricing shoud be reviewed as it's order of magnitude more expensive than any compettipl

Owl Reddy•8/15/24, 1:37 AM

Is there a way to measure how many input and output tokens(or neurons) are being used when using workers AI? For context, other models return token usage details along with response via API but workers AI just seems to return just the response.

vin•8/15/24, 12:01 PM

Is there any plan for supporting FLUX.1 text-to-image on Workers AI?

Xinyue Zhang•8/16/24, 10:21 AM

Is there any plan to launch the QWen2 model

AAleksandar Is there any place to watch out for when beta models go out of beta

Aleksandar•8/16/24, 4:24 PM

Beyondo•8/17/24, 5:12 PM

I believe the text-generation models in Cloudflare AI should at least return the number of tokens counted in the prompt (input) and also generated (output), so we can track the end user's usage and properly rate-limit for future requests like other AI providers.

captain•8/17/24, 9:27 PM

will flux.1 be added to cloudflare ai libraries...?

rob•8/18/24, 1:04 PM

I don't think so , I thought flux only
Partnered with few providers , possibly only certain companies can monetize it ? . maybe the "open" flux dev model but even that the license says no commercialize so idk

ToonyGen•8/18/24, 7:39 PM

where can i find information about how much does it cost to generate one image with stable diffusion?

ToonyGen•8/18/24, 7:40 PM

It only says about neuron which seems to only related to LLM space.

TToonyGen It only says about neuron which seems to only related to LLM space.

Isaac McFadyen•8/18/24, 7:42 PM

All models on Workers AI use neurons, it's not an LLM thing it's just a standardized unit to measure the computational cost of the model.

Isaac McFadyen•8/18/24, 7:42 PM

But all the image models are currently free, any models marked

Beta

Beta

on this page are free until they come out of beta. https://developers.cloudflare.com/workers-ai/models/#text-to-image

ToonyGen•8/18/24, 7:44 PM

Hi, Issac. Is there a way to get rough estimation of neuron usage for a single stable diffusion inference?

ToonyGen•8/18/24, 7:45 PM

For a100, I think it takes 1.x second for sd v1.5 512 x 512 and for sdxl lightening 1024 x 1024 around 3 to 4s.

ToonyGen•8/18/24, 7:49 PM

Please leave comments if you have any information about the pricing model for stable diffusion later.

TToonyGen Hi, Issac. Is there a way to get rough estimation of neuron usage for a single s...

Isaac McFadyen•8/18/24, 8:09 PM

I don't believe there is currently, no. There is a neuron pricing calculator but I'm not sure whether the text-to-image models have been added yet.

Isaac McFadyen•8/18/24, 8:09 PM

https://ai.cloudflare.com/#pricing-calculator

Rrob I don't think so , I thought flux only Partnered with few providers , possibly o...

RaylightOP•8/19/24, 7:51 AM

https://huggingface.co/black-forest-labs/FLUX.1-schnell can be used for commercial purposes. flux.1 dev would require some kind of arrangement, though.

TToonyGen Hi, Issac. Is there a way to get rough estimation of neuron usage for a single s...

vin•8/19/24, 8:47 AM

Maybe you can refer to other online text-to-image generation provider price, for commercial concerns, I think cloudflare won't make it much more expensive or cheaper than other providers. As far as i know, sdxl 1024*1024 20 steps cost about 0.2~0.3 cents per image.

rchaves•8/19/24, 1:03 PM

hey folks, where can I request the addition of new models? The

@hf/thebloke/llamaguard-7b-awq

@hf/thebloke/llamaguard-7b-awq

there is a bit outdated, still uses llama 2 and it's not very smart, what about upgrading it to https://huggingface.co/meta-llama/Llama-Guard-3-8B which uses llama 3.1? And while at it perhaps also add https://huggingface.co/meta-llama/Prompt-Guard-86M

cloudflare is perfect for running those guardrails models which should run as fast as possible

I can't reproduce "undefined: undefined" but haven't tried running locally. That said, make sure tha

Similar Threads