To bind Workers AI to your worker, add the following to the end of your wrangler.toml file: [ai] bi

To bind Workers AI to your worker, add the following to the end of your wrangler.toml file:

[ai]
binding = "AI" #available in your worker via env.AI
source: https://blog.cloudflare.com/workers-ai/

The Cloudflare Blog

Workers AI: serverless GPU-powered inference on Cloudflare’s global...

We are excited to launch Workers AI - an AI inference as a service platform, empowering developers to run AI models with just a few lines of code, all powered by our global network of GPUs

MMangum To bind Workers AI to your worker, add the following to the end of your wrangler...

rob•3/25/24, 3:42 AM

yep, the extra step required need to have it bound in your toml file

MangumOP•3/25/24, 5:30 AM

Hi there, i'm trying to follow this tutorial https://developers.cloudflare.com/workers-ai/tutorials/build-a-retrieval-augmented-generation-ai/
When I enter the command "npx wrangler vectorize create vector-index --dimensions=768 --metric=cosine"
into my terminal, I get: vectorize.not_entitled [code: 1005]
Is this because I'm using a free account and Vectorize is not provisioned?

Cloudflare Docs

Build a Retrieval Augmented Generation (RAG) AI · Cloudflare Worker...

This guide will instruct you through setting up and deploying your first application with Cloudflare AI. You will build a fully-featured AI-powered …

MMangum To bind Workers AI to your worker, add the following to the end of your wrangler...

fakebrin•3/25/24, 7:07 AM

it is there in the wrangler.toml file, if the clone worked correctly

zehawk•3/25/24, 8:51 AM

Unable to run even the simplest code on wrangler dev local (eg https://developers.cloudflare.com/workers-ai/configuration/workers-ai-sdk/). Getting

[wrangler:err] InferenceUpstreamError: {"errors":[{"message":"Server Error","code":1000}],"success":false,"result":{},"messages":[]}

[wrangler:err] InferenceUpstreamError: {"errors":[{"message":"Server Error","code":1000}],"success":false,"result":{},"messages":[]}

AI workers were working fine earlier, and starting giving an error now. Hence testing with boiler plate code.

Raylight•3/25/24, 5:49 PM

Some feedback regarding @cf/runwayml/stable-diffusion-v1-5-inpainting@cf/runwayml/stable-diffusion-v1-5-inpainting (and @cf/runwayml/stable-diffusion-v1-5-img2img@cf/runwayml/stable-diffusion-v1-5-img2img):
The models seem to produce an output with a noticeable shift in color balance. Repeated inpainting will quickly shift the image towards purple. See the attached images below. The first image is the result of doing 10 passes (i.e. using the result as source) of inpainting with an empty (black) mask. The second is the same process but with an added "hack" (i.e. more of a band-aid than a solution) that does histogram matching after each pass. The third image is a typical result from a single pass of inpainting merged with unmasked pixels from the source. (Shown with mask overlayed on source.)

This seems to be a well-known problem that (possibly) can be traced back to the VAE, and (possibly) be solved by using an updated version of the VAE. Whether that's the case here is another question.

inpainting_stp20_g7.5_str1_10_iter_w_histo_matching.png

Raylight•3/25/24, 5:49 PM

(The black to the left is simply because the "safety checker" believes the image is harmful content without it.)

Mmichelle this is so interesting, is it a model problem though or specific to workers ai?

Raylight•3/25/24, 10:38 PM

I believe it's a problem with the model, so maybe it belongs to the catch all "cloudflare-ai". The result should be the same if using the API. Although using workers ai + client side javascript will involve more head-scratching and second-guessing your code.

I found this https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/410 which seems to match my own testing.

playlogo•3/25/24, 10:43 PM

Hey, is there any way we can use mistral-7b-instruct-v0.2mistral-7b-instruct-v0.2 via the workers rest api ?

playlogo•3/25/24, 10:44 PM

Also I'm sometimes getting the same output for slightly different promps. Is this some sort of caching that i can maybe disable ?

rob•3/25/24, 11:00 PM

are you using the ai gateway ?

Dan (Openside)•3/26/24, 1:47 AM

Im using model @hf/thebloke/deepseek-coder-6.7b-instruct-awq@hf/thebloke/deepseek-coder-6.7b-instruct-awq It appears there is a character limit on the response (around 800 chars). Is this a known limit? any way to get the full response?

playlogo•3/26/24, 11:57 AM

According to their API Docs you can set the max_tokensmax_tokens property in the request. But don't see it mentioned anywhere else, so it might not work...

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

YeungKC•3/26/24, 12:45 PM

Hey guys, I would like to integrate an AI interface into worker-rs. Does anyone know where I can find the source code for @cloudflare/ai?

rob•3/26/24, 1:33 PM

https://twitter.com/StabilityAI/status/1772345514023116828

Stability AI (@StabilityAI) on X

Introducing Stable Code Instruct 3B, our new instruction tuned LLM based on Stable Code 3B. With natural language prompting, this model can handle a variety of tasks such as code generation, math and other software engineering related outputs.

This model’s performance rivals…

Twitter

•

3/25/24, 7:31 PM

YYeungKC Hey guys, I would like to integrate an AI interface into worker-rs. Does anyone ...

John Spurlock•3/26/24, 9:51 PM

it's not public afaik, you can always: https://www.npmjs.com/package/@cloudflare/ai?activeTab=code

npm

@cloudflare/ai

Cloudflare's Workers AI SDK. Latest version: 1.1.0, last published: 19 days ago. Start using @cloudflare/ai in your project by running npm i @cloudflare/ainpm i @cloudflare/ai. There are 3 other projects in the npm registry using @cloudflare/ai.

John Spurlock•3/26/24, 9:52 PM

or install locally etc

JJohn Spurlock it's not public afaik, you can always: https://www.npmjs.com/package/@cloudflare...

YeungKC•3/27/24, 1:45 AM

Is it not ready for open source yet, or is there no plan to open source it?

James•3/27/24, 1:53 AM

I believe the goal is to make AI a more native binding in workers and remove the need for the package entirely

YeungKC•3/27/24, 1:58 AM

Thx, I'm really looking forward to it.

rob•3/27/24, 1:58 AM

yah more for calling inside of workers

Nooc•3/28/24, 3:55 AM

I'm calling Workers AI in a nextjs project, the project used the edge runtime in every page, because I want to deploy it on Cloudflare Pages. But the Workers AI calling continuously occur the error '⨯ Error: A Node.js API is used (DecompressionStream) which is not supported in the Edge Runtime.'. Has anyone had the same problem?

Nooc•3/28/24, 3:55 AM

Nooc•3/28/24, 4:02 AM

I am using @cloudflare/next-on-pages@cloudflare/next-on-pages to adapte between nextjs and @cloudflare/ai@cloudflare/ai , so maybe it's because there is no nodejs_compatnodejs_compat compatibility-flag, but I don't how to add that.

Mistic92•3/28/24, 3:15 PM

What version and what parameters are used on Whisper? I'm testing asr and trying different encoders, sample rates and bitrates recording on Android. When I'm using Cloudflare Workers Whisper model it looks like very dumb. I tried Replicate hosted whisper and I got perfect transcription of my audio.

miko1383•3/28/24, 8:37 PM

Hi - not sure if I should post here or in cloudflare-ai. I'm trying the @cf/runwayml/stable-diffusion-v1-5-inpainting model, and it's automatically resizing my images to 512x512. I need to be able to control the size. I don't see any parameters for the size... where can we vote for such features?

LilHuyyyy•3/29/24, 1:58 AM

hi guys i can not deploy my worker ai application it keeps showing binding AI of type ai failed to generate: internal error; please try again later or contact
support [code: 10021]. I can still deploy my worker yesterday but this morning, this error occurs. Can any one help me. Thanks a lot

LilHuyyyy•3/29/24, 2:02 AM

oh thanks so i should sit down drink a coffee and wait :))))

LilHuyyyy•3/29/24, 2:06 AM

anyway i just have another problem :< yesterday when i testing the worker ai inference i got this error. InferenceUpstreamError: ERROR 3001: Unknown internal error. for some models that need to read buffer from file for example. @cf/openai/whisper or some other images classification and image to text model (eg: @cf/microsoft/resnet-50, @cf/unum/uform-gen2-qwen-500m). The weirdo thing is that when i run wrangler dev in my local machine it just works fine but when i deploy my worker the error showed up. Any one have same problem like me and how can i fix it ?

LilHuyyyy•3/29/24, 3:05 AM

Thank a lot sir

LilHuyyyy•3/29/24, 3:05 AM

Ok i can deploy my worker ai now :v

user•3/29/24, 4:12 AM

Is there a way to limit the tokens output like how you can in openAI?

Uuser Is there a way to limit the tokens output like how you can in openAI?

user•3/29/24, 8:51 AM

if not, this would imo be a very important thing to have because uh, https://user.you-sk.id/IPzlj5FWbl54

uwu

owo

user•3/29/24, 8:52 AM

this happens way too frequently...

user•3/29/24, 8:53 AM

3000 tokens from just one AI message because it randomly bugged out isnt exactly the greatest thing

Isaac McFadyen•3/29/24, 3:02 PM

I'm going to take a wild guess and say this is qwen.

Isaac McFadyen•3/29/24, 3:02 PM

I've had similar issues with Qwen running locally (i.e. not just Workers AI).

Isaac McFadyen•3/29/24, 3:03 PM

Even if you prompt in English and tell it to reply in English it seems to randomly switch to Chinese and back. I eventually just gave up and switched models.

Isaac McFadyen•3/29/24, 3:03 PM

Not sure if the repetition is related but that's likely why it randomly replies in Chinese after the actual response.

Mmichelle are you using a chat template at all?

Isaac McFadyen•3/29/24, 3:04 PM

I was, yeah. ChatML.

To bind Workers AI to your worker, add the following to the end of your wrangler.toml file: [ai] bi

Similar Threads

Similar Threads

Similar Threads