it's not public afaik, you can always: https://www.npmjs.com/package/@cloudflare/ai?activeTab=code

npm

Cloudflare's Workers AI SDK. Latest version: 1.1.0, last published: 19 days ago. Start using @cloudflare/ai in your project by running

npm i @cloudflare/ai

npm i @cloudflare/ai

. There are 3 other projects in the npm registry using @cloudflare/ai.

John SpurlockOP•3/26/24, 9:52 PM

or install locally etc

JJohn Spurlock it's not public afaik, you can always: https://www.npmjs.com/package/@cloudflare...

YeungKC•3/27/24, 1:45 AM

Is it not ready for open source yet, or is there no plan to open source it?

James•3/27/24, 1:53 AM

I believe the goal is to make AI a more native binding in workers and remove the need for the package entirely

YeungKC•3/27/24, 1:58 AM

Thx, I'm really looking forward to it.

rob•3/27/24, 1:58 AM

yah more for calling inside of workers

Nooc•3/28/24, 3:55 AM

I'm calling Workers AI in a nextjs project, the project used the edge runtime in every page, because I want to deploy it on Cloudflare Pages. But the Workers AI calling continuously occur the error '⨯ Error: A Node.js API is used (DecompressionStream) which is not supported in the Edge Runtime.'. Has anyone had the same problem?

Nooc•3/28/24, 3:55 AM

Nooc•3/28/24, 4:02 AM

I am using

@cloudflare/next-on-pages

@cloudflare/next-on-pages

to adapte between nextjs and

@cloudflare/ai

@cloudflare/ai

, so maybe it's because there is no

nodejs_compat

nodejs_compat

compatibility-flag, but I don't how to add that.

Mistic92•3/28/24, 3:15 PM

What version and what parameters are used on Whisper? I'm testing asr and trying different encoders, sample rates and bitrates recording on Android. When I'm using Cloudflare Workers Whisper model it looks like very dumb. I tried Replicate hosted whisper and I got perfect transcription of my audio.

miko1383•3/28/24, 8:37 PM

Hi - not sure if I should post here or in cloudflare-ai. I'm trying the @cf/runwayml/stable-diffusion-v1-5-inpainting model, and it's automatically resizing my images to 512x512. I need to be able to control the size. I don't see any parameters for the size... where can we vote for such features?

LilHuyyyy•3/29/24, 1:58 AM

hi guys i can not deploy my worker ai application it keeps showing binding AI of type ai failed to generate: internal error; please try again later or contact
support [code: 10021]. I can still deploy my worker yesterday but this morning, this error occurs. Can any one help me. Thanks a lot

LilHuyyyy•3/29/24, 2:02 AM

oh thanks so i should sit down drink a coffee and wait :))))

LilHuyyyy•3/29/24, 2:06 AM

anyway i just have another problem :< yesterday when i testing the worker ai inference i got this error. InferenceUpstreamError: ERROR 3001: Unknown internal error. for some models that need to read buffer from file for example. @cf/openai/whisper or some other images classification and image to text model (eg: @cf/microsoft/resnet-50, @cf/unum/uform-gen2-qwen-500m). The weirdo thing is that when i run wrangler dev in my local machine it just works fine but when i deploy my worker the error showed up. Any one have same problem like me and how can i fix it ?

LilHuyyyy•3/29/24, 3:05 AM

Thank a lot sir

LilHuyyyy•3/29/24, 3:05 AM

Ok i can deploy my worker ai now :v

user•3/29/24, 4:12 AM

Is there a way to limit the tokens output like how you can in openAI?

Uuser Is there a way to limit the tokens output like how you can in openAI?

user•3/29/24, 8:51 AM

if not, this would imo be a very important thing to have because uh, https://user.you-sk.id/IPzlj5FWbl54

uwu

owo

user•3/29/24, 8:52 AM

this happens way too frequently...

user•3/29/24, 8:53 AM

3000 tokens from just one AI message because it randomly bugged out isnt exactly the greatest thing

Isaac McFadyen•3/29/24, 3:02 PM

I'm going to take a wild guess and say this is qwen.

Isaac McFadyen•3/29/24, 3:02 PM

I've had similar issues with Qwen running locally (i.e. not just Workers AI).

Isaac McFadyen•3/29/24, 3:03 PM

Even if you prompt in English and tell it to reply in English it seems to randomly switch to Chinese and back. I eventually just gave up and switched models.

Isaac McFadyen•3/29/24, 3:03 PM

Not sure if the repetition is related but that's likely why it randomly replies in Chinese after the actual response.

Mmichelle are you using a chat template at all?

Isaac McFadyen•3/29/24, 3:04 PM

I was, yeah. ChatML.

Isaac McFadyen•3/29/24, 3:05 PM

It's possible it doesn't expect ChatML? I can't find many details online actually...

Isaac McFadyen•3/29/24, 3:05 PM

Looks like it wants ChatML so that's probably not the issue...

Isaac McFadyen•3/29/24, 3:05 PM

And my ChatML prompt worked in other models so I don't think it was an issue with how I was formatting it.

Isaac McFadyen•3/29/24, 3:06 PM

Unless they have some Qwen-specific ChatML variation?

Mmichelle which of the 4 qwen models are you trying?

Isaac McFadyen•3/29/24, 3:06 PM

I was trying 14b. Keep in mind this was locally and not Workers AI so perhaps Workers AI is formatting the chat prompt differently?

Mmichelle which model are you using?

user•3/29/24, 6:44 PM

Sorry, just woke up

user•3/29/24, 6:44 PM

The bug does happen on beta models so it's expected to be a bit buggy but still

Mmichelle we do allow you to set max_tokens

user•3/29/24, 6:44 PM

ah ty, couldnt find it in docs

user•3/29/24, 6:45 PM

anyways, the bug happens with all of the qwen models and openchat models

user•3/29/24, 6:45 PM

@cf/openchat/openchat-3.5-0106

@cf/openchat/openchat-3.5-0106

@cf/qwen/qwen1.5-0.5b-chat

@cf/qwen/qwen1.5-0.5b-chat

@cf/qwen/qwen1.5-1.8b-chat

@cf/qwen/qwen1.5-1.8b-chat

@cf/qwen/qwen1.5-14b-chat-awq

@cf/qwen/qwen1.5-14b-chat-awq

@cf/qwen/qwen1.5-7b-chat-awq

@cf/qwen/qwen1.5-7b-chat-awq

user•3/29/24, 6:46 PM

sometimes the models will either just spit out complete nonsense or a bunch of chinese characters

user•3/29/24, 6:46 PM

like i got several full on book-report essays when i just asked openchat "How are you today?" when testing

user•3/29/24, 6:47 PM

if needed I can provide my account ID

user•3/29/24, 6:48 PM

wdym chat template

user•3/29/24, 6:48 PM

Like a system prompt?

user•3/29/24, 6:49 PM

or like premade code

it's not public afaik, you can always: https://www.npmjs.com/package/@cloudflare/ai?activeTab=code

Similar Threads