Hey guys, I would like to integrate an AI interface into worker-rs. Does anyone know where I can fin
Hey guys, I would like to integrate an AI interface into worker-rs. Does anyone know where I can find the source code for @cloudflare/ai?
R
Introducing Stable Code Instruct 3B, our new instruction tuned LLM based on Stable Code 3B. With natural language prompting, this model can handle a variety of tasks such as code generation, math and other software engineering related outputs.
This model’s performance rivals…
This model’s performance rivals…
3/25/24, 7:31 PM
Jnpm
Cloudflare's Workers AI SDK. Latest version: 1.1.0, last published: 19 days ago. Start using @cloudflare/ai in your project by running
npm i @cloudflare/ai. There are 3 other projects in the npm registry using @cloudflare/ai.Jor install locally etc
YIs it not ready for open source yet, or is there no plan to open source it?
JI believe the goal is to make AI a more native binding in workers and remove the need for the package entirely
YThx, I'm really looking forward to it.
Ryah more for calling inside of workers
NI'm calling Workers AI in a nextjs project, the project used the edge runtime in every page, because I want to deploy it on Cloudflare Pages. But the Workers AI calling continuously occur the error '⨯ Error: A Node.js API is used (DecompressionStream) which is not supported in the Edge Runtime.'. Has anyone had the same problem?
N
NI am using
@cloudflare/next-on-pages to adapte between nextjs and @cloudflare/ai , so maybe it's because there is no nodejs_compat compatibility-flag, but I don't how to add that.
MWhat version and what parameters are used on Whisper? I'm testing asr and trying different encoders, sample rates and bitrates recording on Android. When I'm using Cloudflare Workers Whisper model it looks like very dumb. I tried Replicate hosted whisper and I got perfect transcription of my audio.
MHi - not sure if I should post here or in cloudflare-ai. I'm trying the @cf/runwayml/stable-diffusion-v1-5-inpainting model, and it's automatically resizing my images to 512x512. I need to be able to control the size. I don't see any parameters for the size... where can we vote for such features?
Lhi guys i can not deploy my worker ai application it keeps showing binding AI of type ai failed to generate: internal error; please try again later or contact
support [code: 10021]. I can still deploy my worker yesterday but this morning, this error occurs. Can any one help me. Thanks a lot
support [code: 10021]. I can still deploy my worker yesterday but this morning, this error occurs. Can any one help me. Thanks a lot
Loh thanks so i should sit down drink a coffee and wait :))))
Lanyway i just have another problem :< yesterday when i testing the worker ai inference i got this error. InferenceUpstreamError: ERROR 3001: Unknown internal error. for some models that need to read buffer from file for example. @cf/openai/whisper or some other images classification and image to text model (eg: @cf/microsoft/resnet-50, @cf/unum/uform-gen2-qwen-500m). The weirdo thing is that when i run wrangler dev in my local machine it just works fine but when i deploy my worker the error showed up. Any one have same problem like me and how can i fix it ?
LThank a lot sir
LOk i can deploy my worker ai now :v
UIs there a way to limit the tokens output like how you can in openAI?
Uif not, this would imo be a very important thing to have because uh, https://user.you-sk.id/IPzlj5FWbl54
Uthis happens way too frequently...
U3000 tokens from just one AI message because it randomly bugged out isnt exactly the greatest thing
II'm going to take a wild guess and say this is qwen.
II've had similar issues with Qwen running locally (i.e. not just Workers AI).
IEven if you prompt in English and tell it to reply in English it seems to randomly switch to Chinese and back. I eventually just gave up and switched models.
INot sure if the repetition is related but that's likely why it randomly replies in Chinese after the actual response.
II was, yeah. ChatML.
IIt's possible it doesn't expect ChatML? I can't find many details online actually...
ILooks like it wants ChatML so that's probably not the issue...
IAnd my ChatML prompt worked in other models so I don't think it was an issue with how I was formatting it.
IUnless they have some Qwen-specific ChatML variation?
II was trying 14b. Keep in mind this was locally and not Workers AI so perhaps Workers AI is formatting the chat prompt differently?
USorry, just woke up
UThe bug does happen on beta models so it's expected to be a bit buggy but still
Uah ty, couldnt find it in docs
Uanyways, the bug happens with all of the qwen models and openchat models
U@cf/openchat/openchat-3.5-0106 @cf/qwen/qwen1.5-0.5b-chat @cf/qwen/qwen1.5-1.8b-chat @cf/qwen/qwen1.5-14b-chat-awq @cf/qwen/qwen1.5-7b-chat-awqUsometimes the models will either just spit out complete nonsense or a bunch of chinese characters
Ulike i got several full on book-report essays when i just asked openchat "How are you today?" when testing
Uif needed I can provide my account ID
Uwdym chat template
ULike a system prompt?