Hi guys, how can use this worker ai ? Have someone the Java script for it ?

NNicolas92 Hi guys, how can use this worker ai ? Have someone the Java script for it ?

You can follow any of these guides. Use CLI or Dashboard if you want to deploy a Worker, use the API one for if you want to access it externally without a Worker: https://developers.cloudflare.com/workers-ai/get-started/

js•5/31/24, 5:34 AM

Will llama 3 loras be supported soon? https://developers.cloudflare.com/workers-ai/fine-tunes/loras/

Cloudflare Docs

Using LoRA adapters · Cloudflare Workers AI docs

Upload and use LoRA adapters to get fine-tuned inference on Workers AI.

Puliczek•6/1/24, 2:07 PM

Hi, are there any startups that are using "Workers for Platforms"?

PPuliczek Hi, are there any startups that are using "Workers for Platforms"?

Isaac McFadyen•6/1/24, 5:04 PM

This would probably be a better question for #workers-for-platforms. This channel is for Workers AI.

Mmichelle sorry - was at Connect! We are working on making it configurable per request, TB...

Raylight•6/1/24, 6:58 PM

Hmm.. quite a lot going on with that model right now. Was going to give feedback on the compression level as the immediate issue, but now the model is broken for deployed workers that are using SDK 1.0.53 or earlier. Which is kinda.. more immediate. Feels like the model is actively being worked on right in the open, so I'm thinking.. should I just hold off a bit, or try to play catch up with feedback?

Mmichelle are you using native binding? or the npm install?

Raylight•6/1/24, 7:05 PM

I'm using all of them. 1.0.53, 1.1.0, native binding, api.

Telion•6/1/24, 7:28 PM

Hi, I was wondering whether there is any way to improve the results of the LLAMA 3 here. I've been trying

@/hf/meta-llama/meta-llama-3-8b-instruct

@/hf/meta-llama/meta-llama-3-8b-instruct

and

@cf/meta/llama-3-8b-instruct

@cf/meta/llama-3-8b-instruct

and both provide me with almost completely random responses when trying to use a system message along the prompt.

I am trying to classify user prompts by a list of intents. An example of prompt I am currently struggling with is "What would I need to do to learn js?". While example of intents is:
'1 Track progress and view improvements',
'2 Generate a learning plan or prepare learning materials list',
'3 Need an explanation',
'4 Seek motivation',

Here's how I prompt the model with the data:

        const intent = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
          messages: [
            {
              "role": "system",
              "content": `I will help you figure out intent of the message based on available options. Available intents:\n${INTENTS.join('\n')}\n\nI will choose an intent from the list that best matches the user's prompt and return only its number. There will be no other text in my response other then the found number. I will use the following format: [intent number from the list].\n\nExample of response when found:\n3\n\nExample of when intent cannot be determined:\nNo intent`
            },
            {
              "role": "user",
              "content": prompt
            }
          ]
        });

        const intent = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
          messages: [
            {
              "role": "system",
              "content": `I will help you figure out intent of the message based on available options. Available intents:\n${INTENTS.join('\n')}\n\nI will choose an intent from the list that best matches the user's prompt and return only its number. There will be no other text in my response other then the found number. I will use the following format: [intent number from the list].\n\nExample of response when found:\n3\n\nExample of when intent cannot be determined:\nNo intent`
            },
            {
              "role": "user",
              "content": prompt
            }
          ]
        });

The responses to this are pretty random. I get:
1
4
3
2
3

When I moved the message from the system prompt into user prompt it started working better but the results are still really unreliable. I wanted to check with experienced people here whether my prompting is too unprecise or there is something wrong with how I understand the CF's AI.

The real issue here is that when I just use the same model on HF or Meta or even drop it into gemini or gpt-4o/3.5t all of them return perfect results, straight "2"s but not on CF

Telion•6/1/24, 7:30 PM

I've used the same prompt on all of those, the results differ majorly with what I experience on CF's

Telion•6/1/24, 7:32 PM

Although when I put the same message into user's prompt and get rid of the system prompt it starts working a lot better. Yet other prompts still fail randomly, while seem to be percieved correctly on other platforms.

system-prompt-seems-to-not-enforce-behavior-v0-z6f8jse1fz3d1.webp

system-prompt-seems-to-not-enforce-behavior-v0-g6geobw7fz3d1.webp

TTelion Hi, I was wondering whether there is any way to improve the results of the LLAMA...

Unsmart•6/2/24, 2:41 AM

As explained in the Workers AI docs for each model you want to use scoped prompts with hints (https://developers.cloudflare.com/workers-ai/models/llama-3-8b-instruct/#scoped-prompts)

Basically instead of a single system message with the task and response type, you have the system message with the AIs goal followed by examples of how you want it to respond by using the user/assistant roles.

The example that CF specifically has in the docs:

{
  messages: [
    { role: "system", content: "you are a professional computer science assistant" },
    { role: "user", content: "what is WASM?" },
    { role: "assistant", content: "WASM (WebAssembly) is a binary instruction format that is designed to be a platform-agnostic" },
    { role: "user", content: "does Python compile to WASM?" },
    { role: "assistant", content: "No, Python does not directly compile to WebAssembly" },
    { role: "user", content: "what about Rust?" },
  ],
};

{
  messages: [
    { role: "system", content: "you are a professional computer science assistant" },
    { role: "user", content: "what is WASM?" },
    { role: "assistant", content: "WASM (WebAssembly) is a binary instruction format that is designed to be a platform-agnostic" },
    { role: "user", content: "does Python compile to WASM?" },
    { role: "assistant", content: "No, Python does not directly compile to WebAssembly" },
    { role: "user", content: "what about Rust?" },
  ],
};

UUnsmart As explained in the Workers AI docs for each model you want to use scoped prompt...

Telion•6/2/24, 3:40 AM

Right, my issue was that scoped prompts don't improve the results, they actually make it worse. I specifically provided an example of scope prompts first, then with the images I demonstrated that without scoped prompts I get better results but they are still not any close to being what LLAMA on HF or Meta.ai provide.

Telion•6/2/24, 3:40 AM

The first example with scoped prompts I added above results in absolutely random results while putting everything into user prompts actually improves it by a lot

Unsmart•6/2/24, 3:42 AM

The example you showed technically has scoped prompts yes, but not used in a the best way. If you follow the prompting method recommended of

system: you do this
user: an example thing a user might say
assistant: what you want the response to be
user: example 2
assistant: answer 2
user: <actual input>

It will work wonderfully, I have had 0 issues with responses when prompting in that format.

Telion•6/2/24, 3:43 AM

Oh, I missed the assistant part. I noticed that not all models support this feature so I skipped it

Telion•6/2/24, 3:43 AM

In the example though assistant seems to be dynamic, does it get generated or how do I assign the right assistant prompt?

Unsmart•6/2/24, 3:44 AM

Its worked perfectly fine on every model I have used it on, no clue what doesnt support it... maybe the non text based ones

Unsmart•6/2/24, 3:45 AM

The assistant prompts are static, you just give it some generic examples. For a case where you want to have intent scoring you probably want to have like 2-3 examples for each category

Telion•6/2/24, 3:45 AM

Wait, is the assistant prompt just the history of answers?

Telion•6/2/24, 3:45 AM

that the llm made?

Unsmart•6/2/24, 3:45 AM

No that you make

Unsmart•6/2/24, 3:46 AM

You use the assistant to teach it how to respond

Telion•6/2/24, 3:46 AM

oh, wait I see

Telion•6/2/24, 3:46 AM

your example actually includes user prompts as part of the general mechanism describing it how ot behave

Telion•6/2/24, 3:46 AM

Let me try that, thank you!

Telion•6/2/24, 3:47 AM

I didn't think I can or should use user prompts to explain LLM something, I was 100% sure user prompts are jsut the user input and I shouldn't mess with that

UUnsmart No that you make

Telion•6/2/24, 3:50 AM

Wouldn't that mess with user's history a bit?

Telion•6/2/24, 3:51 AM

If I use any term llm will probably remember thatn and just assume that user either already knows about it or is still referring to it

Telion•6/2/24, 4:15 AM

Well, I tried, it got a lot more stable which is a huge improvement so far, although it still deviates a lot from what I get on meta.ai on the same model for some reason. On meta.ai i literally get perfect results. I am not sure what may be different there

Craig Dennis•6/3/24, 8:48 PM

Hi @Aurelios this might be just what you are looking for! https://github.com/craigsdennis/vanilla-chat-workers-ai/

GitHub

GitHub - craigsdennis/vanilla-chat-workers-ai

Contribute to craigsdennis/vanilla-chat-workers-ai development by creating an account on GitHub.

CCraig Dennis Hi @Aurelios this might be just what you are looking for! https://github.com/cra...

Victor•6/5/24, 4:54 PM

yoo, I'm gonna steal your json model list generation, but instead of beta/ga, I'll make it replace

modelMappings

modelMappings

that's in

@cloudflare/ai

@cloudflare/ai

but gone in

@cloudflare/workers-types

@cloudflare/workers-types

MR X•6/6/24, 3:19 AM

hello how to make cross x forwarded for ip in workers

VVictor yoo, I'm gonna steal your json model list generation, but instead of beta/ga, I'...

Craig Dennis•6/6/24, 11:14 PM

Steal away! And yeah modelMappings should be considered on it's way to deprecation

Atony•6/7/24, 2:59 AM

Hello, could you please ask if it is possible to integrate the Qwen2 series models into the Cloudflare AI platform? Thank you.

TurboFarmer9000•6/7/24, 2:31 PM

Was glad to see that cloudflare has whisper, but the particular model is not mentioned anywhere. I searched this channel and now i can see that it is the base model. OpenAI is $0.006 vs $0.0022 BUT it is running the large v3 model. If cloudflare adds more model options it would be attractive

FieraRyan•6/7/24, 10:50 PM

can I use beta llm model in prod? I need the latest model (llama 3-8b) but its not out of beta yet. Do you know the rate limit on free beta models?

FFieraRyan can I use beta llm model in prod? I need the latest model (llama 3-8b) but its n...

Victor•6/7/24, 11:25 PM

From a technical perspective, there's nothing stopping you. However beta comes without SLA guarantees and is simply "best effort". Usage is also free for the duration while in beta, but will be charged after. Also

Beta models may have lower rate limits while we work on performance and scale

https://developers.cloudflare.com/workers-ai/platform/limits/

Cloudflare Docs

Limits · Cloudflare Workers AI docs

Workers AI is now Generally Available. We’ve updated our rate limits to reflect this.

AAtony Hello, could you please ask if it is possible to integrate the Qwen2 series mode...

MissS•6/7/24, 11:45 PM

When qwen2 :O?

FieraRyan•6/8/24, 1:07 AM

Does this mean Paid Workers ($5) give 300000 neurons for free, or is this the upper limit?

Unsmart•6/8/24, 1:08 AM

its included usage, no upper limits beyond standard rate limits

MMissS When qwen2 :O?

Isaac McFadyen•6/8/24, 2:39 AM

Considering it was literally released yesterday, it might be a bit still

UUnsmart its included usage, no upper limits beyond standard rate limits

Fierylion•6/8/24, 1:48 PM

what does included mean? do you mean the $5 includes 300000 neurons?

rob•6/8/24, 7:53 PM

it's confusing

Rrob it's confusing

Unsmart•6/8/24, 8:19 PM

What is? The included usage? Not sure how else they could word it to say that you get usage included

Fierylion•6/8/24, 9:18 PM

are you sure its free 300000?

const intent = await env.AI.run('@cf/meta/llama-3-8b-instruct', { messages: [ { "role": "system", "content": `I will help you figure out intent of the message based on available options. Available intents:\n${INTENTS.join('\n')}\n\nI will choose an intent from the list that best matches the user's prompt and return only its number. There will be no other text in my response other then the found number. I will use the following format: [intent number from the list].\n\nExample of response when found:\n3\n\nExample of when intent cannot be determined:\nNo intent` }, { "role": "user", "content": prompt } ] });

{ messages: [ { role: "system", content: "you are a professional computer science assistant" }, { role: "user", content: "what is WASM?" }, { role: "assistant", content: "WASM (WebAssembly) is a binary instruction format that is designed to be a platform-agnostic" }, { role: "user", content: "does Python compile to WASM?" }, { role: "assistant", content: "No, Python does not directly compile to WebAssembly" }, { role: "user", content: "what about Rust?" }, ], };

Hi guys, how can use this worker ai ? Have someone the Java script for it ?

Similar Threads

Similar Threads

Similar Threads