is the rest api reference accurate, cause it says that llama 3.1 accepts a prompt of up to 131k char

K

KraiiOP•7/27/24, 8:35 PM

K

KraiiOP•7/27/24, 8:36 PM

KKraii is the rest api reference accurate, cause it says that llama 3.1 accepts a promp...

R

Raylight•7/28/24, 8:39 AM

There's also a token limit that depends on which model you use. Looks like llama 3.1 on Cloudflare currently has a limit of ~6000 tokens.

K

KraiiOP•7/29/24, 7:19 AM

yea but i cant find any info on the token limits

KKraii yea but i cant find any info on the token limits

R

Raylight•7/29/24, 2:34 PM

Yeah I know.. With a few exceptions, you won't be able to find it. Ideally, one should be able to get info about limits, token counts, default prompt, supported params/features etc etc via methods and API calls. In any case, here's a list of tokens limits that covers most of the models, based on my own tests: https://gist.github.com/Raylight-PWL/e92b298850625c80efa6ca4ddd206662

K

KraiiOP•7/29/24, 7:37 PM

ow that's great

K

KraiiOP•7/29/24, 7:37 PM

thank you very much :firSweet:

R

Raylight•7/29/24, 10:01 PM

Found a somewhat weird bug: models that start with

@hf/thebloke

@hf/thebloke

@hf/thebloke

@hf/thebloke won't generate a response if max_tokens is set to 597 or higher. (Fails with "3025: Unknown internal error".)

D

David Liu•7/30/24, 5:04 PM

Hey Everyone!
We’re super excited to launch this awesome demo playground where you can use audio, video, images, and text to create multimodal AI applications. We ask that you take it for a spin, try it out and post what you build. There are some preloaded examples to get you started but feel free to build whatever you like! We’ll retweet your posts on X. Try it here https://multi-modal.ai.cloudflare.com/

D

David Liu•7/30/24, 5:05 PM

Here's me using it to turn an image from Cloudflare's website to HTML code

U

user•7/30/24, 6:50 PM

Uhm, did the

runWithTools

runWithTools

runWithTools

runWithTools syntax change?

U

user•7/30/24, 6:53 PM

When I call

runWithTools

runWithTools

runWithTools

runWithTools I get

Error in runWithTools: Error in runAndProcessToolCall: Bad input: must have required property 'prompt', must have required property 'name', must have required property 'description', must have required property 'parameters', must have required property 'type', must have required property 'properties', must match exactly one schema in oneOf, must match exactly one schema in oneOf,

Error in runWithTools: Error in runAndProcessToolCall: Bad input: must have required property 'prompt', must have required property 'name', must have required property 'description', must have required property 'parameters', must have required property 'type', must have required property 'properties', must match exactly one schema in oneOf, must match exactly one schema in oneOf,

Error in runWithTools: Error in runAndProcessToolCall: Bad input: must have required property 'prompt', must have required property 'name', must have required property 'description', must have required property 'parameters', must have required property 'type', must have required property 'properties', must match exactly one schema in oneOf, must match exactly one schema in oneOf,

Error in runWithTools: Error in runAndProcessToolCall: Bad input: must have required property 'prompt', must have required property 'name', must have required property 'description', must have required property 'parameters', must have required property 'type', must have required property 'properties', must match exactly one schema in oneOf, must match exactly one schema in oneOf,

U

user•7/30/24, 7:08 PM

i cant seem to find where the "Bad input" part comes from, is it an upstream issue rather than an ai-utils issue?

E

Egor•7/31/24, 10:42 AM

Hello everyone

! Please tell me if it is possible to understand how many Neurons were used in each request to the model?

Uuser When I call `runWithTools` I get `Error in runWithTools: Error in runAndProcessT...

U

user•7/31/24, 11:44 PM

Sorry for the ping, @Dhravya Shah, but you're the only know I can think of who can help with this xD

U

user•7/31/24, 11:45 PM

same code snippet as last time produces this error, no changes were made

U

user•7/31/24, 11:48 PM

Qwen

U

user•7/31/24, 11:48 PM

Still trying to figure out functions with qwen :ye:

U

user•7/31/24, 11:48 PM

@cf/qwen/qwen1.5-14b-chat-awq

@cf/qwen/qwen1.5-14b-chat-awq

@cf/qwen/qwen1.5-14b-chat-awq

@cf/qwen/qwen1.5-14b-chat-awq

DDhravya Shah looking into it, upstream issue, maybe I'm parsing it wrong or API is returning ...

U

user•7/31/24, 11:49 PM

My immediate guess is the upstream api changed and ai-utils is no longer sending the right data

U

user•7/31/24, 11:50 PM

because the only parts of the error that I could find in ai-utils was

Error in runWithTools: Error in runAndProcessToolCall:

Error in runWithTools: Error in runAndProcessToolCall:

Error in runWithTools: Error in runAndProcessToolCall:

Error in runWithTools: Error in runAndProcessToolCall: , making me think that maybe the "Bad input" part was a response from the api

U

user•7/31/24, 11:59 PM

is it maybe the tool parsing?

U

user•8/1/24, 12:00 AM

i dont see where the "prompt" property would come in as I don't see that documented anywhere, but everything else is properties of a tool

E

E S•8/1/24, 4:24 AM

https://tenor.com/view/dancing-cat-dance-cat-cat-meme-chinese-cat-gif-12629347036627000898

Tenor

K

konedi•8/2/24, 1:16 PM

Hi all, anyway to speed up workers ai through rest api? responses are taking up to 30s, while I have a local llama3.1 running on a rtx3080 and it only takes like 6s?

O

oakwhiz•8/2/24, 8:25 PM

What's the pricing for neurons on text to image generation?

Uuser i dont see where the "prompt" property would come in as I don't see that documen...

R

Raylight•8/2/24, 9:05 PM

These error messages can be quite cryptic. Omitting the property "type" inside a single parameter of a single tool would produce the same error, for example. In your example with

parameters: {}

parameters: {}

parameters: {}

parameters: {} I got past the error by changing it to

parameters: { type: "object", properties: {} }

parameters: { type: "object", properties: {} }

parameters: { type: "object", properties: {} }

parameters: { type: "object", properties: {} }.

RRaylight These error messages can be quite cryptic. Omitting the property "type" inside a...

U

user•8/2/24, 9:07 PM

Interesting.

U

user•8/2/24, 9:07 PM

That is very cryptic indeed

U

user•8/2/24, 9:07 PM

I feel like this is definitely an issue with upstream

A

AWTom•8/3/24, 10:57 PM

I see that four of the text generation models support LoRA adapters. Is LoRA support for text-to-image models currently in the works?

H

heloo_mrinal•8/4/24, 10:28 AM

hey this side mrinal I have a prior project on cloudflare using workers-ai, but when started with another project using ai it showed me the there is likely additional logging output above

H

heloo_mrinal•8/4/24, 11:22 AM

any soution for that ?

R

ring0•8/5/24, 10:46 PM

Anyone know of a full (working) example project with supporting files for an AI worker utilizing OpenAI tool usage? Not just a snippet that assumes everything else is gtg. So far my project is pretty capable of generating frustration and 500 errors.

Ooakwhiz What's the pricing for neurons on text to image generation?

K

Kathy•8/6/24, 5:49 PM

All our text-to-image models are currently in beta, which means they are free to use until they graduate from beta
https://developers.cloudflare.com/workers-ai/models/#text-to-image

Cloudflare Docs

Models · Cloudflare Workers AI docs

Browse our entire catalog of models:

AAWTom I see that four of the text generation models support LoRA adapters. Is LoRA sup...

K

Kathy•8/6/24, 5:51 PM

is there a specific model you are interested in using LoRAs with?

C

Crystal Shardz•8/7/24, 2:31 PM

Hi, I have written a Vue 3 app which uses Vite to talk to the Llama AI worker. This works fine when running locally via the vite dev server as I am using a proxy however, after deploying to Cloudflare Pages, my requests are no longer proxied via the Vite dev server so the requests are going to the Llama AI but the OPTIONS request comes back with a 400 error. Any ideas how I can get this working correctly? Appologies if this needs posing in the Pages section of Discord - it's cross product so felt this was a better place as the issue is with talking from CF Pages to the CF AI

B

Beyondo•8/8/24, 1:46 AM

Is there a way to specify stop tokens? This is very important to me cause I mainly will be using the

raw

raw

raw

raw parameter

RRaylight These error messages can be quite cryptic. Omitting the property "type" inside a...

U

user•8/8/24, 4:14 AM

Were you able to get the functions to actually execute with the model?

U

user•8/8/24, 4:15 AM

because I'm still unable to do so lol

Uuser Were you able to get the functions to actually execute with the model?

R

Raylight•8/8/24, 9:57 AM

With qwen? no. I don't think qwen is supported. I would like to see Cloudflare add support for "stop words" so that I can implement it myself.

W

Wouter J•8/8/24, 9:57 AM

Hey all, any chance CF is working on a Background Removal from image integration? e.g. @imgly/background-removal-node.

I'd love to use it in a worker, but since it's downloading models on the fly (edit: oh for prod you can serve the models statically, but still not possible/ a good thing to do in a Worker I think) I need to implement it in my NodeJS API.

So was just wondering if there was anything in the pipeline for it. Keep rocking.

KKorcan `InferenceUpstreamError: undefined: undefined` ``` at Ai._parseError (cloudflare...

A

Aleksandar•8/8/24, 2:42 PM

Hello. Was this issue with

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined fixed? I'm getting the same error using

@cf/microsoft/resnet-50

@cf/microsoft/resnet-50

@cf/microsoft/resnet-50

@cf/microsoft/resnet-50 with a PNG and this code https://gist.github.com/net-tech/0eb5e0caa05c6be931ab373cbb35f360

Gist

Code + error on CF workers AI

Code + error on CF workers AI. GitHub Gist: instantly share code, notes, and snippets.

SSeanShan can anyone help me with this since our service relies on this

I

Isaac McFadyen•8/8/24, 5:42 PM

Workers AI/Cloudflare won't have any control of this, if it did happen (not sure if it will) it would be due to the underlying model which is

Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc

Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc

Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc

Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc and how it's been trained.

is the rest api reference accurate, cause it says that llama 3.1 accepts a prompt of up to 131k char

Similar Threads

Similar Threads

Similar Threads