Found a somewhat weird bug: models that start with `@hf/thebloke` won't generate a response if max_t

D

David Liu•7/30/24, 5:04 PM

Hey Everyone!
We’re super excited to launch this awesome demo playground where you can use audio, video, images, and text to create multimodal AI applications. We ask that you take it for a spin, try it out and post what you build. There are some preloaded examples to get you started but feel free to build whatever you like! We’ll retweet your posts on X. Try it here https://multi-modal.ai.cloudflare.com/

D

David Liu•7/30/24, 5:05 PM

Here's me using it to turn an image from Cloudflare's website to HTML code

U

user•7/30/24, 6:50 PM

Uhm, did the

runWithTools

runWithTools

runWithTools

runWithTools syntax change?

U

user•7/30/24, 6:53 PM

When I call

runWithTools

runWithTools

runWithTools

runWithTools I get

Error in runWithTools: Error in runAndProcessToolCall: Bad input: must have required property 'prompt', must have required property 'name', must have required property 'description', must have required property 'parameters', must have required property 'type', must have required property 'properties', must match exactly one schema in oneOf, must match exactly one schema in oneOf,

Error in runWithTools: Error in runAndProcessToolCall: Bad input: must have required property 'prompt', must have required property 'name', must have required property 'description', must have required property 'parameters', must have required property 'type', must have required property 'properties', must match exactly one schema in oneOf, must match exactly one schema in oneOf,

Error in runWithTools: Error in runAndProcessToolCall: Bad input: must have required property 'prompt', must have required property 'name', must have required property 'description', must have required property 'parameters', must have required property 'type', must have required property 'properties', must match exactly one schema in oneOf, must match exactly one schema in oneOf,

Error in runWithTools: Error in runAndProcessToolCall: Bad input: must have required property 'prompt', must have required property 'name', must have required property 'description', must have required property 'parameters', must have required property 'type', must have required property 'properties', must match exactly one schema in oneOf, must match exactly one schema in oneOf,

U

user•7/30/24, 7:08 PM

i cant seem to find where the "Bad input" part comes from, is it an upstream issue rather than an ai-utils issue?

E

Egor•7/31/24, 10:42 AM

Hello everyone

! Please tell me if it is possible to understand how many Neurons were used in each request to the model?

Uuser When I call `runWithTools` I get `Error in runWithTools: Error in runAndProcessT...

U

user•7/31/24, 11:44 PM

Sorry for the ping, @Dhravya Shah, but you're the only know I can think of who can help with this xD

U

user•7/31/24, 11:45 PM

same code snippet as last time produces this error, no changes were made

U

user•7/31/24, 11:48 PM

Qwen

U

user•7/31/24, 11:48 PM

Still trying to figure out functions with qwen :ye:

U

user•7/31/24, 11:48 PM

@cf/qwen/qwen1.5-14b-chat-awq

@cf/qwen/qwen1.5-14b-chat-awq

@cf/qwen/qwen1.5-14b-chat-awq

@cf/qwen/qwen1.5-14b-chat-awq

DDhravya Shah looking into it, upstream issue, maybe I'm parsing it wrong or API is returning ...

U

user•7/31/24, 11:49 PM

My immediate guess is the upstream api changed and ai-utils is no longer sending the right data

U

user•7/31/24, 11:50 PM

because the only parts of the error that I could find in ai-utils was

Error in runWithTools: Error in runAndProcessToolCall:

Error in runWithTools: Error in runAndProcessToolCall:

Error in runWithTools: Error in runAndProcessToolCall:

Error in runWithTools: Error in runAndProcessToolCall: , making me think that maybe the "Bad input" part was a response from the api

U

user•7/31/24, 11:59 PM

is it maybe the tool parsing?

U

user•8/1/24, 12:00 AM

i dont see where the "prompt" property would come in as I don't see that documented anywhere, but everything else is properties of a tool

E

E S•8/1/24, 4:24 AM

https://tenor.com/view/dancing-cat-dance-cat-cat-meme-chinese-cat-gif-12629347036627000898

Tenor

K

konedi•8/2/24, 1:16 PM

Hi all, anyway to speed up workers ai through rest api? responses are taking up to 30s, while I have a local llama3.1 running on a rtx3080 and it only takes like 6s?

O

oakwhiz•8/2/24, 8:25 PM

What's the pricing for neurons on text to image generation?

Uuser i dont see where the "prompt" property would come in as I don't see that documen...

R

RaylightOP•8/2/24, 9:05 PM

These error messages can be quite cryptic. Omitting the property "type" inside a single parameter of a single tool would produce the same error, for example. In your example with

parameters: {}

parameters: {}

parameters: {}

parameters: {} I got past the error by changing it to

parameters: { type: "object", properties: {} }

parameters: { type: "object", properties: {} }

parameters: { type: "object", properties: {} }

parameters: { type: "object", properties: {} }.

RRaylight These error messages can be quite cryptic. Omitting the property "type" inside a...

U

user•8/2/24, 9:07 PM

Interesting.

U

user•8/2/24, 9:07 PM

That is very cryptic indeed

U

user•8/2/24, 9:07 PM

I feel like this is definitely an issue with upstream

A

AWTom•8/3/24, 10:57 PM

I see that four of the text generation models support LoRA adapters. Is LoRA support for text-to-image models currently in the works?

H

heloo_mrinal•8/4/24, 10:28 AM

hey this side mrinal I have a prior project on cloudflare using workers-ai, but when started with another project using ai it showed me the there is likely additional logging output above

H

heloo_mrinal•8/4/24, 11:22 AM

any soution for that ?

R

ring0•8/5/24, 10:46 PM

Anyone know of a full (working) example project with supporting files for an AI worker utilizing OpenAI tool usage? Not just a snippet that assumes everything else is gtg. So far my project is pretty capable of generating frustration and 500 errors.

Ooakwhiz What's the pricing for neurons on text to image generation?

K

Kathy•8/6/24, 5:49 PM

All our text-to-image models are currently in beta, which means they are free to use until they graduate from beta
https://developers.cloudflare.com/workers-ai/models/#text-to-image

Cloudflare Docs

Models · Cloudflare Workers AI docs

Browse our entire catalog of models:

AAWTom I see that four of the text generation models support LoRA adapters. Is LoRA sup...

K

Kathy•8/6/24, 5:51 PM

is there a specific model you are interested in using LoRAs with?

C

Crystal Shardz•8/7/24, 2:31 PM

Hi, I have written a Vue 3 app which uses Vite to talk to the Llama AI worker. This works fine when running locally via the vite dev server as I am using a proxy however, after deploying to Cloudflare Pages, my requests are no longer proxied via the Vite dev server so the requests are going to the Llama AI but the OPTIONS request comes back with a 400 error. Any ideas how I can get this working correctly? Appologies if this needs posing in the Pages section of Discord - it's cross product so felt this was a better place as the issue is with talking from CF Pages to the CF AI

B

Beyondo•8/8/24, 1:46 AM

Is there a way to specify stop tokens? This is very important to me cause I mainly will be using the

raw

raw

raw

raw parameter

RRaylight These error messages can be quite cryptic. Omitting the property "type" inside a...

U

user•8/8/24, 4:14 AM

Were you able to get the functions to actually execute with the model?

U

user•8/8/24, 4:15 AM

because I'm still unable to do so lol

Uuser Were you able to get the functions to actually execute with the model?

R

RaylightOP•8/8/24, 9:57 AM

With qwen? no. I don't think qwen is supported. I would like to see Cloudflare add support for "stop words" so that I can implement it myself.

W

Wouter J•8/8/24, 9:57 AM

Hey all, any chance CF is working on a Background Removal from image integration? e.g. @imgly/background-removal-node.

I'd love to use it in a worker, but since it's downloading models on the fly (edit: oh for prod you can serve the models statically, but still not possible/ a good thing to do in a Worker I think) I need to implement it in my NodeJS API.

So was just wondering if there was anything in the pipeline for it. Keep rocking.

KKorcan `InferenceUpstreamError: undefined: undefined` ``` at Ai._parseError (cloudflare...

A

Aleksandar•8/8/24, 2:42 PM

Hello. Was this issue with

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined

InferenceUpstreamError: undefined: undefined fixed? I'm getting the same error using

@cf/microsoft/resnet-50

@cf/microsoft/resnet-50

@cf/microsoft/resnet-50

@cf/microsoft/resnet-50 with a PNG and this code https://gist.github.com/net-tech/0eb5e0caa05c6be931ab373cbb35f360

Gist

Code + error on CF workers AI

Code + error on CF workers AI. GitHub Gist: instantly share code, notes, and snippets.

SSeanShan can anyone help me with this since our service relies on this

I

Isaac McFadyen•8/8/24, 5:42 PM

Workers AI/Cloudflare won't have any control of this, if it did happen (not sure if it will) it would be due to the underlying model which is

Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc

Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc

Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc

Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static-inc and how it's been trained.

A

Alex Potemkin•8/10/24, 10:46 AM

Hey, I wonder how I can be notified when specific AI models on AI workers will be out of beta?

AAleksandar Hello. Was this issue with `InferenceUpstreamError: undefined: undefined` fixed?...

A

Aleksandar•8/10/24, 11:53 AM

Can anyone help with this

A

anywhichway•8/10/24, 2:14 PM

I have a really basic AI request:

export async function onRequest(context) {
    const input = { prompt: "What is the origin of the phrase Hello, World" }
    const answer = await context.env.AI.run('@cf/meta/llama-3.1-8b-instruct', input);
    return Response.json(answer);
}

export async function onRequest(context) {
    const input = { prompt: "What is the origin of the phrase Hello, World" }
    const answer = await context.env.AI.run('@cf/meta/llama-3.1-8b-instruct', input);
    return Response.json(answer);
}

export async function onRequest(context) {
    const input = { prompt: "What is the origin of the phrase Hello, World" }
    const answer = await context.env.AI.run('@cf/meta/llama-3.1-8b-instruct', input);
    return Response.json(answer);
}

export async function onRequest(context) {
    const input = { prompt: "What is the origin of the phrase Hello, World" }
    const answer = await context.env.AI.run('@cf/meta/llama-3.1-8b-instruct', input);
    return Response.json(answer);
}

The function gets called (I can put a debugger break in dev tools ). The AI run never seems to return and I ultimately get the error below. I have authorized wrangler to access my account and confirmed the AI is bound in wrangler.toml:

A ReadableStream branch was created but never consumed. Such branches can be created, for instance, by calling the tee() method on a ReadableStream, or by calling the clone() method on a Request or Response object. If a branch is created but never consumed, it can force the runtime to buffer the entire body of the stream in memory, which may cause the Worker to exceed its memory limit and be terminated. To avoid this, ensure that all branches created are consumed.

A ReadableStream branch was created but never consumed. Such branches can be created, for instance, by calling the tee() method on a ReadableStream, or by calling the clone() method on a Request or Response object. If a branch is created but never consumed, it can force the runtime to buffer the entire body of the stream in memory, which may cause the Worker to exceed its memory limit and be terminated. To avoid this, ensure that all branches created are consumed.

A ReadableStream branch was created but never consumed. Such branches can be created, for instance, by calling the tee() method on a ReadableStream, or by calling the clone() method on a Request or Response object. If a branch is created but never consumed, it can force the runtime to buffer the entire body of the stream in memory, which may cause the Worker to exceed its memory limit and be terminated. To avoid this, ensure that all branches created are consumed.

A ReadableStream branch was created but never consumed. Such branches can be created, for instance, by calling the tee() method on a ReadableStream, or by calling the clone() method on a Request or Response object. If a branch is created but never consumed, it can force the runtime to buffer the entire body of the stream in memory, which may cause the Worker to exceed its memory limit and be terminated. To avoid this, ensure that all branches created are consumed.

M

Muhammadreza Haghiri•8/10/24, 5:21 PM

Hey guys, what is up?
I'm curious, is there a way to request to add a model to the list? I'm just looking for opportunities for my models to be added to public lists/tools such as workers AI.

T

thipperz•8/10/24, 7:21 PM

A good Image Embedding model someday...? Would fit so nicely with Vectorize wouldn't it?

AAleksandar Can anyone help with this

R

RaylightOP•8/11/24, 9:20 AM

I can't reproduce "undefined: undefined" but haven't tried running locally. That said, make sure that the image is smaller than 2.5-3 MB. (

JSON.stringify(inputs).length

JSON.stringify(inputs).length

JSON.stringify(inputs).length

JSON.stringify(inputs).length needs to be less than 9.5 MB, to be precise.)

J

JCBahn•8/11/24, 9:32 AM

When I make a query in a non-English language, the response comes out broken. Has anyone else experienced this? This doesn't happen when I use llama3.1 with ollama. Please improve support for non-English queries. Especially for Llama 3.1.

RRaylight I can't reproduce "undefined: undefined" but haven't tried running locally. That...

A

Aleksandar•8/11/24, 12:12 PM

that was happening locally. ill check the image size and get back to you

Found a somewhat weird bug: models that start with `@hf/thebloke` won't generate a response if max_t

Similar Threads

Similar Threads

Similar Threads