Custom models are not currently supported. Only models that are added by the Cloudflare team (so not

Custom models are not currently supported. Only models that are added by the Cloudflare team (so not any HuggingFace model, just the ones that are on the list)

_querty123_•9/11/24, 12:23 AM

all listed models enter on paid plan? i mean, if i pay all moedls listed will be discount of my usage?

_querty123_•9/11/24, 12:23 AM

from my neurons?

_querty123_•9/11/24, 12:25 AM

im starting with ia workers, ive created one worker with await env.AI.run("@hf/meta-llama/meta-llama-3-8b-instruct" on it, i want to know: when i call endpoint, usage is discounted from where? here?

IIsaac McFadyen Custom models are not currently supported. Only models that are added by the Clo...

Jadugar_Jaggu•9/11/24, 4:57 AM

I think, using custom models will be a more competitive feature, and can summon the whole Ai industry to CF.

CF should look into this

lee•9/11/24, 7:31 AM

Can the AI models used by Worker AI only be those provided by Cloudflare officially, or can users upload their own private AI models for use?

vin•9/11/24, 9:22 AM

For now users can not upload custom models

vin•9/11/24, 9:24 AM

about a year ago, cloudflare introduced a project called Constellation, its APIs are much much lower level https://blog.cloudflare.com/introducing-constellation

The Cloudflare Blog

Introducing Constellation, bringing AI to the Cloudflare stack

Today, we're excited to welcome Constellation to the Cloudflare stack. Constellation allows you to run fast, low-latency inference tasks on pre-trained machine learning models natively on Workers.

vin•9/11/24, 9:26 AM

you can upload any small ONNX models to constellation, and do inference things. but it's hardly to use, you have to translate all inference preprocess/postprocess to workers code

vin•9/11/24, 9:27 AM

and it seems completely closed now

lee•9/11/24, 12:06 PM

Oh，got it！Thank you!

user•9/12/24, 12:46 AM

@cf/qwen/qwen1.5-14b-chat-awq

@cf/qwen/qwen1.5-14b-chat-awq

still isnt working with functions, is there any ETA on this?

IIsaac McFadyen Yeaaah Stable Diffusion isn't great at text sadly \:(

JustinNoel•9/12/24, 1:00 PM

DALL-E is much better.

JJustinNoel DALL-E is much better.

Isaac McFadyenOP•9/12/24, 1:10 PM

Not surprised, yeah. The newer models are decent.

Isaac McFadyenOP•9/12/24, 1:10 PM

FLUX.1 in particular.

JJustinNoel DALL-E is much better.

a4amaan•9/12/24, 6:15 PM

is DALL-E Available on cloudflare?

Aa4amaan is DALL-E Available on cloudflare?

Flask•9/12/24, 6:54 PM

No for Workers AI at the moment, but you could use the OpenAI Endpoint in AI Gateway to make request to OpenAI: https://developers.cloudflare.com/ai-gateway/providers/openai/

Cloudflare Docs

OpenAI | Cloudflare AI Gateway docs

OpenAI

helps you build with ChatGPT.

FFlask No for Workers AI at the moment, but you could use the OpenAI Endpoint in AI Gat...

a4amaan•9/12/24, 7:03 PM

ok thanks

Uuser `@cf/qwen/qwen1.5-14b-chat-awq` still isnt working with functions, is there any ...

user•9/13/24, 12:45 AM

Anything?

Ian•9/13/24, 2:20 PM

same with detr-resnet-50, I want to use custom weights

Ian•9/14/24, 1:52 AM

While I am still at the drawing board stage therefore do not want to hit this form. Anyone know if this incurs extra or one-time charges and how expensive they are. I am looking for running yolov8 on workers AI.

cookie•9/14/24, 10:43 PM

is support for json mode planned

user•9/15/24, 2:21 AM

I keep randomly getting the exception

Error in runWithTools: Error in runAndProcessToolCall: Unknown internal error

Error in runWithTools: Error in runAndProcessToolCall: Unknown internal error

DoctorU•9/15/24, 3:47 AM

How do you find out your usage of workers AI models API? It always shows 0/10,000 neurons for me, although I have sent possibly hundreds of requests.

DDoctorU How do you find out your usage of workers AI models API? It always shows 0/10,00...

user•9/15/24, 3:48 AM

Youre likely using a beta model

Uuser Youre likely using a beta model

DoctorU•9/15/24, 3:49 AM

Yes, I am using multiple beta models. Are they not counted?

user•9/15/24, 3:50 AM

Nope

user•9/15/24, 3:50 AM

For as long as theyre in beta you have inf usage

DoctorU•9/15/24, 3:50 AM

That's really cool! Thanks for clarifying this for me.

Stéphane Magnenat•9/16/24, 8:23 AM

Hello. Does anyone knows the neuron cost of the model

llama-3.1-8b-instruct-awq

llama-3.1-8b-instruct-awq

? I could not find this information. I am particularly interested in knowing whether that one is significantly cheaper than

llama-3.1-8b-instruct-fp8

llama-3.1-8b-instruct-fp8

. It seems that overall the neuron cost table is not up to date with the model list.

Stéphane Magnenat•9/16/24, 8:33 AM

Oh I see, sorry for the confusion!

Stéphane Magnenat•9/16/24, 8:34 AM

Do you know whether there will be a cost difference between the

awq

awq

and the

fp8

fp8

once they are out of beta? Just to know whether it is worth trying to stick with the smaller weight one during development.

FelikZ•9/16/24, 10:49 AM

Hello, do you plan to add "deepseek-coder-v2" anytime soon? Thanks

Ccodebam 4.20240620.0

acusti•9/17/24, 5:29 PM

@kian i get this same type error with

4.20240909.0

4.20240909.0

, the current most recent version of workers-types. however, i only started getting the error when i migrated from using the now deprecated

@cloudflare/ai

@cloudflare/ai

package to using

await env.AI.run(…)

await env.AI.run(…)

directly, so it seems like the

@cloudflare/ai

@cloudflare/ai

types allow me to use one of the

BaseAiTextGenerationModels

BaseAiTextGenerationModels

, whereas the types for the direct

AI

AI

binding (which use function overloading for usage with all possible types of AI generation) are, in my case, forcing the

model

model

argument to be type

BaseAiImageToTextModels

BaseAiImageToTextModels

. i am using typescript v5.6.2.

edit: discord makes it hard to follow the context. the type error i am getting is:

Argument of type '"@cf/meta/llama-3-8b-instruct" | "@hf/thebloke/zephyr-7b-beta-awq" | "@hf/nousresearch/hermes-2-pro-mistral-7b" | "@hf/nexusflow/starling-lm-7b-beta" | "@cf/defog/sqlcoder-7b-2" | "@cf/openchat/openchat-3.5-0106"' is not assignable to parameter of type 'BaseAiImageToTextModels'.

Argument of type '"@cf/meta/llama-3-8b-instruct" | "@hf/thebloke/zephyr-7b-beta-awq" | "@hf/nousresearch/hermes-2-pro-mistral-7b" | "@hf/nexusflow/starling-lm-7b-beta" | "@cf/defog/sqlcoder-7b-2" | "@cf/openchat/openchat-3.5-0106"' is not assignable to parameter of type 'BaseAiImageToTextModels'.

(my code has an enum of models that can be used with retry logic in case one of the models doesn't give a good result)

hari•9/19/24, 5:59 AM

what are the ratelimits of workers ai beta models? can you help to understand these limits please?

vin•9/19/24, 2:53 PM

here is the document https://developers.cloudflare.com/workers-ai/platform/limits/

Cloudflare Docs

Limits | Cloudflare Workers AI docs

Workers AI is now Generally Available. We’ve updated our rate limits to reflect this.

scotto•9/25/24, 7:37 PM

plans / date to support llama 3.2 3B ?

scotto•9/25/24, 7:37 PM

ideal model for cloudflore (size)

Ashish Rawat•9/26/24, 6:21 AM

check here: https://github.com/cloudflare/cloudflare-docs/pull/17119/files

GitHub

Workers AI Birthday Week Updates by mchenco · Pull Request #17119 ·...

Summary
new pricing page and changelog
dash refresh in separate PR
l3.2 in separate PR

Screenshots (optional)

Documentation checklist

[x ] The documentation style guide has been adhered to.

cookie•9/26/24, 6:42 AM

already here https://playground.ai.cloudflare.com/

crazyjack12•9/26/24, 7:45 AM

https://forms.gle/h7FcaTF4Zo5dzNb68 not open yet

Google Forms: Sign-in

Access Google Forms with a personal Google account or Google Workspace account (for business use).

akazwz•9/26/24, 4:22 PM

Keebs•9/26/24, 5:21 PM

const response = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
        prompt: "What is the origin of the phrase Hello, World",
      });

const response = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
        prompt: "What is the origin of the phrase Hello, World",
      });

anyone know why the local example code takes 25 seconds to respond?

Keebs•9/26/24, 5:34 PM

oh it might be the model

Keebs•9/26/24, 5:45 PM

no still slow 20 seconds minimum

Keebs•9/26/24, 5:45 PM

Response time: 716 milliseconds
[wrangler:inf] GET / 200 OK (739ms)

Response time: 716 milliseconds
[wrangler:inf] GET / 200 OK (739ms)

this is when i make a request to groq...

Bibi•9/26/24, 6:35 PM

Groq doesn’t use gpus so it’s obviously way faster but 25 seconds is indeed quite long

Argument of type '"@cf/meta/llama-3-8b-instruct" | "@hf/thebloke/zephyr-7b-beta-awq" | "@hf/nousresearch/hermes-2-pro-mistral-7b" | "@hf/nexusflow/starling-lm-7b-beta" | "@cf/defog/sqlcoder-7b-2" | "@cf/openchat/openchat-3.5-0106"' is not assignable to parameter of type 'BaseAiImageToTextModels'.

Custom models are not currently supported. Only models that are added by the Cloudflare team (so not

Similar Threads

Similar Threads

Similar Threads