i think its 10,000 neurons per day for free plan

SSunny i think its 10,000 neurons per day for free plan

A

a4amaan•9/17/24, 5:12 PM

yes i've read that.. i have not used neurons and getting about 300 words of article? is there any limit for single call?

S

SunnyOP•9/17/24, 5:15 PM

don't think so, it doesn't seem that you are setting a max_token limit here so it might be something to do with the llama 3 model?

S

SunnyOP•9/17/24, 5:15 PM

found this on hugging face forums for the llama 3 model

S

SunnyOP•9/17/24, 5:15 PM

"The output takes the same space as the input. The model completes the 8k token space with the response. You could slide the contxt window and get more output, but then you're losing context at the beginning.

Example, if your input is 100 tokens, you have ~7900 tokens for completion. But if your input is 7900 tokens, you have ~100 tokens left for response until your input starts getting trimmed out. The model can only pay attention to 8k tokens at most."

S

SunnyOP•9/17/24, 5:15 PM

I use llama 3.1 which has 128k token context window, it might be worth the upgrade in your use case if you are going over limits

SSunny I use llama 3.1 which has 128k token context window, it might be worth the upgra...

A

a4amaan•9/17/24, 5:18 PM

that suites good, thanks

A

a4amaan•9/17/24, 6:01 PM

@Sunny still not getting more than 300 words with 3.1

A

a4amaan•9/17/24, 6:04 PM

at the end it look like something stopped it, incomplete article

S

SunnyOP•9/17/24, 6:09 PM

it seems like there's a limit being set, maybe try manually setting max_tokens/max_length to be a higher limit? You could just try arbitrarily setting it to 100k?

SSunny it seems like there's a limit being set, maybe try manually setting max_tokens/m...

A

a4amaan•9/17/24, 6:21 PM

Thanks it worked:

const answer = await ctx.env.AI.run('@cf/meta/llama-3.1-8b-instruct', {
        messages,
        stream: true,
        max_tokens:10000
    });

const answer = await ctx.env.AI.run('@cf/meta/llama-3.1-8b-instruct', {
        messages,
        stream: true,
        max_tokens:10000
    });

const answer = await ctx.env.AI.run('@cf/meta/llama-3.1-8b-instruct', {
        messages,
        stream: true,
        max_tokens:10000
    });

const answer = await ctx.env.AI.run('@cf/meta/llama-3.1-8b-instruct', {
        messages,
        stream: true,
        max_tokens:10000
    });

A

Avocadio•9/18/24, 11:48 AM

Hey. I uploaded my finetuned model to cloudflare successfully but typescript gave that error and it request gives 3028 unknown internal error

A

Avocadio•9/18/24, 11:49 AM

It gives that error when i add lora parameter

A

Avocadio•9/18/24, 11:49 AM

Doesnt work with tsignore neither

A

Avocadio•9/18/24, 11:49 AM

Can you guys help

A

Avocadio•9/18/24, 11:55 AM

https://community.cloudflare.com/t/unknown-internal-error-for-lora-adapters/692895/3
Some other people faced the same issue

Ddigendee <@1136363561472491551> any updates?

K

Kathy•9/18/24, 8:38 PM

could i get your account ID and script name?

KKathy could i get your account ID and script name?

A

ajgeiss0702•9/19/24, 1:27 AM

i am currently getting the same issue (the 3028 error, not typescript). My account ID is

f55b85c8a963663b11036975203c63c0

f55b85c8a963663b11036975203c63c0

. I am using the http api to run it so i dont have a script name for this

A

ajgeiss0702•9/19/24, 1:28 AM

I am trying with the

@cf/mistral/mistral-7b-instruct-v0.1

@cf/mistral/mistral-7b-instruct-v0.1

@cf/mistral/mistral-7b-instruct-v0.1

@cf/mistral/mistral-7b-instruct-v0.1 model because i didnt know the v0.2 model existed on cloudflare until 2 minutes ago

A

ajgeiss0702•9/19/24, 1:29 AM

{"errors":[{"message":"AiError: AiError: Unknown internal error","code":3028}],"success":false,"result":{},"messages":[]}

{"errors":[{"message":"AiError: AiError: Unknown internal error","code":3028}],"success":false,"result":{},"messages":[]}

{"errors":[{"message":"AiError: AiError: Unknown internal error","code":3028}],"success":false,"result":{},"messages":[]}

{"errors":[{"message":"AiError: AiError: Unknown internal error","code":3028}],"success":false,"result":{},"messages":[]}

Aajgeiss0702 i am currently getting the same issue (the 3028 error, not typescript). My accou...

K

Kathy•9/19/24, 10:17 AM

wait i dont think this is the same issue at all. @digendee 's issue is because their worker is getting routed through HKG (which is blocked by Google Gemini) even though they're based in Melbourne (allowed by Google Gemini)

M

Mr.Spok•9/19/24, 5:44 PM

Hello everyone, please tell me about the application for Windows Cloudflare Warp - how to enable autoload? I didn't find anything on this topic in the FAQ, settings and documentation on the site.

I

Isaac McFadyen•9/20/24, 2:18 AM

Both of these questions probably should go in general-discussions , they're not related to Workers AI.

R

raj.js•9/20/24, 6:53 PM

is there any issue with vision models on cloudflare ai when trying to use from openai sdk? getting the below error

Error processing data: BadRequestError: 400 status code (no body)
at APIError.generate (file:/.........modules/openai/src/error.ts:70:14)

KKathy could i get your account ID and script name?

D

digendee•9/24/24, 6:55 AM

Sure, I'll DM you

�

😈 Donkey 💫•9/26/24, 6:51 AM

https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct
Hi,
docs 404 not found, please check

�😈 Donkey 💫https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruc...

C

cookie•9/26/24, 6:59 AM

they haven't announced it yet, so very soon probably

A

akazwz•9/26/24, 7:46 AM

soon, today blog will announced it ?

M

Moixllik•9/26/24, 10:28 AM

Are there models for Text-to-Speech and Text-to-Video? Will it be coming in the future? plus1green

K

Keebs•9/26/24, 3:11 PM

anyone here trained their own and has some pointers to get a quicker start to use it with workers? anything is appropriated ;)

F

Fernando Dilland•9/26/24, 7:53 PM

I am starting to consume the lightweight llama model using the Cloudflare API, but where can I see my consumption? I do not want to exceed 10,000 neurons per day, and if I exceed that, what happens? (I'm not using worker, I'm consuming the cloudflare API directly)

FFernando Dilland I am starting to consume the lightweight llama model using the Cloudflare API, b...

F

Fernando Dilland•9/26/24, 7:55 PM

My question is, do you request them for example: curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/meta/llama-3.2-11b-v...

Where or how can I see my consumption?

S

Stanleasy•9/27/24, 3:04 AM

the flux model documentation on API schema for input parameters all a sudden got a lot shorter in the couple hours i was afk. there were a bunch of parameters like image size and negative prompts that i don't see anymore
https://developers.cloudflare.com/workers-ai/models/flux-1-schnell

Cloudflare Docs

flux-1-schnell | Cloudflare Workers AI docs

FLUX.1 [schnell] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions.

C

Chaika•9/27/24, 3:24 AM

hmm yea https://github.com/cloudflare/cloudflare-docs/pull/17153/files 5h ago

GitHub

update Flux schema by LoganGrasby · Pull Request #17153 · cloudflar...

Summary

Screenshots (optional)

Documentation checklist

The documentation style guide has been adhered to.
If a larger change - such as adding a new page- an issue has been opened in relation ...

T

twentytwohalo•9/27/24, 6:11 AM

Hi I need an urgent support from you guys.
I have setup cloudflare on my domain (https://www.twentytwohalo.com/) and on Cloudflare dashboard it ask me Nameserver update pending.
I have updated the nameservers on my domain registrar. And now the cloudflare dashboard is showing an active status.

But when I update the nameserver of my hosting company and added A record and CNAME records in cloudflare DNS records (my site goes down and never up again even after waiting several hours).

Below are my domain and hosting companies:
Domain Registrar: Namecheap
Hosting: Dreamhost

Any help would be appreciated.
Thanks

22° H Δ L Oadmin

Home – EVENTS

Ttwentytwohalo Hi I need an urgent support from you guys. I have setup cloudflare on my domain...

D

dexter•9/27/24, 12:24 PM

I recommend you use another channel for nameservers/domains related question

SStanleasy the flux model documentation on API schema for input parameters all a sudden got...

D

dexter•9/27/24, 12:24 PM

Flux also returns a 500 error for me and other users in this discord, any idea why?

T

Tony Leung•9/29/24, 9:48 AM

it seems like seed in all stable diffusion models does not work at all. I'm trying to use the same seed and it all produced different output

T

Tony Leung•9/29/24, 10:00 AM

am I doing anything wrong :?

DDeleted User i am getting Error: Too many subrequests while running text embedding "@cf/baai/...

K

Kathy•9/30/24, 8:47 PM

https://developers.cloudflare.com/workers-ai/platform/limits/#text-embeddings

Cloudflare Docs

Limits | Cloudflare Workers AI docs

Workers AI is now Generally Available. We’ve updated our rate limits to reflect this.

CChaika hmm yea https://github.com/cloudflare/cloudflare-docs/pull/17153/files 5h ago

A

Advany•10/3/24, 8:47 AM

to bad they removed all the options...

K

kamlesh•10/3/24, 1:18 PM

Hi everyone! Does anyone know how to disable or remove the data-cfsrc tag? It automatically applies on mobile devices.

�

😈 Donkey 💫•10/4/24, 8:02 AM

Tried a test with llama 3.2 vision.
I put an image url of 2.2MB, but somehow the arrayBuffer inside worker inflating the payload that much 10MB then error

    if (attachment) {
        const res = await fetch("https://img.xyz" + attachment).catch(() => null);
        blob = await res?.arrayBuffer();
    }
    const response = await env.AI.run(model as any || '@cf/meta/llama-3.1-8b-instruct', {
        stream: stream,
        messages,
        max_tokens: 512,model (meta llama)
        ...(blob && { image: [...new Uint8Array(blob)] }),
    }

    if (attachment) {
        const res = await fetch("https://img.xyz" + attachment).catch(() => null);
        blob = await res?.arrayBuffer();
    }
    const response = await env.AI.run(model as any || '@cf/meta/llama-3.1-8b-instruct', {
        stream: stream,
        messages,
        max_tokens: 512,model (meta llama)
        ...(blob && { image: [...new Uint8Array(blob)] }),
    }

    if (attachment) {
        const res = await fetch("https://img.xyz" + attachment).catch(() => null);
        blob = await res?.arrayBuffer();
    }
    const response = await env.AI.run(model as any || '@cf/meta/llama-3.1-8b-instruct', {
        stream: stream,
        messages,
        max_tokens: 512,model (meta llama)
        ...(blob && { image: [...new Uint8Array(blob)] }),
    }

    if (attachment) {
        const res = await fetch("https://img.xyz" + attachment).catch(() => null);
        blob = await res?.arrayBuffer();
    }
    const response = await env.AI.run(model as any || '@cf/meta/llama-3.1-8b-instruct', {
        stream: stream,
        messages,
        max_tokens: 512,model (meta llama)
        ...(blob && { image: [...new Uint8Array(blob)] }),
    }

how can handle this? any limitation docs?

�

😈 Donkey 💫•10/4/24, 8:09 AM

and CF dashboard of gateway hanging too

C

csmykay•10/4/24, 7:06 PM

Who would we talk to about considering other models to add to the list of models currently listed on CloudFlare?

M

MarvinKR•10/7/24, 10:10 AM

how does cloudfare ai compares to the vercel ai sdk?

L

LM•10/7/24, 2:26 PM

exists model ai for detect texts written by ai? For example ZeroGPT

MMarvinKR how does cloudfare ai compares to the vercel ai sdk?

I

Isaac McFadyen•10/7/24, 4:44 PM

Vercel AI uses third-party providers like OpenAI, Anthropic, etc. Workers AI runs fully on the Cloudflare platform (with the exception of AI Gateway which is designed to use third-party providers).

IIsaac McFadyen Vercel AI uses third-party providers like OpenAI, Anthropic, etc. Workers AI run...

M

MarvinKR•10/7/24, 4:55 PM

Thanks for explaining this! What are the benefits of running fully on cloudfare?

i think its 10,000 neurons per day for free plan

Similar Threads

Similar Threads

Similar Threads