Cloudflare Developers•2y ago

Hi All. Is there any possibilities to host our own models on Cloudflare's AI?

MMuhammadreza Haghiri Hi All. Is there any possibilities to host our own models on Cloudflare's AI?

Victor•8/2/24, 11:16 PM

Part of constellation, but will return to workers ai soonsoontm

VVictor Part of constellation, but will return to workers ai <:soonsoontm:90367663474438...

Muhammadreza HaghiriOP•8/2/24, 11:19 PM

Nice. I am looking forward for this.

VVictor Part of constellation, but will return to workers ai <:soonsoontm:90367663474438...

Muhammadreza HaghiriOP•8/3/24, 7:51 AM

When will it return to Workers?

kcbrewron•8/3/24, 2:15 PM

Will workers AI update the version of llamaguard soon. Recently the llama 3.0 model was built for workers AI. The initial model doesn't include some of the specialized advice ML topology that I need in my industry. If possible, I'd love to see updates to allow llamaguard 3 and prompt guard

MMuhammadreza Haghiri When will it return to Workers?

Isaac McFadyen•8/3/24, 2:20 PM

There's no timeline right now, just word that it will happen eventually.

MMuhammadreza Haghiri When will it return to Workers?

Victor•8/3/24, 3:14 PM

If you do need it now urgently, that's why constellation is still around, albeit depreciated. So use at your own risk

Kkcbrewron Will workers AI update the version of llamaguard soon. Recently the llama 3.0 mo...

Victor•8/3/24, 3:14 PM

Currently (as of a month ago) there are no plans to, as their current focus is host your own models. But you will be able to run them that way once it launches

VVictor If you do need it now urgently, that's why constellation is still around, albeit...

Muhammadreza HaghiriOP•8/3/24, 3:21 PM

How can I use constellation? Couldn’t find the links or stuff

MMuhammadreza Haghiri How can I use constellation? Couldn’t find the links or stuff

Victor•8/3/24, 3:29 PM

yea, it's depreciated. Looks like the removed account entitlements for everyone for it unless you had talked with them and were actively using it

Victor•8/3/24, 3:30 PM

Honestly at this point, if you weren't using it, I wouldn't go down this path. Either wait for it (whenever it does) come back to workers ai, or use a different model hosting provider in the meantime

VVictor Honestly at this point, if you weren't using it, I wouldn't go down this path. E...

Muhammadreza HaghiriOP•8/3/24, 3:31 PM

any other providers recommendations?

MMuhammadreza Haghiri any other providers recommendations?

Victor•8/3/24, 3:34 PM

I don't have any recommendations as I don't run any custom models myself, but the only one I know of is hugging face (with the enterprise plan): iirc, you can have private models and use their inference api on it (albeit with their rate limits)

VVictor If you do need it now urgently, that's why constellation is still around, albeit...

Isaac McFadyen•8/3/24, 4:02 PM

I'd highly recommend against Constellation at this point, beyond being deprecated it's been removed from wrangler altogether.

Elektra•8/4/24, 2:51 PM

Hello everyone. I'm currently testing Cloudflare Whisper, and I received an error that the file is too large. The MP3 file is less than 4 MB. I searched the documentation and also searched here but couldn't find an answer. Could someone tell me what are the limits for whisper please. I do see 720 requests per minute. Does that imply that the size of audio should be small. That I should break it into smaller parts to fit a predetermined size? Any answer would be awesome. Thank you

EElektra Hello everyone. I'm currently testing Cloudflare Whisper, and I received an erro...

Raylight•8/4/24, 6:58 PM

There is an undocumented limit/bug/feature/something when it comes to file size. The limit is roughly 2.6 MB. Additionally, it has been mentioned here that whisper works best with audio < 30 seconds.

Elektra•8/5/24, 1:59 AM

Thank you @Raylight! This is very helpful.

1.1.1.1APP•8/24/24, 6:24 AM

DNS over Discord: GitHub

View the DNS over Discord source code on GitHub at https://dns-over-discord.v4.wtf/github

1.1.1.1APP•8/24/24, 6:28 AM

DNS over Discord: A records

lykcloud.me A @9.9.9.9 +noall +answerlykcloud.me A @9.9.9.9 +noall +answer

NAME        | TTL  | DATA          
------------+------+---------------
lykcloud.me | 300s | 104.21.89.11  
lykcloud.me | 300s | 172.67.155.128

NAME        | TTL  | DATA          
------------+------+---------------
lykcloud.me | 300s | 104.21.89.11  
lykcloud.me | 300s | 172.67.155.128

diggy diggy hole

•

8/24/24, 10:52 PM

1.1.1.1APP•8/24/24, 10:52 PM

DNS over Discord: A records

siteforgepro.nl A @1.1.1.1 +noall +answersiteforgepro.nl A @1.1.1.1 +noall +answer

NAME            | TTL  | DATA          
----------------+------+---------------
siteforgepro.nl | 282s | 172.67.144.149
siteforgepro.nl | 282s | 104.21.28.64

NAME            | TTL  | DATA          
----------------+------+---------------
siteforgepro.nl | 282s | 172.67.144.149
siteforgepro.nl | 282s | 104.21.28.64

diggy diggy hole

•

8/24/24, 10:52 PM

1.1.1.1APP•8/24/24, 11:10 PM

DNS over Discord: A records

smp-central.com. A @8.8.8.8 +noall +answersmp-central.com. A @8.8.8.8 +noall +answer

No records found

diggy diggy hole

•

10/25/24, 9:17 PM

TK•8/27/24, 6:08 PM

Hey, my Cloudflare neurons 'daily usage' count doesn't seem to be resetting at 00:00UTC as in the docs. Is the overage limit for a paid account really 10,000 in that 24hr period, or is it 300k a month? Account id 240e340132a0949a7f970e9c2d0e1758240e340132a0949a7f970e9c2d0e1758

Aleksandar•8/31/24, 11:09 AM

Is there any place to watch out for when beta models go out of beta? particularly detr-resnet-50

Gyurmatag•9/1/24, 6:00 PM

Hi!
How can I upload images to AI models (for example Llama 3.1) from workers? I didn't find it in the documentation, here: https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct

gx•9/3/24, 6:32 PM

For only one account we have in cloudflare and all domians are working fine yesterday, today after enable the 2FA dashboard not showing domains . after that revoking 2FA but still same issue help ASAP...

Flare•9/3/24, 6:49 PM

Please do not post your question in multiple channels/post it multiple times per the rules at ⁠No Access. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

�

😈 Donkey 💫•9/10/24, 4:09 AM

@cf/meta/m2m100-1.2b can't event translate any long text (>300 or 400 characters), in response of cut-off translated result.

can't translate short-text (2-8 words), in response of meaningless or error-like translated result.

I don't understand why It's in general available charge, without any limitation docs

Jess•9/10/24, 5:27 AM

I would like to know if there is any advise for captcha error when trying to log in using citrix?

a4amaan•9/12/24, 6:17 PM

Its generating near about 300 words is there any limit of neurons for free users?

const answer = await ctx.env.AI.run('@cf/meta/llama-3-8b-instruct', {
        prompt: "Write 2000 words article",
        stream: true
    });

const answer = await ctx.env.AI.run('@cf/meta/llama-3-8b-instruct', {
        prompt: "Write 2000 words article",
        stream: true
    });

a4amaan•9/12/24, 6:34 PM

which AI model will be good to rewrite news articles?

net2income•9/12/24, 8:48 PM

Hello, I am trying to use Cloudflare OpenAi compatible API, but I am getting this error: "could not route to /accounts/xxxx/ai/v1". Note xxxx is my account ID

can anyone please help.
Thanks

Nnet2income Hello, I am trying to use Cloudflare OpenAi compatible API, but I am getting thi...

Isaac McFadyen•9/12/24, 9:02 PM

You need to add one of the OpenAI endpoints to the end of the URL.

Isaac McFadyen•9/12/24, 9:02 PM

For example, /v1/chat/completions/v1/chat/completions or /v1/completions/v1/completions

Artucuno•9/13/24, 12:57 AM

Hey! Is there a way to do real time transcription with Whisper?

AArtucuno Hey! Is there a way to do real time transcription with Whisper?

Isaac McFadyen•9/13/24, 1:50 AM

Not really, because Whisper isn't a streaming transcription model - it needs all of the content in advance. What you can do, and what most things that are "streaming" with Whisper do, is chunk the audio into smaller chunks and process each one with a slight overlap.

Isaac McFadyen•9/13/24, 1:50 AM

The accuracy won't be quite as good but it will probably be as close as you can get to 'streaming' with Whisper.

digendee•9/14/24, 10:26 AM

Hi all. We are having issues trying to access Google's Gemini API from a Cloudflare Pages Function / Worker.

In the worker code, we have tried accessing Google's Gemini API using both direct requests to:

https://generativelanguage.googleapis.com/...

and through the Cloudflare AI gateway:
https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_name}/google-ai-studio/...

Both methods above result in the following error being returned from the Google Gemini API:
{
"error": {
"code": 400,
"message": "User location is not supported for the API use.",
"status": "FAILED_PRECONDITION"
}
}

Upon investigation, this error is supposedly returned when the user is not in a supported location listed here:
https://ai.google.dev/gemini-api/docs/available-regions

However, we ARE in a supported location (Melbourne, Australia). When we access the Gemini API from our local dev env (even if we access it via the Cloudflare AI Gateway), the Gemini API works correctly.

Anyone know what is going on?

Google AI for Developers

Available regions for Google AI Studio and Gemini API | Google AI...

Ddigendee Hi all. We are having issues trying to access Google's Gemini API from a Cloudfl...

Isaac McFadyen•9/14/24, 1:52 PM

First, please don't crosspost - you posted in both cloudflare-ai and ai-gateway

Isaac McFadyen•9/14/24, 1:52 PM

Second, can you try going to https://cloudflare.com/cdn-cgi/trace and see what the

colo

colo

field is?

Isaac McFadyen•9/14/24, 1:53 PM

It's possible because of some network routing intricacies, you're being routed out of Australia and so Gemini is blocking that.

IIsaac McFadyen It's possible because of some network routing intricacies, you're being routed o...

digendee•9/15/24, 1:23 AM

Thank you. Yes I crossposted because I wasn't sure which channel to post the question in.

When visiting that trace URL you posted, my "colo" is MEL and "loc" is AU - both of which are allowed to access Gemini.

Any other ideas?

digendee•9/15/24, 1:26 AM

Actually, in the trace link you posted above, I get "MEL" as the colo, but with my Pages Function / worker, I get "HKG" as the colo.

digendee•9/15/24, 1:31 AM

Indeed, Hong Kong (HKG) is NOT in the allowed access to Google Gemini list. This would be the reason I am getting that Gemini error. Thanks again @Isaac McFadyen

Ddigendee Indeed, Hong Kong (HKG) is NOT in the allowed access to Google Gemini list. This...

Isaac McFadyen•9/15/24, 1:45 AM

No problem... unfortunately because of the way Cloudflare works there's no way to "force" your Worker to use MEL.

digendee•9/15/24, 1:46 AM

I am submitting a feature request now to Cloudflare to avoid putting workers in these types of locations, as this problem will only get worse as LLM API adoption grows

Isaac McFadyen•9/15/24, 1:46 AM

Going to ping the AI gateway PM here because this issue has come up before (cc @Kathy).

TLDR: Gemini isn't allowed to be access from Hong Kong but AI gateway is routing requests through HKG, causing them to be denied. I believe this came up with OpenAI before? Not sure what the resolution was or if you forced the routing to avoid certain countries internally?

digendee•9/15/24, 1:54 AM

Feature request is here in case you want to contribute / vote:

digendee•9/15/24, 1:54 AM

https://community.cloudflare.com/t/do-not-put-workers-in-llm-forbidden-locations/711754

Hi All. Is there any possibilities to host our own models on Cloudflare's AI?

Similar Threads

Similar Threads

Similar Threads