In case anyone is wondering since its not written anywhere, gateway does support responses enpoint u

K

KevinOP•8/20/25, 5:01 PM

The cost feature is automatic or do I have to send it? Because the cost is still 0 while using workers-ai provider.

Y

yinxingmaiming6409•8/21/25, 2:11 AM

How can AI be used in Cherry Studio? It seems impossible.

Yyinxingmaiming6409 How can AI be used in Cherry Studio? It seems impossible.

I

Isaac McFadyen•8/21/25, 2:37 AM

?crossposting Please don't post in multiple channels.

S

SuperHelpflare•8/21/25, 2:37 AM

Please do not post your question in multiple channels/post it multiple times per the rules at #

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

T

Tristan•8/21/25, 7:46 AM

hey, can someone from cloudflare can take a look at my message, it slipped through: ai-gateway

B

ben_makes_stuff•8/25/25, 2:57 AM

Any plans to make the OpenAI-compatible endpoint work with Amazon Bedrock? They're very lazy about their implementation and recommend customers use an AWS lambda function with custom code if using the OpenAI SDK, they don't provide an OpenAI-compatible endpoint like Google does

D

dave•8/26/25, 6:51 PM

Looks like pricing for Anthropic models with caching has been fixed?

H

hurrytospring•8/28/25, 5:34 AM

We are a startup tech company considering using Cloudflare AI Gateway. Before proceeding, we have the following questions:
It seems that there is no support for the DeepSeek V3 or Qwen 3 models. Is there any other way to access them? They are really cost - effective.
Is the AI Gateway ready for production use? What are the limits of queries per second (QPS) and tokens per minute (TPM)?
Compared to native providers like Vertex/ Bedrock, when making requests from Google Cloud Platform (GCP) or Amazon Web Services (AWS), how does the latency or token speed differ? By the way, how does the price compare?

W

wangzuo•8/28/25, 2:08 PM

Can we specify the AI gateway region? Some Cloudflare proxy servers in certain locations do not have access to specific AI providers.

A

ari•8/29/25, 3:17 AM

AI Gateway behavior appears to have changed this afternoon, and requests that were previously returning successfully after 3 minutes are now failing after 60 seconds. (I'm using AI Gateway for requests to a number of different vendor models, including Anthropic's Opus 4.1, where responses to complex requests can take 1-3 minutes to generate.) Cloudflare is reporting this as a 504, but when I looked at the corresponding log in Anthropic, they're reporting it as a 499 "client disconnected" after 60 seconds, suggesting that Cloudflare terminated the connection. This definitely breaks slow models like Opus. Has anyone else run into this today?

D

dave•8/29/25, 3:54 PM

https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/

Great job!!

Cloudflare Docs

BYOK (Store Keys)

Bring your own keys (BYOK) is a feature in Cloudflare AI Gateway that allows you to securely store your AI provider API keys directly in the Cloudflare dashboard. Instead of including API keys in every request to your AI models, you can configure them once in the dashboard, and reference them in your gateway configuration.

Aari AI Gateway behavior appears to have changed this afternoon, and requests that we...

A

ari•8/29/25, 11:04 PM

(Seems like this was a temporary issue. Longer calls are working again.)

Aari AI Gateway behavior appears to have changed this afternoon, and requests that we...

D

dave•8/30/25, 1:09 AM

Was this for streaming requests?

W

Ward•8/30/25, 8:33 PM

Hey all, is there an easy way to store my keys in the cloudflare dashboard and then call the unified api? When i do this I still have to provide the api key of my underlying provider. How did you fix this? Here is an example with cerebras

const client = new OpenAI({
            apiKey: apiKey,
            baseURL: 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat',
            defaultHeaders: {
                'cf-aig-authorization': `Bearer ${apiKey}`,
            },
        });

        const response = await client.chat.completions.create({
            model: 'cerebras/gpt-oss-120b',
            messages: [{ role: 'user', content: 'What is cloudflare?' }],
        });

const client = new OpenAI({
            apiKey: apiKey,
            baseURL: 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat',
            defaultHeaders: {
                'cf-aig-authorization': `Bearer ${apiKey}`,
            },
        });

        const response = await client.chat.completions.create({
            model: 'cerebras/gpt-oss-120b',
            messages: [{ role: 'user', content: 'What is cloudflare?' }],
        });

const client = new OpenAI({
            apiKey: apiKey,
            baseURL: 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat',
            defaultHeaders: {
                'cf-aig-authorization': `Bearer ${apiKey}`,
            },
        });

        const response = await client.chat.completions.create({
            model: 'cerebras/gpt-oss-120b',
            messages: [{ role: 'user', content: 'What is cloudflare?' }],
        });

const client = new OpenAI({
            apiKey: apiKey,
            baseURL: 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat',
            defaultHeaders: {
                'cf-aig-authorization': `Bearer ${apiKey}`,
            },
        });

        const response = await client.chat.completions.create({
            model: 'cerebras/gpt-oss-120b',
            messages: [{ role: 'user', content: 'What is cloudflare?' }],
        });

apiKey is the gateway key.
Or did I get it wrong that the unified api (openai compatible) doesn't yet work with the BYOK (Store keys) feature?
Would love to hear if anyone knows.

Ddave Was this for streaming requests?

A

ari•9/1/25, 12:05 AM

No. Just a basic call to

/anthropic/v1/messages

/anthropic/v1/messages

/anthropic/v1/messages

/anthropic/v1/messages

J

joe•9/1/25, 12:26 AM

am I misunderstanding how metadata filtering is supposed to work? if I attach say

asn

asn

asn

asn and

ip

ip

ip

ip to the

metadata

metadata

metadata

metadata key I cannot actually negatively filter them

J

joe•9/1/25, 12:27 AM

??

J

joe•9/1/25, 12:27 AM

i don't know which role is for moderators here but spamming in a load of channels isn't a good look

J

joe•9/1/25, 12:28 AM

if I do

Metadata Value

Metadata Value

Metadata Value

Metadata Value <does not equal>

Some IP

Some IP

Some IP

Some IP then it negatively matches all the ASNs because they do not have the value of the IP

J

joe•9/1/25, 12:29 AM

@andrew | as1003 hey winner it's your old pal joe

J

joe•9/1/25, 12:29 AM

can you advise on reporting spammers here lol

J

joe•9/1/25, 12:29 AM

user id: 521883426496970762 @Subscription (just deleted their messages)

Aari No. Just a basic call to `/anthropic/v1/messages`

D

dave•9/1/25, 12:30 AM

That could be either streaming or not steaming. Depends if you used stream: true or not

Jjoe @andrew | as1003 hey winner it's your old pal joe

A

andrew_nyr•9/1/25, 12:37 AM

hey pal, ill take a look

H

harry•9/2/25, 4:20 AM

Is the Groq provider down? As I request Groq via AI Gateway, I got:

{
  "error": {
    "message": "Not Found"
  }
}

{
  "error": {
    "message": "Not Found"
  }
}

{
  "error": {
    "message": "Not Found"
  }
}

{
  "error": {
    "message": "Not Found"
  }
}

For the Groq provider, I've tried

chat/completions

chat/completions

chat/completions

chat/completions,

responses

responses

responses

responses and configured the URL endpoint using this pattern https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/groq , as well as the OpenAI compat approach but both don't work.

FYI, OpenAI provider via AI Gateway and direct call to Groq endpoint work normally.

S

Seasenbkr56•9/2/25, 5:07 AM

https://cdjsjxc.com/#/recordlist

E

Egor•9/2/25, 7:06 AM

Hello, I started using AI Gateway and I'm randomly getting:

"message": "403 Country, region, or territory not supported",
"name": "Error",
"stack": "Error: 403 Country, region, or territory not supported\n"

I have a Workers Paid plan and it happens to users in Asia. Is there a way to fix it? I'd expect AI Gateway to not run in forbidden locations. I don't remember having this issue before I switched to AI Gateway

EEgor Hello, I started using AI Gateway and I'm randomly getting: "message": "403 Cou...

H

harry•9/2/25, 9:12 AM

Second this. I also tried to deploy Workers using smart placement but the issue persists.

G

gianluca.henze•9/2/25, 12:44 PM

Hi there,
I am running into weird issues where out of nowhere the latency of the Autorag spikes to up to 30 seconds. From the logs to me it seems, that it's not the finding process or the LLM, but something else. I would highly apreciate help, if I can fix this as this problem never used to happen or if this is a server side problem today. Thanks!

I

idham•9/2/25, 4:12 PM

Is it possible to use custom openai compatible API for provider? The provider we use is not available in the list

B

brylan•9/3/25, 12:48 PM

Is function calling possible on the new GPT oss?

K

Kepro•9/3/25, 2:16 PM

Hello, SRE team gave me access to our domains and they gave me access to
AI Gateway - Read
AI Gateway - Edit
but I'm still getting 403 "Authentication error" from the UI and also from CLI, any idea what we are missing? When they tried to give me access to workers that started working right away...

B

brylan•9/4/25, 12:04 AM

Gateway error: {"errors":[{"message":"AiError: AiError: 'NoneType' object has no attribute 'startswith' (d9c72023-0abe-41ce-982f-197dbd233576)","code":3030}],"success":false,"result":{},"messages":[]}

R

RU;5JI•9/5/25, 1:59 AM

https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/
in Cloudflare AI Gateway's WebSocket implementation where BYOK (Bring Your Own Key) authentication doesn't work properly

https://github.com/jiangzhuo/cloudflare-ai-gateway-realtime-poc

Cloudflare Docs

BYOK (Store Keys)

Bring your own keys (BYOK) is a feature in Cloudflare AI Gateway that allows you to securely store your AI provider API keys directly in the Cloudflare dashboard. Instead of including API keys in every request to your AI models, you can configure them once in the dashboard, and reference them in your gateway configuration.

GitHub

GitHub - jiangzhuo/cloudflare-ai-gateway-realtime-poc

Contribute to jiangzhuo/cloudflare-ai-gateway-realtime-poc development by creating an account on GitHub.

RRU;5JI https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/ ...

M

Mirza•9/5/25, 4:45 AM

Unauthorised 401 is coming for me, direct provider working but not through Gateway.....

Did anyone get this working ? Let me know what I might be doing wrong...

rn_image_picker_lib_temp_89382b74-c6c0-4e50-9552-116cbfb5e0ca.jpg

rn_image_picker_lib_temp_d0ac9692-37b0-46a7-bb97-17d9fa66f6c4.jpg

O

ozan•9/5/25, 1:48 PM

How can I use Vercel AI SDK with BYOK? In my createOpenAI call if I don't pass an apiKey parameter, I get an error about env.OPENAI_API_KEY missing. If I pass empty string, AI Gateway tries to use this as the API key and it it fails with "Incorrect API key provided"

S

SmallShen•9/5/25, 7:58 PM

Is this intended? streaming reqeusts input output does not get counted?
Even tho the data is available

Z

Zack•9/8/25, 8:45 PM

I've been really struggling to get the ai gateway to work, I can't seem to get an api token that has the correct permissions.

curl https://gateway.ai.cloudflare.com/v1/{redacted}/{redacted}/workers-ai/@cf/huggingface/distilbert-sst-2-int8 \
 --header 'Authorization: Bearer {redacted}' \
 --header 'Content-Type: application/json' \
 --data '{ "text": "Cloudflare docs are amazing!" }'

curl https://gateway.ai.cloudflare.com/v1/{redacted}/{redacted}/workers-ai/@cf/huggingface/distilbert-sst-2-int8 \
 --header 'Authorization: Bearer {redacted}' \
 --header 'Content-Type: application/json' \
 --data '{ "text": "Cloudflare docs are amazing!" }'

curl https://gateway.ai.cloudflare.com/v1/{redacted}/{redacted}/workers-ai/@cf/huggingface/distilbert-sst-2-int8 \
 --header 'Authorization: Bearer {redacted}' \
 --header 'Content-Type: application/json' \
 --data '{ "text": "Cloudflare docs are amazing!" }'

curl https://gateway.ai.cloudflare.com/v1/{redacted}/{redacted}/workers-ai/@cf/huggingface/distilbert-sst-2-int8 \
 --header 'Authorization: Bearer {redacted}' \
 --header 'Content-Type: application/json' \
 --data '{ "text": "Cloudflare docs are amazing!" }'

gives me an error

{"success":false,"result":[],"messages":[],"error":[{"code":2009,"message":"Unauthorized"}]}

{"success":false,"result":[],"messages":[],"error":[{"code":2009,"message":"Unauthorized"}]}

{"success":false,"result":[],"messages":[],"error":[{"code":2009,"message":"Unauthorized"}]}

{"success":false,"result":[],"messages":[],"error":[{"code":2009,"message":"Unauthorized"}]}

Y

yureckey•9/9/25, 4:50 AM

I'm trying to use guardrails in ai-gateway.
It's not quite working as expected. I'm running image generation with test NSFW prompt. In the ai-gateway log I see that llamaguard is passed (green shield icon) although it clearly states that it is not safe (I've set config to block on S12)
{
"response": {
"safe": false,
"categories": [
"S12"
]
}
}

Hharshil1712 forwarding this to the team.

C

cosbgn•9/9/25, 8:48 AM

Is there any update on that? It would be great if we could blacklist some regions like HK and Taiwan. I have some users in Japan getting router to these regions and blocked by openAI

C

Comrade Nalaxone•9/9/25, 10:53 PM

Why the hell do you all have like 20 localized pages of this blog and not a single english one? https://blog.cloudflare.com/pl-pl/fedramphigh-ai/

C

Comrade Nalaxone•9/9/25, 10:53 PM

That's the weirdest thing I've ever seen. Is AI doing this?

C

Comrade Nalaxone•9/9/25, 10:54 PM

@kian | DevDocs ???

CComrade Nalaxone Why the hell do you all have like 20 localized pages of this blog and not a sing...

I

Isaac McFadyen•9/9/25, 11:38 PM

That is because I believe the blog was pulled, so you were seeing a cached version that is no longer there.

IIsaac McFadyen That is because I believe the blog was pulled, so you were seeing a cached versi...

C

Comrade Nalaxone•9/9/25, 11:57 PM

Makes sense. Thanks for updating me. What's the status on FedRamp + Cloudflare AI?

C

Comrade Nalaxone•9/9/25, 11:57 PM

Is it still something that's happening?

D

dave•9/11/25, 2:45 AM

FYI there is a bug in cost calculation where it's set to zero if input tokens is zero

D

dave•9/11/25, 2:45 AM

Legit example from anthropic:

"usage": {
    "input_tokens": 0,
    "cache_creation_input_tokens": 575,
    "cache_read_input_tokens": 106910,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 575,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 957,
    "service_tier": "standard"
  }

"usage": {
    "input_tokens": 0,
    "cache_creation_input_tokens": 575,
    "cache_read_input_tokens": 106910,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 575,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 957,
    "service_tier": "standard"
  }

"usage": {
    "input_tokens": 0,
    "cache_creation_input_tokens": 575,
    "cache_read_input_tokens": 106910,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 575,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 957,
    "service_tier": "standard"
  }

"usage": {
    "input_tokens": 0,
    "cache_creation_input_tokens": 575,
    "cache_read_input_tokens": 106910,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 575,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 957,
    "service_tier": "standard"
  }

D

dave•9/11/25, 12:25 PM

It would be nice if the binding allowed use to provider endpoints instead of just the universial one, since otherwise we have to manually provide an auth token for AI Gateway.

https://developers.cloudflare.com/ai-gateway/configuration/authentication/#expected-behavior

Jjoe am I misunderstanding how metadata filtering is supposed to work? if I attach sa...

J

joe•9/11/25, 5:03 PM

just echoing this? it completely negates the point of being able to filter by metadata keys

In case anyone is wondering since its not written anywhere, gateway does support responses enpoint u

Similar Threads

Similar Threads

Similar Threads