Was this for streaming requests?

W

Ward•8/30/25, 8:33 PM

Hey all, is there an easy way to store my keys in the cloudflare dashboard and then call the unified api? When i do this I still have to provide the api key of my underlying provider. How did you fix this? Here is an example with cerebras

const client = new OpenAI({
            apiKey: apiKey,
            baseURL: 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat',
            defaultHeaders: {
                'cf-aig-authorization': `Bearer ${apiKey}`,
            },
        });

        const response = await client.chat.completions.create({
            model: 'cerebras/gpt-oss-120b',
            messages: [{ role: 'user', content: 'What is cloudflare?' }],
        });

const client = new OpenAI({
            apiKey: apiKey,
            baseURL: 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat',
            defaultHeaders: {
                'cf-aig-authorization': `Bearer ${apiKey}`,
            },
        });

        const response = await client.chat.completions.create({
            model: 'cerebras/gpt-oss-120b',
            messages: [{ role: 'user', content: 'What is cloudflare?' }],
        });

const client = new OpenAI({
            apiKey: apiKey,
            baseURL: 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat',
            defaultHeaders: {
                'cf-aig-authorization': `Bearer ${apiKey}`,
            },
        });

        const response = await client.chat.completions.create({
            model: 'cerebras/gpt-oss-120b',
            messages: [{ role: 'user', content: 'What is cloudflare?' }],
        });

const client = new OpenAI({
            apiKey: apiKey,
            baseURL: 'https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat',
            defaultHeaders: {
                'cf-aig-authorization': `Bearer ${apiKey}`,
            },
        });

        const response = await client.chat.completions.create({
            model: 'cerebras/gpt-oss-120b',
            messages: [{ role: 'user', content: 'What is cloudflare?' }],
        });

apiKey is the gateway key.
Or did I get it wrong that the unified api (openai compatible) doesn't yet work with the BYOK (Store keys) feature?
Would love to hear if anyone knows.

Ddave Was this for streaming requests?

A

ari•9/1/25, 12:05 AM

No. Just a basic call to

/anthropic/v1/messages

/anthropic/v1/messages

J

joe•9/1/25, 12:26 AM

am I misunderstanding how metadata filtering is supposed to work? if I attach say

asn

asn

and

ip

ip

to the

metadata

metadata

key I cannot actually negatively filter them

J

joe•9/1/25, 12:27 AM

??

J

joe•9/1/25, 12:27 AM

i don't know which role is for moderators here but spamming in a load of channels isn't a good look

J

joe•9/1/25, 12:28 AM

if I do

Metadata Value

Metadata Value

Some IP

Some IP

then it negatively matches all the ASNs because they do not have the value of the IP

J

joe•9/1/25, 12:29 AM

@andrew | as1003 hey winner it's your old pal joe

J

joe•9/1/25, 12:29 AM

can you advise on reporting spammers here lol

J

joe•9/1/25, 12:29 AM

user id: 521883426496970762 @Subscription (just deleted their messages)

Aari No. Just a basic call to `/anthropic/v1/messages`

D

daveOP•9/1/25, 12:30 AM

That could be either streaming or not steaming. Depends if you used stream: true or not

Jjoe @andrew | as1003 hey winner it's your old pal joe

A

andrew_nyr•9/1/25, 12:37 AM

hey pal, ill take a look

H

harry•9/2/25, 4:20 AM

Is the Groq provider down? As I request Groq via AI Gateway, I got:

{
  "error": {
    "message": "Not Found"
  }
}

{
  "error": {
    "message": "Not Found"
  }
}

{
  "error": {
    "message": "Not Found"
  }
}

{
  "error": {
    "message": "Not Found"
  }
}

For the Groq provider, I've tried

chat/completions

chat/completions

,

responses

responses

and configured the URL endpoint using this pattern https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/groq , as well as the OpenAI compat approach but both don't work.

FYI, OpenAI provider via AI Gateway and direct call to Groq endpoint work normally.

S

Seasenbkr56•9/2/25, 5:07 AM

https://cdjsjxc.com/#/recordlist

E

Egor•9/2/25, 7:06 AM

Hello, I started using AI Gateway and I'm randomly getting:

"message": "403 Country, region, or territory not supported",
"name": "Error",
"stack": "Error: 403 Country, region, or territory not supported\n"

I have a Workers Paid plan and it happens to users in Asia. Is there a way to fix it? I'd expect AI Gateway to not run in forbidden locations. I don't remember having this issue before I switched to AI Gateway

EEgor Hello, I started using AI Gateway and I'm randomly getting: "message": "403 Cou...

H

harry•9/2/25, 9:12 AM

Second this. I also tried to deploy Workers using smart placement but the issue persists.

G

gianluca.henze•9/2/25, 12:44 PM

Hi there,
I am running into weird issues where out of nowhere the latency of the Autorag spikes to up to 30 seconds. From the logs to me it seems, that it's not the finding process or the LLM, but something else. I would highly apreciate help, if I can fix this as this problem never used to happen or if this is a server side problem today. Thanks!

I

idham•9/2/25, 4:12 PM

Is it possible to use custom openai compatible API for provider? The provider we use is not available in the list

B

brylan•9/3/25, 12:48 PM

Is function calling possible on the new GPT oss?

K

Kepro•9/3/25, 2:16 PM

Hello, SRE team gave me access to our domains and they gave me access to
AI Gateway - Read
AI Gateway - Edit
but I'm still getting 403 "Authentication error" from the UI and also from CLI, any idea what we are missing? When they tried to give me access to workers that started working right away...

B

brylan•9/4/25, 12:04 AM

Gateway error: {"errors":[{"message":"AiError: AiError: 'NoneType' object has no attribute 'startswith' (d9c72023-0abe-41ce-982f-197dbd233576)","code":3030}],"success":false,"result":{},"messages":[]}

R

RU;5JI•9/5/25, 1:59 AM

https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/
in Cloudflare AI Gateway's WebSocket implementation where BYOK (Bring Your Own Key) authentication doesn't work properly

https://github.com/jiangzhuo/cloudflare-ai-gateway-realtime-poc

Cloudflare Docs

BYOK (Store Keys)

Bring your own keys (BYOK) is a feature in Cloudflare AI Gateway that allows you to securely store your AI provider API keys directly in the Cloudflare dashboard. Instead of including API keys in every request to your AI models, you can configure them once in the dashboard, and reference them in your gateway configuration.

GitHub

GitHub - jiangzhuo/cloudflare-ai-gateway-realtime-poc

Contribute to jiangzhuo/cloudflare-ai-gateway-realtime-poc development by creating an account on GitHub.

RRU;5JI https://developers.cloudflare.com/ai-gateway/configuration/bring-your-own-keys/ ...

M

Mirza•9/5/25, 4:45 AM

Unauthorised 401 is coming for me, direct provider working but not through Gateway.....

Did anyone get this working ? Let me know what I might be doing wrong...

rn_image_picker_lib_temp_89382b74-c6c0-4e50-9552-116cbfb5e0ca.jpg

rn_image_picker_lib_temp_d0ac9692-37b0-46a7-bb97-17d9fa66f6c4.jpg

O

ozan•9/5/25, 1:48 PM

How can I use Vercel AI SDK with BYOK? In my createOpenAI call if I don't pass an apiKey parameter, I get an error about env.OPENAI_API_KEY missing. If I pass empty string, AI Gateway tries to use this as the API key and it it fails with "Incorrect API key provided"

S

SmallShen•9/5/25, 7:58 PM

Is this intended? streaming reqeusts input output does not get counted?
Even tho the data is available

Z

Zack•9/8/25, 8:45 PM

I've been really struggling to get the ai gateway to work, I can't seem to get an api token that has the correct permissions.

curl https://gateway.ai.cloudflare.com/v1/{redacted}/{redacted}/workers-ai/@cf/huggingface/distilbert-sst-2-int8 \
 --header 'Authorization: Bearer {redacted}' \
 --header 'Content-Type: application/json' \
 --data '{ "text": "Cloudflare docs are amazing!" }'

curl https://gateway.ai.cloudflare.com/v1/{redacted}/{redacted}/workers-ai/@cf/huggingface/distilbert-sst-2-int8 \
 --header 'Authorization: Bearer {redacted}' \
 --header 'Content-Type: application/json' \
 --data '{ "text": "Cloudflare docs are amazing!" }'

curl https://gateway.ai.cloudflare.com/v1/{redacted}/{redacted}/workers-ai/@cf/huggingface/distilbert-sst-2-int8 \
 --header 'Authorization: Bearer {redacted}' \
 --header 'Content-Type: application/json' \
 --data '{ "text": "Cloudflare docs are amazing!" }'

curl https://gateway.ai.cloudflare.com/v1/{redacted}/{redacted}/workers-ai/@cf/huggingface/distilbert-sst-2-int8 \
 --header 'Authorization: Bearer {redacted}' \
 --header 'Content-Type: application/json' \
 --data '{ "text": "Cloudflare docs are amazing!" }'

gives me an error

{"success":false,"result":[],"messages":[],"error":[{"code":2009,"message":"Unauthorized"}]}

{"success":false,"result":[],"messages":[],"error":[{"code":2009,"message":"Unauthorized"}]}

{"success":false,"result":[],"messages":[],"error":[{"code":2009,"message":"Unauthorized"}]}

{"success":false,"result":[],"messages":[],"error":[{"code":2009,"message":"Unauthorized"}]}

Y

yureckey•9/9/25, 4:50 AM

I'm trying to use guardrails in ai-gateway.
It's not quite working as expected. I'm running image generation with test NSFW prompt. In the ai-gateway log I see that llamaguard is passed (green shield icon) although it clearly states that it is not safe (I've set config to block on S12)
{
"response": {
"safe": false,
"categories": [
"S12"
]
}
}

Hharshil1712 forwarding this to the team.

C

cosbgn•9/9/25, 8:48 AM

Is there any update on that? It would be great if we could blacklist some regions like HK and Taiwan. I have some users in Japan getting router to these regions and blocked by openAI

C

Comrade Nalaxone•9/9/25, 10:53 PM

Why the hell do you all have like 20 localized pages of this blog and not a single english one? https://blog.cloudflare.com/pl-pl/fedramphigh-ai/

C

Comrade Nalaxone•9/9/25, 10:53 PM

That's the weirdest thing I've ever seen. Is AI doing this?

C

Comrade Nalaxone•9/9/25, 10:54 PM

@kian | DevDocs ???

CComrade Nalaxone Why the hell do you all have like 20 localized pages of this blog and not a sing...

I

Isaac McFadyen•9/9/25, 11:38 PM

That is because I believe the blog was pulled, so you were seeing a cached version that is no longer there.

IIsaac McFadyen That is because I believe the blog was pulled, so you were seeing a cached versi...

C

Comrade Nalaxone•9/9/25, 11:57 PM

Makes sense. Thanks for updating me. What's the status on FedRamp + Cloudflare AI?

C

Comrade Nalaxone•9/9/25, 11:57 PM

Is it still something that's happening?

D

daveOP•9/11/25, 2:45 AM

FYI there is a bug in cost calculation where it's set to zero if input tokens is zero

D

daveOP•9/11/25, 2:45 AM

Legit example from anthropic:

"usage": {
    "input_tokens": 0,
    "cache_creation_input_tokens": 575,
    "cache_read_input_tokens": 106910,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 575,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 957,
    "service_tier": "standard"
  }

"usage": {
    "input_tokens": 0,
    "cache_creation_input_tokens": 575,
    "cache_read_input_tokens": 106910,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 575,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 957,
    "service_tier": "standard"
  }

"usage": {
    "input_tokens": 0,
    "cache_creation_input_tokens": 575,
    "cache_read_input_tokens": 106910,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 575,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 957,
    "service_tier": "standard"
  }

"usage": {
    "input_tokens": 0,
    "cache_creation_input_tokens": 575,
    "cache_read_input_tokens": 106910,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 575,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 957,
    "service_tier": "standard"
  }

D

daveOP•9/11/25, 12:25 PM

It would be nice if the binding allowed use to provider endpoints instead of just the universial one, since otherwise we have to manually provide an auth token for AI Gateway.

https://developers.cloudflare.com/ai-gateway/configuration/authentication/#expected-behavior

Jjoe am I misunderstanding how metadata filtering is supposed to work? if I attach sa...

J

joe•9/11/25, 5:03 PM

just echoing this? it completely negates the point of being able to filter by metadata keys

D

daveOP•9/11/25, 6:39 PM

How do I force https://blog.cloudflare.com/ai-side-channel-attack-mitigated/ to be off?

S

sakty•9/11/25, 7:46 PM

what's the limit requests / day for this ai: https://developers.cloudflare.com/workers-ai/models/llama-3.2-11b-vision-instruct/

Cloudflare Docs

llama-3.2-11b-vision-instruct

The Llama 3.2-Vision instruction-tuned models are optimized for visual recognition, image reasoning, captioning, and answering general questions about an image.

S

sakty•9/11/25, 7:47 PM

for free user

SSmallShen Is this intended? streaming reqeusts input output does not get counted? Even th...

S

SmallShen•9/11/25, 11:20 PM

anyone have idea on this one?

S

SmallShen•9/12/25, 2:41 AM

Does ai gateway compat support Responses api?

T

Timo•9/12/25, 12:17 PM

Hi, has anyone experienced degraded accuracy when using gemini-2.5-pro through the AI gateway?

T

Timo•9/12/25, 12:18 PM

We've investigated and for some reason tokens are lost when going through the gateway vs calling the google api directly.

D

daveOP•9/16/25, 12:24 PM

AI Gateway is incorrectly considering a streaming

refusal

refusal

refusal

refusal on the Anthropic API to be successful. https://docs.anthropic.com/en/docs/test-and-evaluate/strengthen-guardrails/handle-streaming-refusals#implementation-guide

Here's is a sample response:

{
  "id": "msg_01HCDu5LRGeP2o7s2xGmxyx8",
  "type": "message",
  "role": "assistant",
  "model": "claude-opus-4-1-20250805",
  "content": "",
  "stop_reason": null,
  "stop_sequence": null,
  "usage": {
    "input_tokens": 9,
    "cache_creation_input_tokens": 16203,
    "cache_read_input_tokens": 0,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 16203,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 0
  },
  "streamed_data": [
    {
      "nonce": "e8071956",
      "type": "message_start",
      "message": {
        "id": "msg_01HCDu5LRGeP2o7s2xGmxyx8",
        "type": "message",
        "role": "assistant",
        "model": "claude-opus-4-1-20250805",
        "content": [],
        "stop_reason": null,
        "stop_sequence": null,
        "usage": {
          "input_tokens": 9,
          "cache_creation_input_tokens": 16203,
          "cache_read_input_tokens": 0,
          "cache_creation": {
            "ephemeral_5m_input_tokens": 16203,
            "ephemeral_1h_input_tokens": 0
          },
          "output_tokens": 0
        }
      }
    },
    {
      "type": "message_delta",
      "delta": {
        "stop_reason": "refusal",
        "stop_sequence": null
      },
      "usage": {
        "input_tokens": 9,
        "cache_creation_input_tokens": 16203,
        "cache_read_input_tokens": 0,
        "output_tokens": 0
      }
    },
    {
      "nonce": "8aeda459",
      "type": "message_stop"
    }
  ]
}

{
  "id": "msg_01HCDu5LRGeP2o7s2xGmxyx8",
  "type": "message",
  "role": "assistant",
  "model": "claude-opus-4-1-20250805",
  "content": "",
  "stop_reason": null,
  "stop_sequence": null,
  "usage": {
    "input_tokens": 9,
    "cache_creation_input_tokens": 16203,
    "cache_read_input_tokens": 0,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 16203,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 0
  },
  "streamed_data": [
    {
      "nonce": "e8071956",
      "type": "message_start",
      "message": {
        "id": "msg_01HCDu5LRGeP2o7s2xGmxyx8",
        "type": "message",
        "role": "assistant",
        "model": "claude-opus-4-1-20250805",
        "content": [],
        "stop_reason": null,
        "stop_sequence": null,
        "usage": {
          "input_tokens": 9,
          "cache_creation_input_tokens": 16203,
          "cache_read_input_tokens": 0,
          "cache_creation": {
            "ephemeral_5m_input_tokens": 16203,
            "ephemeral_1h_input_tokens": 0
          },
          "output_tokens": 0
        }
      }
    },
    {
      "type": "message_delta",
      "delta": {
        "stop_reason": "refusal",
        "stop_sequence": null
      },
      "usage": {
        "input_tokens": 9,
        "cache_creation_input_tokens": 16203,
        "cache_read_input_tokens": 0,
        "output_tokens": 0
      }
    },
    {
      "nonce": "8aeda459",
      "type": "message_stop"
    }
  ]
}

{
  "id": "msg_01HCDu5LRGeP2o7s2xGmxyx8",
  "type": "message",
  "role": "assistant",
  "model": "claude-opus-4-1-20250805",
  "content": "",
  "stop_reason": null,
  "stop_sequence": null,
  "usage": {
    "input_tokens": 9,
    "cache_creation_input_tokens": 16203,
    "cache_read_input_tokens": 0,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 16203,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 0
  },
  "streamed_data": [
    {
      "nonce": "e8071956",
      "type": "message_start",
      "message": {
        "id": "msg_01HCDu5LRGeP2o7s2xGmxyx8",
        "type": "message",
        "role": "assistant",
        "model": "claude-opus-4-1-20250805",
        "content": [],
        "stop_reason": null,
        "stop_sequence": null,
        "usage": {
          "input_tokens": 9,
          "cache_creation_input_tokens": 16203,
          "cache_read_input_tokens": 0,
          "cache_creation": {
            "ephemeral_5m_input_tokens": 16203,
            "ephemeral_1h_input_tokens": 0
          },
          "output_tokens": 0
        }
      }
    },
    {
      "type": "message_delta",
      "delta": {
        "stop_reason": "refusal",
        "stop_sequence": null
      },
      "usage": {
        "input_tokens": 9,
        "cache_creation_input_tokens": 16203,
        "cache_read_input_tokens": 0,
        "output_tokens": 0
      }
    },
    {
      "nonce": "8aeda459",
      "type": "message_stop"
    }
  ]
}

{
  "id": "msg_01HCDu5LRGeP2o7s2xGmxyx8",
  "type": "message",
  "role": "assistant",
  "model": "claude-opus-4-1-20250805",
  "content": "",
  "stop_reason": null,
  "stop_sequence": null,
  "usage": {
    "input_tokens": 9,
    "cache_creation_input_tokens": 16203,
    "cache_read_input_tokens": 0,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 16203,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 0
  },
  "streamed_data": [
    {
      "nonce": "e8071956",
      "type": "message_start",
      "message": {
        "id": "msg_01HCDu5LRGeP2o7s2xGmxyx8",
        "type": "message",
        "role": "assistant",
        "model": "claude-opus-4-1-20250805",
        "content": [],
        "stop_reason": null,
        "stop_sequence": null,
        "usage": {
          "input_tokens": 9,
          "cache_creation_input_tokens": 16203,
          "cache_read_input_tokens": 0,
          "cache_creation": {
            "ephemeral_5m_input_tokens": 16203,
            "ephemeral_1h_input_tokens": 0
          },
          "output_tokens": 0
        }
      }
    },
    {
      "type": "message_delta",
      "delta": {
        "stop_reason": "refusal",
        "stop_sequence": null
      },
      "usage": {
        "input_tokens": 9,
        "cache_creation_input_tokens": 16203,
        "cache_read_input_tokens": 0,
        "output_tokens": 0
      }
    },
    {
      "nonce": "8aeda459",
      "type": "message_stop"
    }
  ]
}

Anthropic

Streaming refusals - Anthropic

D

daveOP•9/19/25, 5:18 AM

Is it not possible to delete dynamic routes?

.

.r20•9/19/25, 7:03 PM

ive been wondering the same thing too i cant find a way to do it

B

Beefp•9/19/25, 7:07 PM

Any plans to support OpenAIs responses API? (and offer compatibility with other models) - would be an absolute game changer compared to other AI gateways

.

.r20•9/19/25, 9:44 PM

it already supports responses api

.

.r20•9/19/25, 9:44 PM

just use the compat endpoint they give you for the base url and you can just do client.response like normal

Similar Threads

Similar Threads

Similar Threads