Did you figure this out? I'm seeing the same thing with Anthropic. My workaround are to either inclu

Eengnadeau **Does Cloudflare AI Gateway BYOK actually support the OpenAI SDK?** I’m testin...

K

KrisOP•10/18/25, 5:02 PM

Saw @ozan's post first.

Trying to figure out the same thing.

KKris Did you figure this out? I'm seeing the same thing with Anthropic. My workaround...

O

ozan•10/19/25, 7:50 AM

No, I could not integrate secrets store with ai sdk

S

sodepr•10/19/25, 11:45 PM

Hi everyone,

Does AI gateway support streaming?

https://community.cloudflare.com/t/streaming-responses-does-not-work-with-ai-gateway-google-ai-studio/837798

A

Arman•10/20/25, 10:39 AM

Hi

K

koalaspider•10/20/25, 1:15 PM

Both my dev and deploy version for AI worker used to work perfectly fine. But last few days, the deploy version Ai worker stop working. Have been trying troubleshoot for the last two days. The dev version still working fine. Anyone know why?

N

NtTestAlert•10/21/25, 4:50 AM

Hi, I'm trying to use AI Gateway via websocket (I am aware that it is in beta, but working around max concurrent connections otherwise is a nightmare.
It works for success responses, but for error responses it does not include the metadata (or meta doesnt contain eventId) - impossible to correlate which req the error happened in

Happy path (has eventId in metadata):

Sending pending {"type":"universal.create","eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","request":{"eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","headers":{"Authorization":"Bearer sk-or-v1-*****"},"provider":"openrouter","endpoint":"v1/chat/completions","query":{"model":"google/gemini-2.5-flash","messages":[{"role":"system","content":"Tell me a joke."}]}}}
Received "{\"type\":\"universal.created\",\"metadata\":{\"cacheStatus\":\"MISS\",\"eventId\":\"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3\",\"logId\":\"01K82FV50QQGRQWGDT*****\",\"step\":\"0\",\"contentType\":\"application/json\"},\"response\":{\"id\":\"gen-1761020318-TQij*****eHDCaT\",\"provider\":\"Google\",\"model\":\"google/gemini-2.5-flash\",\"object\":\"chat.completion\",\"created\":1761020318,\"choices\":[{\"logprobs\":null,\"finish_reason\":\"stop\",\"native_finish_reason\":\"STOP\",\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"A three-legged dog walks into a saloon and says, \\\"I'm looking for the man who shot my paw.\\\"\",\"refusal\":null,\"reasoning\":null}}],\"usage\":{\"prompt_tokens\":5,\"completion_tokens\":25,\"total_tokens\":30,\"prompt_tokens_details\":{\"cached_tokens\":0},\"completion_tokens_details\":{\"reasoning_tokens\":0,\"image_tokens\":0}}}}"

Sending pending {"type":"universal.create","eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","request":{"eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","headers":{"Authorization":"Bearer sk-or-v1-*****"},"provider":"openrouter","endpoint":"v1/chat/completions","query":{"model":"google/gemini-2.5-flash","messages":[{"role":"system","content":"Tell me a joke."}]}}}
Received "{\"type\":\"universal.created\",\"metadata\":{\"cacheStatus\":\"MISS\",\"eventId\":\"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3\",\"logId\":\"01K82FV50QQGRQWGDT*****\",\"step\":\"0\",\"contentType\":\"application/json\"},\"response\":{\"id\":\"gen-1761020318-TQij*****eHDCaT\",\"provider\":\"Google\",\"model\":\"google/gemini-2.5-flash\",\"object\":\"chat.completion\",\"created\":1761020318,\"choices\":[{\"logprobs\":null,\"finish_reason\":\"stop\",\"native_finish_reason\":\"STOP\",\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"A three-legged dog walks into a saloon and says, \\\"I'm looking for the man who shot my paw.\\\"\",\"refusal\":null,\"reasoning\":null}}],\"usage\":{\"prompt_tokens\":5,\"completion_tokens\":25,\"total_tokens\":30,\"prompt_tokens_details\":{\"cached_tokens\":0},\"completion_tokens_details\":{\"reasoning_tokens\":0,\"image_tokens\":0}}}}"

Sending pending {"type":"universal.create","eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","request":{"eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","headers":{"Authorization":"Bearer sk-or-v1-*****"},"provider":"openrouter","endpoint":"v1/chat/completions","query":{"model":"google/gemini-2.5-flash","messages":[{"role":"system","content":"Tell me a joke."}]}}}
Received "{\"type\":\"universal.created\",\"metadata\":{\"cacheStatus\":\"MISS\",\"eventId\":\"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3\",\"logId\":\"01K82FV50QQGRQWGDT*****\",\"step\":\"0\",\"contentType\":\"application/json\"},\"response\":{\"id\":\"gen-1761020318-TQij*****eHDCaT\",\"provider\":\"Google\",\"model\":\"google/gemini-2.5-flash\",\"object\":\"chat.completion\",\"created\":1761020318,\"choices\":[{\"logprobs\":null,\"finish_reason\":\"stop\",\"native_finish_reason\":\"STOP\",\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"A three-legged dog walks into a saloon and says, \\\"I'm looking for the man who shot my paw.\\\"\",\"refusal\":null,\"reasoning\":null}}],\"usage\":{\"prompt_tokens\":5,\"completion_tokens\":25,\"total_tokens\":30,\"prompt_tokens_details\":{\"cached_tokens\":0},\"completion_tokens_details\":{\"reasoning_tokens\":0,\"image_tokens\":0}}}}"

Sending pending {"type":"universal.create","eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","request":{"eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","headers":{"Authorization":"Bearer sk-or-v1-*****"},"provider":"openrouter","endpoint":"v1/chat/completions","query":{"model":"google/gemini-2.5-flash","messages":[{"role":"system","content":"Tell me a joke."}]}}}
Received "{\"type\":\"universal.created\",\"metadata\":{\"cacheStatus\":\"MISS\",\"eventId\":\"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3\",\"logId\":\"01K82FV50QQGRQWGDT*****\",\"step\":\"0\",\"contentType\":\"application/json\"},\"response\":{\"id\":\"gen-1761020318-TQij*****eHDCaT\",\"provider\":\"Google\",\"model\":\"google/gemini-2.5-flash\",\"object\":\"chat.completion\",\"created\":1761020318,\"choices\":[{\"logprobs\":null,\"finish_reason\":\"stop\",\"native_finish_reason\":\"STOP\",\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"A three-legged dog walks into a saloon and says, \\\"I'm looking for the man who shot my paw.\\\"\",\"refusal\":null,\"reasoning\":null}}],\"usage\":{\"prompt_tokens\":5,\"completion_tokens\":25,\"total_tokens\":30,\"prompt_tokens_details\":{\"cached_tokens\":0},\"completion_tokens_details\":{\"reasoning_tokens\":0,\"image_tokens\":0}}}}"

N

NtTestAlert•10/21/25, 4:50 AM

bad path:

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"Input must have at least 1 token.\",\"code\":400,\"metadata\":{\"provider_name\":null}}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"Input must have at least 1 token.\",\"code\":400,\"metadata\":{\"provider_name\":null}}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"Input must have at least 1 token.\",\"code\":400,\"metadata\":{\"provider_name\":null}}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"Input must have at least 1 token.\",\"code\":400,\"metadata\":{\"provider_name\":null}}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"success\":false,\"result\":[],\"messages\":[],\"error\":[{\"code\":2008,\"message\":\"Invalid provider\"}]}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"success\":false,\"result\":[],\"messages\":[],\"error\":[{\"code\":2008,\"message\":\"Invalid provider\"}]}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"success\":false,\"result\":[],\"messages\":[],\"error\":[{\"code\":2008,\"message\":\"Invalid provider\"}]}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"success\":false,\"result\":[],\"messages\":[],\"error\":[{\"code\":2008,\"message\":\"Invalid provider\"}]}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"No auth credentials found\",\"code\":401}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"No auth credentials found\",\"code\":401}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"No auth credentials found\",\"code\":401}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"No auth credentials found\",\"code\":401}}}"

N

NtTestAlert•10/21/25, 4:50 AM

The logs tab in dashboard shows correct event-id for happy path ~~but some random generated one for non-happy it seems~~ (only one of them is wrong in logs but it might be my mistake) bad path also has proper eventId in dashboard, it is just not sent in the websocket error forwarding (also notcie provider_name is null ? .o0)

N

NtTestAlert•10/21/25, 4:54 AM

I mean... I'll just wait for timeout for errors for now I guess :/ but this really is a nightmare, tried some solution with background processed durable objects with internal queueing to bypass the 6 connection limit, but that's also buggy

N

NtTestAlert•10/21/25, 4:54 AM

this seemed like exactly what I wanted to implement myself but it looks like it doesnt work properly either :/

N

NtTestAlert•10/21/25, 5:30 AM

Since the logs contain the correct eventId I assume this is a trivial fix, and its been broken like this for a while...

EDIT: this problem is tracked at https://github.com/cloudflare/workers-sdk/issues/11040

A

adhami•10/21/25, 8:23 PM

using cohere rerank through the AI gateway breaks because it attempts to str

documents

documents

documents

documents in the json body

K

koalaspider•10/21/25, 10:14 PM

Have been working on a project to include AI gateway with a LLM over 1 week ago, both dev and deploy version has been working fine. Since 2 days ago, the AI functionality does not work on the deploy version but the dev version works fine. I rebuilt everything on the deploy version yesterday. The built process was completed and no error. But when it comes to wrangler deploy, it said "A request to the Cloudflare API failed. Authentication error [code: 10000]" I have set all the permission that I can think of for the build token. It seems that the npx wrangler deploy command has some issue. The dev version still working fine. Does anyone know why the deploy version failed? Is there a way to debug into the wrangler?

K

koalaspider•10/21/25, 10:20 PM

FYI I am actually connected to my github account and build the project without the CLI. I have tried to build and deploy the project using CLI but still the AI functionality failed. The dev version has been connected to the remote AI gateway and never have any issue at all. Is there a way solve this error?

K

koalaspider•10/22/25, 3:05 AM

Finally solve the wrangler deployment issues. but the authentication error happened.... OMG.

ZZack I've been really struggling to get the ai gateway to work, I can't seem to get a...

K

koalaspider•10/22/25, 3:10 AM

Hi, @Zack, I am having the same issue... Have you managed to solve the bug/error here?

Kkoalaspider Hi, @Zack, I am having the same issue... Have you managed to solve the bug/error...

Z

Zack•10/22/25, 3:38 AM

Oh we did. Eventually we had to set the token and the specific header the have for the secure token. It’s the same value and a bug but it makes it work

K

koalaspider•10/22/25, 4:54 AM

@Zack Finally figure how the protocol works. the worker managed to talk to the AI gateway.... how much brain cells I have destroyed LOL

Kkoalaspider @Zack Finally figure how the protocol works. the worker managed to talk to the ...

Z

Zack•10/22/25, 4:55 AM

I hear you. It took a while but after its pretty good

K

koalaspider•10/22/25, 11:58 AM

@Zack It would be nice to be able to find good documentation

Z

Zack•10/22/25, 12:14 PM

In the teams defense it is a fast evolving space

A

adhami•10/24/25, 8:21 PM

it doesn't seem that

cache_control

cache_control

cache_control

cache_control is being passed quite right for anthropic when using the openai compat endpoint

D

Davendra•10/29/25, 3:16 AM

Hi Will you support Minimx M2 as. Provider seems like a kick ass open source model - https://www.minimax.io/news/minimax-m2

MiniMax AI ｜ AGI-Powered Foundation Models & Voice/Text/Image/Vid...

Build with next-gen AGI: MiniMax offers Foundation AI models (text/audio/image/video). Power products like Talkie, Hailuo AI & enterprise APIs. Scalable, secure AI solutions for 50K+ global businesses.

Ddave AI Gateway is incorrectly considering a streaming `refusal` on the Anthropic API...

D

dave•10/31/25, 8:50 PM

FYI AI Gateway is even incorrectly labeling an API overloaded error as successful, because it's not correctly looking at the streamed error.

A

adhami•11/3/25, 7:38 PM

google vertex AI is responding with 400s out of nowhere, but it works without using the gateway, there's no response attached

Z

Zetax•11/4/25, 2:02 PM

Are people still being accepted for the unified billing for the AI gateway? I applied a while back but never got accepted

J

Jeremy Kreutzbender•11/5/25, 12:00 AM

We set up Logpush to get our AI Gateway logs exported, but one thing I saw was that the Cost data wasn't available. I see the token usage, but no cost. Is this intentional or did I do something wrong when exporting?

C

cosbgn•11/6/25, 1:48 PM

Any updates on how this works: ai-gateway ?

K

koalaspider•11/6/25, 2:49 PM

I am having so much issue with AI gateway. Even the same coding, same tokens, same configuration and same permission was working the day before but won't work the next day. There is no consistency here. I am thinking I am giving up on working on anything around AI gateway. No helpful support from cloudflare. Sometimes, wasted lots of time to troubleshoot. Can't work with something that work occassionally. Such a disppointment! I need to find another platform. Even GCP doesn't have so much issue but lack of security. I am so disappointed with cloud flare.

B

Bryan Chen•11/7/25, 12:27 AM

I am trying to setup some dynamic routes, the visual editor is nightmare to deal with... it is buggy and I have to constantly refresh to get it working and the visualization isn't that helpful. My workflow isn't that complicated (just some rate limiting + fallbacks). We are developers and this is for developers, so can we just have some nice JSON/YAML editor instead? I write if-else better than dragging boxes.

K

koalaspider•11/7/25, 3:43 PM

Something is wrong with the AI gateway and/worker AI since a few days ago. The AI gateway stop responding on other questions except for "What is Cloudflare?". It took me a few days wondering why everything doesn't work since a few days ago

T

teeter•11/11/25, 6:40 AM

according to your experience, wts the best web search API + model routing for AI agents :happy: and why

A

Andrew Hansen•11/12/25, 8:34 PM

The ai gateway keeps flagging the AutoRAG content as jailbreak (I'm just sending Mechanical Engineering content), I've tried adding it to the system message and also as an assitant message later. Any tips on preventing this? We'd like to keep it enabled if possible

C

cosbgn•11/13/25, 8:43 AM

What would be amazing in gateway would be some sort of user analytics, openai and other proviers allow passing a

user

user

key to idenitify requests for different users

Ccosbgn What would be amazing in gateway would be some sort of user analytics, openai an...

J

Jeremy Kreutzbender•11/13/25, 4:50 PM

You can do this by passing in metadata. Here is the Typescript code we use

export function buildAIGatewayHeaders(
  env: SharedHonoEnv,
  headers?: LLMHeaders
): Record<string, string> {
  const metadata: Record<string, string> = {};

  // Allowlist: only include specific headers we want to pass to AIGateway
  if (headers?.userToken) {
    metadata.userToken = headers.userToken;
  }

  if (headers?.accountToken) {
    metadata.accountToken = headers.accountToken;
  }

  if (headers?.requestId) {
    metadata.requestId = headers.requestId;
  }

  const aigHeaders = {
    "cf-aig-authorization": `Bearer ${env.CLOUDFLARE_AI_GATEWAY_API_KEY}`,
    "cf-aig-metadata": JSON.stringify(metadata),
  };

  return aigHeaders;
}

export function buildAIGatewayHeaders(
  env: SharedHonoEnv,
  headers?: LLMHeaders
): Record<string, string> {
  const metadata: Record<string, string> = {};

  // Allowlist: only include specific headers we want to pass to AIGateway
  if (headers?.userToken) {
    metadata.userToken = headers.userToken;
  }

  if (headers?.accountToken) {
    metadata.accountToken = headers.accountToken;
  }

  if (headers?.requestId) {
    metadata.requestId = headers.requestId;
  }

  const aigHeaders = {
    "cf-aig-authorization": `Bearer ${env.CLOUDFLARE_AI_GATEWAY_API_KEY}`,
    "cf-aig-metadata": JSON.stringify(metadata),
  };

  return aigHeaders;
}

export function buildAIGatewayHeaders(
  env: SharedHonoEnv,
  headers?: LLMHeaders
): Record<string, string> {
  const metadata: Record<string, string> = {};

  // Allowlist: only include specific headers we want to pass to AIGateway
  if (headers?.userToken) {
    metadata.userToken = headers.userToken;
  }

  if (headers?.accountToken) {
    metadata.accountToken = headers.accountToken;
  }

  if (headers?.requestId) {
    metadata.requestId = headers.requestId;
  }

  const aigHeaders = {
    "cf-aig-authorization": `Bearer ${env.CLOUDFLARE_AI_GATEWAY_API_KEY}`,
    "cf-aig-metadata": JSON.stringify(metadata),
  };

  return aigHeaders;
}

export function buildAIGatewayHeaders(
  env: SharedHonoEnv,
  headers?: LLMHeaders
): Record<string, string> {
  const metadata: Record<string, string> = {};

  // Allowlist: only include specific headers we want to pass to AIGateway
  if (headers?.userToken) {
    metadata.userToken = headers.userToken;
  }

  if (headers?.accountToken) {
    metadata.accountToken = headers.accountToken;
  }

  if (headers?.requestId) {
    metadata.requestId = headers.requestId;
  }

  const aigHeaders = {
    "cf-aig-authorization": `Bearer ${env.CLOUDFLARE_AI_GATEWAY_API_KEY}`,
    "cf-aig-metadata": JSON.stringify(metadata),
  };

  return aigHeaders;
}

J

Jeremy Kreutzbender•11/13/25, 4:52 PM

Then you can filter the data in the dashboard

C

cosbgn•11/14/25, 8:09 AM

Nice

C

cosbgn•11/14/25, 8:09 AM

thanks

BBryan Chen I want to use AI gateway dynamic routing. However, it seems only support the bas...

B

Bryan Chen•11/17/25, 9:21 PM

B

Bryan Chen•11/17/25, 9:21 PM

any help plz?

N

nikiv.dev•11/19/25, 4:41 PM

CLOUDFLARE_GATEWAY_HOST=https://gateway.ai.cloudflare.com
CLOUDFLARE_GATEWAY_ENDPOINT=/v1/e1a1737083ffd849ad2638b7e6069dec/lin/compat
CLOUDFLARE_GATEWAY_API_KEY=
CLOUDFLARE_GATEWAY_MODEL=google/gemini-2.0-flash

CLOUDFLARE_GATEWAY_HOST=https://gateway.ai.cloudflare.com
CLOUDFLARE_GATEWAY_ENDPOINT=/v1/e1a1737083ffd849ad2638b7e6069dec/lin/compat
CLOUDFLARE_GATEWAY_API_KEY=
CLOUDFLARE_GATEWAY_MODEL=google/gemini-2.0-flash

CLOUDFLARE_GATEWAY_HOST=https://gateway.ai.cloudflare.com
CLOUDFLARE_GATEWAY_ENDPOINT=/v1/e1a1737083ffd849ad2638b7e6069dec/lin/compat
CLOUDFLARE_GATEWAY_API_KEY=
CLOUDFLARE_GATEWAY_MODEL=google/gemini-2.0-flash

CLOUDFLARE_GATEWAY_HOST=https://gateway.ai.cloudflare.com
CLOUDFLARE_GATEWAY_ENDPOINT=/v1/e1a1737083ffd849ad2638b7e6069dec/lin/compat
CLOUDFLARE_GATEWAY_API_KEY=
CLOUDFLARE_GATEWAY_MODEL=google/gemini-2.0-flash

N

nikiv.dev•11/19/25, 4:41 PM

i need to get these values apparently

N

nikiv.dev•11/19/25, 4:41 PM

N

nikiv.dev•11/19/25, 4:41 PM

but i don't get where

N

nikiv.dev•11/19/25, 4:41 PM

no envs here

N

nikiv.dev•11/19/25, 4:41 PM

i do have google ai studio there

Nnikiv.dev ``` CLOUDFLARE_GATEWAY_HOST=https://gateway.ai.cloudflare.com CLOUDFLARE_GATEWAY...

I

Isaac McFadyen•11/19/25, 4:57 PM

API Key needs to be generated from https://dash.cloudflare.com/?to=/:account/api-tokens and needs to have the AI Gateway permission. Gateway Endpoint looks correct already, assuming that's your proper Account ID in there.

S

Shados•11/20/25, 8:26 PM

Do CF worker AI models have rate limits for tokens? On the limts page, I can see RPM limits but not TPMs (beyond the context window of the model)

N

Nils•11/22/25, 3:36 PM

Scribe v2 best model and not availiable?

N

nwp•11/23/25, 10:23 AM

It is my understanding that if we enable billing we should be able to setup an authenticated gateway and then call any LLM provider supported by AI Gateway without providing the provider’s API key (since CF is handing the billing). However, if I issue a curl and do not provide the underlying provider’s API key, I get an error stating that I must provide it (in addition to the AI Gateway’s token which I generated in CF with the run permission). This makes no sense to me. Isn’t the whole point to be able to provide ONE key, access any provider and pay one bill to CF for all inferencing. This is the way the Vercel AI Gateway works but I want to host fully on CF.

Did you figure this out? I'm seeing the same thing with Anthropic. My workaround are to either inclu

Similar Threads