What kind of rate limits apply when using unified billing on a partner provider, e.g. sonnet 4.5?

C

cosbgnOP•10/3/25, 8:34 AM

I think by default anthropic has something like 60 requests per minute, but I woudn't be using my own api key as I would use the gateway-billing

C

cosbgnOP•10/3/25, 8:35 AM

So I'm wondering if I can safely move all my traffic there and cloudflare will automatically use another api key if one hits a limit

Z

Zetax•10/4/25, 7:08 PM

Does Cloudflaare AI gateway allow me to call workers ai from any application? If not, that would be such a great feature

E

ElishaKay•10/8/25, 7:10 AM

Am I supposed to get logs of ai-search within API gateway?

I'm only seeing logs of my embedding calls - but not on my llm_completion (inference) calls

M

muhtasim•10/8/25, 8:46 AM

how to get llm usage (neuron) via api?

S

stephen•10/8/25, 9:27 PM

AI Gateway BYOK Issue: Google Vertex AI Imagen Authentication Fails
Problem
Cloudflare AI Gateway's BYOK (Provider Keys) feature successfully authenticates third-party Vertex AI models (Mistral) but fails to authenticate Google's native models (Imagen), despite using the same service account configuration.

---

What Works

Mistral OCR via AI Gateway BYOK:

await env.AI.gateway("my-gateway").run({
provider: "google-vertex-ai",
endpoint: "v1/projects/my-project/locations/us-central1/publishers/mistralai/models/mistral-ocr-2505:rawPredict",
query: { model: "mistral-ocr-2505", document: {...} }
})
//

200 OK - Works perfectly

---

What Fails

Imagen 4 via AI Gateway BYOK:

await env.AI.gateway("my-gateway").run({
provider: "google-vertex-ai",
endpoint: "v1/projects/my-project/locations/us-central1/publishers/google/models/imagen-4.0-generate-001:predict",
query: {
instances: [{ prompt: "test image" }],
parameters: { sampleCount: 1 }
}
})
//

401 from Google: "CREDENTIALS_MISSING"

---

Configurations Tried

Service account has Vertex AI User IAM role

Added "region": "us-central1" to service account JSON

Tested both :predict and :rawPredict endpoints - both fail

Verified credentials work when calling Vertex AI directly
---

Observation
Third-party publishers (publishers/mistralai/): BYOK authentication works
Google publishers (publishers/google/): BYOK authentication fails

The AI Gateway appears to not generate or pass OAuth2 credentials correctly for Google's native Vertex AI models.

---

Question: Is this a known limitation of BYOK with Google Vertex AI native models, or should this be working?

Sstephen AI Gateway BYOK Issue: Google Vertex AI Imagen Authentication Fails Problem Clou...

M

Marco Ancona•10/8/25, 9:53 PM

Not only Vertex. In the last 4 hours, we saw that all requests to Google AI Studio BYOK via AI Gateway are failing with the error "400 API key not valid. Please pass a valid API key."

We tried to generate a new key and tested the key locally, confirming it works. Still, the error occurs when running via AI Gateway.

Calls to OpenAI via AI Gateway work.

M

Marco Ancona•10/9/25, 7:01 AM

Is there any way to report a problem with a ticket to Cloudflare rather than just posting to void here?

MMarco Ancona Is there any way to report a problem with a ticket to Cloudflare rather than jus...

P

Peps•10/9/25, 12:33 PM

?support

S

SuperHelpflare•10/9/25, 12:33 PM

To contact Cloudflare Support about an issue, please visit the Support Portal and fill in the form on the portal. After submission, you will receive confirmation over email.

Some issues, such as Account or Billing related issues, cannot be solved by the community.
Any plan level (including Free plans) can open tickets for Account, Billing or Registrar ticket categories. Make sure to select the correct category to ensure it goes to the right place.
For more information on the methods by which you can contact Support for your plan level, see Contacting Cloudflare Support - Cloudflare Docs

Sstephen AI Gateway BYOK Issue: Google Vertex AI Imagen Authentication Fails Problem Clou...

S

stephen•10/9/25, 3:08 PM

My workaround:

Initial attempt (failed):

Provider: google-vertex-ai
Issue: Cloudflare AI Gateway BYOK doesn't support Vertex AI's Imagen endpoints (:predict or :rawPredict on
publishers/google/models/*)
Only Mistral models work with BYOK on Vertex AI (:rawPredict on publishers/mistralai/models/*)Working solution:
Provider: google-ai-studio
Model: imagen-4.0-generate-001:predict
Why it works: Google AI Studio is a separate provider that Cloudflare AI Gateway supports with BYOK, and Imagen 4 isavailable through itKey insight: Even though both Vertex AI and Google AI Studio host the same Imagen 4 model, AI Gateway's BYOK only works with Google AI Studio for Imagen, not Vertex AI. This is likely because they use different authentication mechanisms (Vertex uses service account JSON, AI Studio uses simple API keys).

S

SmallShen•10/9/25, 4:11 PM

Any plan on support openai responses api?

R

Razmjoo•10/9/25, 8:44 PM

Hi, is there any way to use

@cf/tomarkdown

@cf/tomarkdown

@cf/tomarkdown

@cf/tomarkdown /

@cf/markdown

@cf/markdown

@cf/markdown

@cf/markdown using the AI Gateway?

M

misterPaul•10/10/25, 12:08 AM

love the velocity of new changes, re: overview section quick CSS update will make this section more usable - see screenshots

min-h with calc vh is a bit funky, but I don't think there's another way to achieve a decent height on this element without this, or a layout refactor. Making the code section bigger is the ultimate objective.

M

misterPaul•10/10/25, 12:09 AM

End result below - much more high impact

These CSS fixes brought to you by an LLM executed through AI Gateway

M

misterPaul•10/10/25, 12:11 AM

This overview section is big value btw so thanks for adding; implementing AI Gateway with just the documentation was quite a challenge for a new user

A

Anton•10/13/25, 8:54 PM

I setup an agent based on the agent-starter template backed by openai's "gpt-4o" model which is working fine. Then I tried setting up AI Gateway by adding the

…/compat

…/compat

…/compat

…/compat-baseURL and updating the model name to "openai/gpt-4o". However I'm now getting 500 internal server errors with AI Gateway and don't see those requests in the dashboard.
Works without AI Gateway:

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY,
  // baseURL: env.GATEWAY_BASE_URL,
});
const model = openai("gpt-4o");

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY,
  // baseURL: env.GATEWAY_BASE_URL,
});
const model = openai("gpt-4o");

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY,
  // baseURL: env.GATEWAY_BASE_URL,
});
const model = openai("gpt-4o");

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY,
  // baseURL: env.GATEWAY_BASE_URL,
});
const model = openai("gpt-4o");

Fails with AI Gateway:

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY,
  baseURL: env.GATEWAY_BASE_URL,
});
const model = openai("openai/gpt-4o");

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY,
  baseURL: env.GATEWAY_BASE_URL,
});
const model = openai("openai/gpt-4o");

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY,
  baseURL: env.GATEWAY_BASE_URL,
});
const model = openai("openai/gpt-4o");

const openai = createOpenAI({
  apiKey: env.OPENAI_API_KEY,
  baseURL: env.GATEWAY_BASE_URL,
});
const model = openai("openai/gpt-4o");

Checking the logs on the worker/do, it turns out, it's calling

…/compat/responses

…/compat/responses

…/compat/responses

…/compat/responses with AI Gateway enabled, which fails. Shouldn't this be calling

…/compat/chat/completions

…/compat/chat/completions

…/compat/chat/completions

…/compat/chat/completions (works via curl)?

K

Kepro•10/14/25, 8:17 AM

Hello, is streaming working for you when using the AI Gateway? I found just some posts here and here but no response from CF

KKepro Hello, is streaming working for you when using the AI Gateway? I found just some...

B

Brandon Studio•10/15/25, 1:52 AM

I think the last reply is ai-gateway

Ddave This is a regression afaik, iirc @Kathy might remember we talked about it and I ...

K

Kepro•10/15/25, 7:08 AM

Hello @dave | ai.moda and @Kathy | Browser Rendering PM , is there any update on this? It’s kind of a blocker for us right now

Thank you

WWard Hey all, is there an easy way to store my keys in the cloudflare dashboard and t...

E

engnadeau•10/15/25, 12:40 PM

@Ward did you figure out a solution to this issue?

E

engnadeau•10/15/25, 12:59 PM

Does Cloudflare AI Gateway BYOK actually support the OpenAI SDK?

I’m testing BYOK (“Store Keys”) with the Cloudflare AI Gateway using the OpenAI JS SDK. The docs only show curl examples, but the SDK always requires an apiKey.

If I omit it → Missing credentials.
If I include a dummy key → 401 Authorization Required

The docs say that BYOK Gateway should expect only cf-aig-authorization

Can someone from CF confirm whether BYOK is officially supported with the OpenAI SDK / Unified API, or if it’s currently curl-only?

A

alkgaming•10/17/25, 2:05 PM

oh

S

Strazarachne•10/17/25, 2:16 PM

Is there an up to date list on regions or networks by prefix not allowed through cloudflare front end proxy currently that services chatgpt.com?

S

sodepr•10/17/25, 7:05 PM

I’ve seen several posts mentioning that Cloudflare AI Gateway routes requests to restricted regions, causing some APIs to stop working. Has this issue been resolved yet?

Oozan How can I use Vercel AI SDK with BYOK? In my createOpenAI call if I don't pass a...

K

Kris•10/18/25, 5:01 PM

Did you figure this out? I'm seeing the same thing with Anthropic. My workaround are to either include the API key at source (which won't work once we move to unified billing) or use the Cloudflare AI Gateway provider for the Vercal AI SDK (haven't tried yet, but expecting it might not work quite as well as the native provider).

Eengnadeau **Does Cloudflare AI Gateway BYOK actually support the OpenAI SDK?** I’m testin...

K

Kris•10/18/25, 5:02 PM

Saw @ozan's post first.

Trying to figure out the same thing.

KKris Did you figure this out? I'm seeing the same thing with Anthropic. My workaround...

O

ozan•10/19/25, 7:50 AM

No, I could not integrate secrets store with ai sdk

S

sodepr•10/19/25, 11:45 PM

Hi everyone,

Does AI gateway support streaming?

https://community.cloudflare.com/t/streaming-responses-does-not-work-with-ai-gateway-google-ai-studio/837798

A

Arman•10/20/25, 10:39 AM

Hi

K

koalaspider•10/20/25, 1:15 PM

Both my dev and deploy version for AI worker used to work perfectly fine. But last few days, the deploy version Ai worker stop working. Have been trying troubleshoot for the last two days. The dev version still working fine. Anyone know why?

N

NtTestAlert•10/21/25, 4:50 AM

Hi, I'm trying to use AI Gateway via websocket (I am aware that it is in beta, but working around max concurrent connections otherwise is a nightmare.
It works for success responses, but for error responses it does not include the metadata (or meta doesnt contain eventId) - impossible to correlate which req the error happened in

Happy path (has eventId in metadata):

Sending pending {"type":"universal.create","eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","request":{"eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","headers":{"Authorization":"Bearer sk-or-v1-*****"},"provider":"openrouter","endpoint":"v1/chat/completions","query":{"model":"google/gemini-2.5-flash","messages":[{"role":"system","content":"Tell me a joke."}]}}}
Received "{\"type\":\"universal.created\",\"metadata\":{\"cacheStatus\":\"MISS\",\"eventId\":\"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3\",\"logId\":\"01K82FV50QQGRQWGDT*****\",\"step\":\"0\",\"contentType\":\"application/json\"},\"response\":{\"id\":\"gen-1761020318-TQij*****eHDCaT\",\"provider\":\"Google\",\"model\":\"google/gemini-2.5-flash\",\"object\":\"chat.completion\",\"created\":1761020318,\"choices\":[{\"logprobs\":null,\"finish_reason\":\"stop\",\"native_finish_reason\":\"STOP\",\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"A three-legged dog walks into a saloon and says, \\\"I'm looking for the man who shot my paw.\\\"\",\"refusal\":null,\"reasoning\":null}}],\"usage\":{\"prompt_tokens\":5,\"completion_tokens\":25,\"total_tokens\":30,\"prompt_tokens_details\":{\"cached_tokens\":0},\"completion_tokens_details\":{\"reasoning_tokens\":0,\"image_tokens\":0}}}}"

Sending pending {"type":"universal.create","eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","request":{"eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","headers":{"Authorization":"Bearer sk-or-v1-*****"},"provider":"openrouter","endpoint":"v1/chat/completions","query":{"model":"google/gemini-2.5-flash","messages":[{"role":"system","content":"Tell me a joke."}]}}}
Received "{\"type\":\"universal.created\",\"metadata\":{\"cacheStatus\":\"MISS\",\"eventId\":\"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3\",\"logId\":\"01K82FV50QQGRQWGDT*****\",\"step\":\"0\",\"contentType\":\"application/json\"},\"response\":{\"id\":\"gen-1761020318-TQij*****eHDCaT\",\"provider\":\"Google\",\"model\":\"google/gemini-2.5-flash\",\"object\":\"chat.completion\",\"created\":1761020318,\"choices\":[{\"logprobs\":null,\"finish_reason\":\"stop\",\"native_finish_reason\":\"STOP\",\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"A three-legged dog walks into a saloon and says, \\\"I'm looking for the man who shot my paw.\\\"\",\"refusal\":null,\"reasoning\":null}}],\"usage\":{\"prompt_tokens\":5,\"completion_tokens\":25,\"total_tokens\":30,\"prompt_tokens_details\":{\"cached_tokens\":0},\"completion_tokens_details\":{\"reasoning_tokens\":0,\"image_tokens\":0}}}}"

Sending pending {"type":"universal.create","eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","request":{"eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","headers":{"Authorization":"Bearer sk-or-v1-*****"},"provider":"openrouter","endpoint":"v1/chat/completions","query":{"model":"google/gemini-2.5-flash","messages":[{"role":"system","content":"Tell me a joke."}]}}}
Received "{\"type\":\"universal.created\",\"metadata\":{\"cacheStatus\":\"MISS\",\"eventId\":\"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3\",\"logId\":\"01K82FV50QQGRQWGDT*****\",\"step\":\"0\",\"contentType\":\"application/json\"},\"response\":{\"id\":\"gen-1761020318-TQij*****eHDCaT\",\"provider\":\"Google\",\"model\":\"google/gemini-2.5-flash\",\"object\":\"chat.completion\",\"created\":1761020318,\"choices\":[{\"logprobs\":null,\"finish_reason\":\"stop\",\"native_finish_reason\":\"STOP\",\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"A three-legged dog walks into a saloon and says, \\\"I'm looking for the man who shot my paw.\\\"\",\"refusal\":null,\"reasoning\":null}}],\"usage\":{\"prompt_tokens\":5,\"completion_tokens\":25,\"total_tokens\":30,\"prompt_tokens_details\":{\"cached_tokens\":0},\"completion_tokens_details\":{\"reasoning_tokens\":0,\"image_tokens\":0}}}}"

Sending pending {"type":"universal.create","eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","request":{"eventId":"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3","headers":{"Authorization":"Bearer sk-or-v1-*****"},"provider":"openrouter","endpoint":"v1/chat/completions","query":{"model":"google/gemini-2.5-flash","messages":[{"role":"system","content":"Tell me a joke."}]}}}
Received "{\"type\":\"universal.created\",\"metadata\":{\"cacheStatus\":\"MISS\",\"eventId\":\"fc7cd3f3-4885-4039-9e74-9dbe2f017ea3\",\"logId\":\"01K82FV50QQGRQWGDT*****\",\"step\":\"0\",\"contentType\":\"application/json\"},\"response\":{\"id\":\"gen-1761020318-TQij*****eHDCaT\",\"provider\":\"Google\",\"model\":\"google/gemini-2.5-flash\",\"object\":\"chat.completion\",\"created\":1761020318,\"choices\":[{\"logprobs\":null,\"finish_reason\":\"stop\",\"native_finish_reason\":\"STOP\",\"index\":0,\"message\":{\"role\":\"assistant\",\"content\":\"A three-legged dog walks into a saloon and says, \\\"I'm looking for the man who shot my paw.\\\"\",\"refusal\":null,\"reasoning\":null}}],\"usage\":{\"prompt_tokens\":5,\"completion_tokens\":25,\"total_tokens\":30,\"prompt_tokens_details\":{\"cached_tokens\":0},\"completion_tokens_details\":{\"reasoning_tokens\":0,\"image_tokens\":0}}}}"

N

NtTestAlert•10/21/25, 4:50 AM

bad path:

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"Input must have at least 1 token.\",\"code\":400,\"metadata\":{\"provider_name\":null}}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"Input must have at least 1 token.\",\"code\":400,\"metadata\":{\"provider_name\":null}}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"Input must have at least 1 token.\",\"code\":400,\"metadata\":{\"provider_name\":null}}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"Input must have at least 1 token.\",\"code\":400,\"metadata\":{\"provider_name\":null}}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"success\":false,\"result\":[],\"messages\":[],\"error\":[{\"code\":2008,\"message\":\"Invalid provider\"}]}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"success\":false,\"result\":[],\"messages\":[],\"error\":[{\"code\":2008,\"message\":\"Invalid provider\"}]}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"success\":false,\"result\":[],\"messages\":[],\"error\":[{\"code\":2008,\"message\":\"Invalid provider\"}]}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"success\":false,\"result\":[],\"messages\":[],\"error\":[{\"code\":2008,\"message\":\"Invalid provider\"}]}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"No auth credentials found\",\"code\":401}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"No auth credentials found\",\"code\":401}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"No auth credentials found\",\"code\":401}}}"

Received "{\"type\":\"universal.created\",\"metadata\":{\"contentType\":\"application/json\"},\"response\":{\"error\":{\"message\":\"No auth credentials found\",\"code\":401}}}"

N

NtTestAlert•10/21/25, 4:50 AM

The logs tab in dashboard shows correct event-id for happy path ~~but some random generated one for non-happy it seems~~ (only one of them is wrong in logs but it might be my mistake) bad path also has proper eventId in dashboard, it is just not sent in the websocket error forwarding (also notcie provider_name is null ? .o0)

N

NtTestAlert•10/21/25, 4:54 AM

I mean... I'll just wait for timeout for errors for now I guess :/ but this really is a nightmare, tried some solution with background processed durable objects with internal queueing to bypass the 6 connection limit, but that's also buggy

N

NtTestAlert•10/21/25, 4:54 AM

this seemed like exactly what I wanted to implement myself but it looks like it doesnt work properly either :/

N

NtTestAlert•10/21/25, 5:30 AM

Since the logs contain the correct eventId I assume this is a trivial fix, and its been broken like this for a while...

EDIT: this problem is tracked at https://github.com/cloudflare/workers-sdk/issues/11040

A

adhami•10/21/25, 8:23 PM

using cohere rerank through the AI gateway breaks because it attempts to str

documents

documents

in the json body

K

koalaspider•10/21/25, 10:14 PM

Have been working on a project to include AI gateway with a LLM over 1 week ago, both dev and deploy version has been working fine. Since 2 days ago, the AI functionality does not work on the deploy version but the dev version works fine. I rebuilt everything on the deploy version yesterday. The built process was completed and no error. But when it comes to wrangler deploy, it said "A request to the Cloudflare API failed. Authentication error [code: 10000]" I have set all the permission that I can think of for the build token. It seems that the npx wrangler deploy command has some issue. The dev version still working fine. Does anyone know why the deploy version failed? Is there a way to debug into the wrangler?

K

koalaspider•10/21/25, 10:20 PM

FYI I am actually connected to my github account and build the project without the CLI. I have tried to build and deploy the project using CLI but still the AI functionality failed. The dev version has been connected to the remote AI gateway and never have any issue at all. Is there a way solve this error?

K

koalaspider•10/22/25, 3:05 AM

Finally solve the wrangler deployment issues. but the authentication error happened.... OMG.

ZZack I've been really struggling to get the ai gateway to work, I can't seem to get a...

K

koalaspider•10/22/25, 3:10 AM

Hi, @Zack, I am having the same issue... Have you managed to solve the bug/error here?

Kkoalaspider Hi, @Zack, I am having the same issue... Have you managed to solve the bug/error...

Z

Zack•10/22/25, 3:38 AM

Oh we did. Eventually we had to set the token and the specific header the have for the secure token. It’s the same value and a bug but it makes it work

K

koalaspider•10/22/25, 4:54 AM

@Zack Finally figure how the protocol works. the worker managed to talk to the AI gateway.... how much brain cells I have destroyed LOL

Kkoalaspider @Zack Finally figure how the protocol works. the worker managed to talk to the ...

Z

Zack•10/22/25, 4:55 AM

I hear you. It took a while but after its pretty good

What kind of rate limits apply when using unified billing on a partner provider, e.g. sonnet 4.5?

Similar Threads

Similar Threads

Similar Threads