cURL does if you specify the `-N` (nobuffer) flag

IIsaac McFadyen cURL does if you specify the `-N` (nobuffer) flag

N

Nico•6/4/25, 3:23 PM

Thanks! Didn't know this. Behavior didn't change

I

Isaac McFadyenOP•6/4/25, 3:23 PM

Did you also specify

"stream": true

"stream": true

"stream": true

"stream": true?

N

Nico•6/4/25, 3:24 PM

yes

I

Isaac McFadyenOP•6/4/25, 3:24 PM

Interesting, maybe it doesn't support it then.

N

Nico•6/4/25, 3:24 PM

Yeah must be. Curious what the error with the sdk version is

NNico Yeah must be. Curious what the error with the sdk version is

I

Isaac McFadyenOP•6/4/25, 3:30 PM

When you specified the URL with the SDK what URL did you use? IIRC the OpenAI SDK automatically adds

/chat/completion

/chat/completion

/chat/completion

/chat/completion to the end, so your URL shouldn't have that in it with the SDK

NNico Yeah must be. Curious what the error with the sdk version is

N

Nico•6/4/25, 3:32 PM

I used

https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions

https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions

https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions

https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat/chat/completions which is what is in the docs. But I'll try it without

/chat/completions

/chat/completions

/chat/completions

/chat/completions!

N

Nico•6/4/25, 3:33 PM

Yes that was the issue. The docs are wrong.

I

Isaac McFadyenOP•6/4/25, 3:47 PM

I'll submit a PR - thanks.

C

camudo•6/4/25, 4:22 PM

Are you guys able to see the AI Gateway logs normally?
general-helpAI Gateway bug?

C

camudo•6/4/25, 4:23 PM

ok seems out

Captura_de_Tela_2025-06-04_as_13.23.11.png

A

aflores•6/4/25, 6:17 PM

does llm streaming not work with the byob api keys way of using api gateway? kind of unusable for a lot of front facing client work if i can't stream an api response with my own keys and instead have to use some workers ai thing with the websocket?
don't see clarification in the docs and quick search here didn't turn up much. i can set streaming true in the grok version of the call and see it coming in as chucnks but it buffers them all before sending them down as a single response

�

😈 Donkey 💫•6/4/25, 6:32 PM

AI gateway failed to render data

�

😈 Donkey 💫•6/4/25, 6:32 PM

Invalid date and no content

�😈 Donkey 💫AI gateway failed to render data

K

Kathy•6/4/25, 9:46 PM

thanks for reporting! we've fixed this bug

KKathy thanks for reporting! we've fixed this bug

�

😈 Donkey 💫•6/6/25, 9:02 AM

Hello,

Now, Vertex AI accept location_id with "global" instead of specific regions like the us-central1... I can call with endpoint aiplatform.googleapis.com

But I can't call "global" via AI Gateway, it throws 404

D

dave•6/6/25, 5:08 PM

Could we have support for the new

https://*.openai.azure.com/openai/v1/*

https://*.openai.azure.com/openai/v1/*

https://*.openai.azure.com/openai/v1/*

https://*.openai.azure.com/openai/v1/* Azure OpenAI endpoint? I can't seem to get it working with AI Gateway; I'm guessing something is being mutated on the AI Gateway side incorrectly?

https://learn.microsoft.com/en-us/azure/ai-services/openai/reference-preview-latest

D

DrWayne•6/6/25, 5:18 PM

FYI, I was using the non-streaming websockets interface yesterday and the docs just say "Alternatively, we also support authentication via the sec-websocket-protocol header if you are using a browser WebSocket." without telling how. Fortunately, I found a snippet in a blog post about DO and websockets.

cf-aig-authorization.${AI_GATEWAY_TOKEN}

cf-aig-authorization.${AI_GATEWAY_TOKEN}

cf-aig-authorization.${AI_GATEWAY_TOKEN}

cf-aig-authorization.${AI_GATEWAY_TOKEN}

It would be nice if that was explicitly in the docs since it is non-obvious how to form that sec-websocket-protocol header to encapsulate the cf-aig-authorization header we'dve used in Node.js instead of the browser...

B

Bryan Chen•6/8/25, 9:34 AM

AI Gateway Universal Endpoint stops working for me. It seems like the auth header failed to be passed and always getting auth error from providers

�😈 Donkey 💫Hello, Now, Vertex AI accept location_id with "global" instead of specific regi...

O

Oblomki•6/10/25, 6:04 PM

Same problem. AI Gateway seems to replace 'global' with a region, leading to a 404 error.

O

Oli•6/13/25, 12:36 PM

Question for yall who are more experienced with ai gateway,

Were calling anthropic,openai,azure,grok,groq,deepseek:s sse streaming endpoints serverside, so essentially, its only called after any rate-limiting, auth, etc.

But ive been looking at the ai gateway to see if there would be any benefit to calling the providers trough ai gateway,
Would there be any benefit to calling those api:s trough ai gateway? if the calls are user-independant, and with rate-limiting essentially disabled?

How does the caching work?
Is it just simple like if the exact same prompt is sent twice it will be a cache hit? or does it work with variations of the same prompt etc?

Also, does it handle retries etc?

--
Question 2:
We serve all models from all providers over ws to client.
Is it possible with ai gateway, to instead of serverside > ai-gateway > aiprovider. Somehow do:
client > aiprovider > our-backend ?
Using a custom api?

D

dave•6/13/25, 8:02 PM

Would be nice if we could get token pricing for a bunch of the new models from Anthropic and OpenAI

M

Marino•6/14/25, 2:49 PM

Does the universal endpoint support tools?

Sspongebhav I didn’t fully understand your question. By tools do you mean MCPs? or if we can...

M

Marino•6/14/25, 5:36 PM

I mean tools as in function calling, I couldn't find anything in the docs.

MMarino Does the universal endpoint support tools?

U

U.Senthil Kumar | Contributor•6/15/25, 7:13 AM

Yes, the Universal Endpoint supports tools by allowing multi-provider payloads with specific configurations per message. It also handles retries and fallback logic, making tool use more flexible across providers.

UU.Senthil Kumar | Contributor Yes, the Universal Endpoint supports tools by allowing multi-provider payloads w...

M

Marino•6/15/25, 2:59 PM

Thank you.

B

Bryan Chen•6/17/25, 8:03 AM

I disabled gateway auth and that got it working

B

Bryan Chen•6/17/25, 8:03 AM

zero responses from CF

N

newqueue•6/17/25, 9:00 AM

# seems google-ai-studio streaming api not work, anybody know how to fix it?
curl "https://gateway.ai.cloudflare.com/v1/7{account_id}/{gateway_id}/google-ai-studio/v1beta/models/gemini-2.5-flash-preview-05-20:streamGenerateContent?alt=sse" \
           --header 'content-type: application/json' \
           --header 'x-goog-api-key: '(pass GEMINI_API_KEY) \
           --data '{
            "contents": [
                {
                  "role":"user",
                  "parts": [
                    {"text":"What is Cloudflare?"}
                  ]
                }
              ]
            }'

# seems google-ai-studio streaming api not work, anybody know how to fix it?
curl "https://gateway.ai.cloudflare.com/v1/7{account_id}/{gateway_id}/google-ai-studio/v1beta/models/gemini-2.5-flash-preview-05-20:streamGenerateContent?alt=sse" \
           --header 'content-type: application/json' \
           --header 'x-goog-api-key: '(pass GEMINI_API_KEY) \
           --data '{
            "contents": [
                {
                  "role":"user",
                  "parts": [
                    {"text":"What is Cloudflare?"}
                  ]
                }
              ]
            }'

# seems google-ai-studio streaming api not work, anybody know how to fix it?
curl "https://gateway.ai.cloudflare.com/v1/7{account_id}/{gateway_id}/google-ai-studio/v1beta/models/gemini-2.5-flash-preview-05-20:streamGenerateContent?alt=sse" \
           --header 'content-type: application/json' \
           --header 'x-goog-api-key: '(pass GEMINI_API_KEY) \
           --data '{
            "contents": [
                {
                  "role":"user",
                  "parts": [
                    {"text":"What is Cloudflare?"}
                  ]
                }
              ]
            }'

# seems google-ai-studio streaming api not work, anybody know how to fix it?
curl "https://gateway.ai.cloudflare.com/v1/7{account_id}/{gateway_id}/google-ai-studio/v1beta/models/gemini-2.5-flash-preview-05-20:streamGenerateContent?alt=sse" \
           --header 'content-type: application/json' \
           --header 'x-goog-api-key: '(pass GEMINI_API_KEY) \
           --data '{
            "contents": [
                {
                  "role":"user",
                  "parts": [
                    {"text":"What is Cloudflare?"}
                  ]
                }
              ]
            }'

Mmotin I just posted in the community forums about a regression about the universal end...

J

Jorel•6/23/25, 8:14 AM

I was working with this using the Cloudflare AI Gateway provider from Vercel's AI SDK, but could NOT get it to work.

Sounds like it's related, even tho i haven't inspected the actual resulting traffic. My workaround was to

Not use the built-in gateway provider, instead adding the gateway url manually

Sending the cloudflare AI Gateway authentication as

cf-aig-authorization

cf-aig-authorization

cf-aig-authorization

cf-aig-authorization header

createOpenAI({
baseURL: `https://gateway.ai.cloudflare.com/v1/${env.CLOUDFLARE_ACCOUNT_ID}/${customer.gatewayId}/openai`,
apiKey: env.OPENAI_API_KEY,
headers: {
 'cf-aig-authorization': env.CLOUDFLARE_API_KEY,
},
});

createOpenAI({
baseURL: `https://gateway.ai.cloudflare.com/v1/${env.CLOUDFLARE_ACCOUNT_ID}/${customer.gatewayId}/openai`,
apiKey: env.OPENAI_API_KEY,
headers: {
 'cf-aig-authorization': env.CLOUDFLARE_API_KEY,
},
});

createOpenAI({
baseURL: `https://gateway.ai.cloudflare.com/v1/${env.CLOUDFLARE_ACCOUNT_ID}/${customer.gatewayId}/openai`,
apiKey: env.OPENAI_API_KEY,
headers: {
 'cf-aig-authorization': env.CLOUDFLARE_API_KEY,
},
});

createOpenAI({
baseURL: `https://gateway.ai.cloudflare.com/v1/${env.CLOUDFLARE_ACCOUNT_ID}/${customer.gatewayId}/openai`,
apiKey: env.OPENAI_API_KEY,
headers: {
 'cf-aig-authorization': env.CLOUDFLARE_API_KEY,
},
});

A

AngusMa•6/23/25, 1:31 PM

I've heard that workers AI has OpenAI compatible endpoints, does it include

v1/models

v1/models

v1/models

v1/models endpoint or only contain

/v1/chat/completions

/v1/chat/completions

/v1/chat/completions

/v1/chat/completions and

/v1/embeddings

/v1/embeddings

/v1/embeddings

/v1/embeddings?

O

Oblomki•6/23/25, 5:38 PM

Does AI Gateway return any kind of info when using the Universal Endpoint to know which model was successfully used (if using multiple providers in the fallback request)?

OOblomki Does AI Gateway return any kind of info when using the Universal Endpoint to kno...

K

Kathy•6/23/25, 10:30 PM

https://developers.cloudflare.com/ai-gateway/configuration/fallbacks/#response-headercf-aig-step

Cloudflare Docs

Fallbacks

Specify model or provider fallbacks with your Universal endpoint to handle request failures and ensure reliability.

D

dave•6/24/25, 4:35 PM

Anthropic Claude Sonnet 4 and Opus 4 pricing are still very wrong, they aren't taking into account cache related tokens at all.

T

Tait•6/24/25, 8:55 PM

I was trying to https://github.com/googleapis/js-genai with AI Gateway. Is that supported? My inital take after fighting with this for a while is that I should use the deprecated google library https://github.com/google-gemini/deprecated-generative-ai-js

Am I completely wrong? I hope so. I'm trying to use gemini to process short videos and I'd love to track my codst.

GitHub

GitHub - googleapis/js-genai: TypeScript/JavaScript SDK for Gemini ...

TypeScript/JavaScript SDK for Gemini and Vertex AI. - googleapis/js-genai

GitHub

GitHub - google-gemini/deprecated-generative-ai-js: This SDK is now...

This SDK is now deprecated, use the new unified Google GenAI SDK. - google-gemini/deprecated-generative-ai-js

P

piffie•6/25/25, 7:42 AM

Is there a better Frontend for the AI-gateway? where i can see maybe prompt versions or similar?

TTait I was trying to https://github.com/googleapis/js-genai with AI Gateway. Is that...

A

AngusMa•6/29/25, 3:22 AM

I think AI Gateway is based on REST API?

S

Shados•6/29/25, 5:17 AM

I'm using a gateway with a Worker AI binding (the "basic usage" in section 2 on https://developers.cloudflare.com/ai-gateway/integrations/worker-binding-methods/)

I don't get any error and my app works, but I don't see anything in the gateway's logs.

If I use the REST endpoint to my gateway to make the same request to the same model, then I see log entries. Is that a bug or am I doing something wrong?

If I use this.env.AI.gateway then gateway.run, then it works as I expected, but since the doc and the typescript type explicitely state the gateway feature should be supported directly, I found the behavior surprising.

S

Shados•6/29/25, 5:50 AM

Even then something seems off. Some features like input/output tokens and cost don't register when I go through the

gateway.run

gateway.run

gateway.run

gateway.run function inside workers.

So so far:

using a regular Worker AI binding with the gateway property: doesn't show up in logs at all
Getting the gateway by id from the binding and calling gateway.run: shows up in logs, no token/cost show up
Using REST api via curl: everything works as expected.

(it case it matters, im using the Worker AI models in this case)

Edit: Ok token/cost is not REST api vs gateway binding, its Streaming vs Non-Streaming.

So that only leaves the issue of: Why do I not get logs if I'm using the bindings without gateway.run? Is it because it only logs queries that go through the API and direct bindings bypass that? In which case what features do I actually get, since logs are kind of the big one?

A

ac•6/29/25, 7:12 PM

We are having issues where OpenAI requests are now getting blocked because they are originating from unsupported countries (via AI Gateway). This didn't start happening until this week for us, but it seems to be a new error as of the past few weeks (see https://community.cloudflare.com/t/cloudflare-worker-blocked-by-openai/804750

https://community.cloudflare.com/t/ai-gateway-routes-to-restricted-regions-causing-api-blocks/808571/6).

an anyone at Cloudflare comment on this?

Aac We are having issues where OpenAI requests are now getting blocked because they ...

H

harshil1712•6/30/25, 10:33 AM

forwarding this to the team.

Hharshil1712 forwarding this to the team.

A

ac•6/30/25, 10:37 AM

Thank you! This is a big issue for us.

J

jedi•6/30/25, 12:16 PM

hi guys

cURL does if you specify the `-N` (nobuffer) flag

Similar Threads