I also had similar a couple of days ago, and it's back again. 100% of POST/PUT/PATCH requests agains

B

Brett WillisOP•8/27/24, 8:36 PM

This is actually becoming a huge days-long downtime issue (on our paid workers plan)... is anyone from the Cloudflare team aware of it because I can't see it on the status dashboard?

BBrett Willis I also had similar a couple of days ago, and it's back again. 100% of POST/PUT/P...

C

Cyb3r-Jak3•8/27/24, 9:25 PM

I can't recreate. I have https://hello-world-billowing-brook-b77a.cyberjake.workers.dev/ and I can hit it with any method.

CCyb3r-Jak3 I can't recreate. I have https://hello-world-billowing-brook-b77a.cyberjake.work...

B

Brett WillisOP•8/27/24, 9:41 PM

Hmm... seems like the others reckoned it might something to do with smart placement. I see you have smart placement on too, but it is placed locally. Mine is placed in

remote-SEA

remote-SEA

.
Try

POST https://api-dev.hhc.workers.dev

POST https://api-dev.hhc.workers.dev

. That would normally give you 404 from the worker except it isn't even reaching the worker (no logs), instead 520.

BBrett Willis Hmm... seems like the others reckoned it might something to do with smart placem...

C

Chaika•8/27/24, 9:44 PM

using a specific body or something?

curl --request POST   --url https://api-dev.hhc.workers.dev/   --header 'Content-Type: application/json'   --header 'User-Agent: insomnia/2023.5.8'   --data '{}'  -vvv

curl --request POST   --url https://api-dev.hhc.workers.dev/   --header 'Content-Type: application/json'   --header 'User-Agent: insomnia/2023.5.8'   --data '{}'  -vvv

curl --request POST   --url https://api-dev.hhc.workers.dev/   --header 'Content-Type: application/json'   --header 'User-Agent: insomnia/2023.5.8'   --data '{}'  -vvv

curl --request POST   --url https://api-dev.hhc.workers.dev/   --header 'Content-Type: application/json'   --header 'User-Agent: insomnia/2023.5.8'   --data '{}'  -vvv

gives me

< HTTP/2 404
< date: Tue, 27 Aug 2024 21:44:09 GMT
< content-type: text/plain;charset=UTF-8
< content-length: 23
< cache-control: private, no-store
< cf-placement: remote-SEA
< report-to: {"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=tZaSH8jYa0BFaB%2Bmmq%2BSBFg2i9tN%2BNx3GXyaAe2cI65i0LiPX4YodMyfQKb1otEwjeh46Av5w4WE93VbpQzhmPIZlGlTEn3V7KzQiTS4XbhHlhDe8Pj5B7Kj3UChSwgMSifL4n778Xxe%2BQ%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 8b9f3dbf9ba841a6-EWR
< alt-svc: h3=":443"; ma=86400
{"message":"Not Found"}

C

Chaika•8/27/24, 9:45 PM

What exact headers do you get on the 520?

CChaika using a specific body or something? `curl --request POST --url https://api-dev...

B

Brett WillisOP•8/27/24, 9:47 PM

Huh yeah just got the same as you with curl but always 520 with Postman (and browsers), let me try find what the determining factor is...
No body/payload. See attached.

BBrett Willis Huh yeah just got the same as you with curl but always 520 with Postman (and bro...

C

Chaika•8/27/24, 9:50 PM

hmm you get back a response w/ 11 bytes, what's the response?

CChaika hmm you get back a response w/ 11 bytes, what's the response?

B

Brett WillisOP•8/27/24, 9:51 PM

Response is

Bad Gateway

Bad Gateway

.
Ah seemingly it's the presence of a

Content-Length: 0

Content-Length: 0

header:

% curl -i https://api-dev.hhc.workers.dev --header 'Content-Length: 0' 
HTTP/2 520 
date: Tue, 27 Aug 2024 21:50:25 GMT
content-type: text/plain;charset=UTF-8
content-length: 11
cf-placement: remote-SEA
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=j51afelgSe9%2FfmZXkPa9RvAJi7U2pWUpArXDewyLBkKu0sqdUNZzQfrHbo7KzZ3v1ckABzd14UCS30SF5UIObVLlOk7ATBmn5KNLT9yqPHZzJZQbAqQ7hUnKWrNZY6BtuL0xRSeyWcBQ9uXS4kqsul7hQ77DDQ%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 8b9f46efedd31c4d-AKL
alt-svc: h3=":443"; ma=86400

Bad Gateway

% curl -i https://api-dev.hhc.workers.dev --header 'Content-Length: 0' 
HTTP/2 520 
date: Tue, 27 Aug 2024 21:50:25 GMT
content-type: text/plain;charset=UTF-8
content-length: 11
cf-placement: remote-SEA
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=j51afelgSe9%2FfmZXkPa9RvAJi7U2pWUpArXDewyLBkKu0sqdUNZzQfrHbo7KzZ3v1ckABzd14UCS30SF5UIObVLlOk7ATBmn5KNLT9yqPHZzJZQbAqQ7hUnKWrNZY6BtuL0xRSeyWcBQ9uXS4kqsul7hQ77DDQ%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 8b9f46efedd31c4d-AKL
alt-svc: h3=":443"; ma=86400

Bad Gateway

% curl -i https://api-dev.hhc.workers.dev --header 'Content-Length: 0' 
HTTP/2 520 
date: Tue, 27 Aug 2024 21:50:25 GMT
content-type: text/plain;charset=UTF-8
content-length: 11
cf-placement: remote-SEA
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=j51afelgSe9%2FfmZXkPa9RvAJi7U2pWUpArXDewyLBkKu0sqdUNZzQfrHbo7KzZ3v1ckABzd14UCS30SF5UIObVLlOk7ATBmn5KNLT9yqPHZzJZQbAqQ7hUnKWrNZY6BtuL0xRSeyWcBQ9uXS4kqsul7hQ77DDQ%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 8b9f46efedd31c4d-AKL
alt-svc: h3=":443"; ma=86400

Bad Gateway

% curl -i https://api-dev.hhc.workers.dev --header 'Content-Length: 0' 
HTTP/2 520 
date: Tue, 27 Aug 2024 21:50:25 GMT
content-type: text/plain;charset=UTF-8
content-length: 11
cf-placement: remote-SEA
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=j51afelgSe9%2FfmZXkPa9RvAJi7U2pWUpArXDewyLBkKu0sqdUNZzQfrHbo7KzZ3v1ckABzd14UCS30SF5UIObVLlOk7ATBmn5KNLT9yqPHZzJZQbAqQ7hUnKWrNZY6BtuL0xRSeyWcBQ9uXS4kqsul7hQ77DDQ%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 8b9f46efedd31c4d-AKL
alt-svc: h3=":443"; ma=86400

Bad Gateway

B

Brett WillisOP•8/27/24, 9:52 PM

Regardless of the method. POST/PUT/PATCH method just causes the browser/postman to attach that header.

C

Chaika•8/27/24, 9:53 PM

curl -i https://api-dev.hhc.workers.dev --header 'Content-Length: 0'
HTTP/2 404
date: Tue, 27 Aug 2024 21:52:24 GMT
content-type: text/plain;charset=UTF-8
content-length: 23
cache-control: private, no-store
cf-placement: remote-SEA
report-to: {"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=mgSlH9e9mLRJNTkvw8TtayWcmmpz1mkZO1vbMMv9xFVajVdHaQDm4JvRQ6kRImJgwsvUaRpAyVjMOhuTsmfSYO5H5v8AVBe9RKV9YznggMdXwMYicrltGl8OLUz8Trn7vYFROWgikON5LA%3D%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 8b9f49d5aa9f0cc0-EWR
alt-svc: h3=":443"; ma=86400

Maybe it's because you're hitting AKL? Do you always get 520's with that or just sometimes? The fact that it has cf-placement seems like it's getting to SEA and wouldn't be colo-specific tho

C

Chaika•8/27/24, 9:53 PM

above, were you saying it only occurs with smart placement on?

B

Brett WillisOP•8/27/24, 9:55 PM

Yeah just confirmed, with smart placement turned off it functions normally (404). Obviously serving from AKL in my case.

B

Brett WillisOP•8/27/24, 9:58 PM

It's always 520 for me, but always via AKL colo in my case. It's a staging env so don't have other locations hitting. Did you get 520 from your colo?

BBrett Willis It's always 520 for me, but always via AKL colo in my case. It's a staging env s...

C

Chaika•8/27/24, 9:59 PM

no workers-and-pages-discussions hmm

C

Chaika•8/27/24, 10:01 PM

do you have any other envs to test with? No way that it could be related to any sort of corporate mitm/firewall/etc?

B

Brett WillisOP•8/27/24, 10:04 PM

Ok ok, so I'm using Cloudflare WARP. With WARP off my colo is SYD, and it works fine. With WARP on, 520.

C

Chaika•8/27/24, 10:08 PM

hmm warp does weird stuff sometimes too, do you have Zero trust proxy w/ the gateway tcp/udp proxy on or just normal warp?

J

JoeDoe•8/27/24, 10:09 PM

does a sleep() function add to cpu time?

CChaika hmm warp does weird stuff sometimes too, do you have Zero trust proxy w/ the gat...

B

Brett WillisOP•8/27/24, 10:12 PM

Just normal WARP (personal install, default settings).
Interesting why WARP only trips over here with the

workers.dev

workers.dev

domain and not a custom domain route?
Anyway it seems more related to WARP, so sorry for over-hyping that "workers downtime issue".

JJoeDoe does a sleep() function add to cpu time?

C

Chaika•8/27/24, 10:39 PM

js sleep is just done using Promises/setTimeout which isn't cpu time

BBrett Willis Just normal WARP (personal install, default settings). Interesting why WARP only...

C

Chaika•8/27/24, 10:39 PM

I tried reproing with WARP but I can't, wonder if it's related to akl

W

Walshy•8/27/24, 10:44 PM

let me hit it from akl

B

Brett WillisOP•8/27/24, 10:46 PM

Yeah I was just about to say, not sure it's completely a WARP issue. The 520 responses take ~500ms which is enough time go go all the way around the world (definitely beyond my machine and local colo which is normally 10-20ms away, SYD is 30-40ms).
And it's a combination of ALL these factors:

Smart placement ON (or served by SEA, not sure how to isolate but placement OFF + SYD works fine)
workers.dev domain (custom domains are fine)
Content-Length: 0 header present
WARP ON or AKL colo (haven't been able to isolate those two just yet)

W

Walshy•8/27/24, 10:46 PM

< HTTP/2 404
< date: Tue, 27 Aug 2024 22:46:02 GMT
< content-type: text/plain;charset=UTF-8
< content-length: 23
< cache-control: private, no-store
< cf-placement: remote-SEA
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=cYbKoFuCk5Dnjg57XLc%2Bd%2F%2FZtpZOn%2FwmSg%2BvVCeXENxdBtfzpywCkmgGzjg57C3Pcf7L61v4FVSlBRx61wVe48xXFDBKN0tMegx9GZMrvGuRIkG0keTz2lT14pqi4Gbtbfa6MnqLFt3UBg%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 8b9f9867883150bf-AKL

< HTTP/2 404
< date: Tue, 27 Aug 2024 22:46:02 GMT
< content-type: text/plain;charset=UTF-8
< content-length: 23
< cache-control: private, no-store
< cf-placement: remote-SEA
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=cYbKoFuCk5Dnjg57XLc%2Bd%2F%2FZtpZOn%2FwmSg%2BvVCeXENxdBtfzpywCkmgGzjg57C3Pcf7L61v4FVSlBRx61wVe48xXFDBKN0tMegx9GZMrvGuRIkG0keTz2lT14pqi4Gbtbfa6MnqLFt3UBg%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 8b9f9867883150bf-AKL

< HTTP/2 404
< date: Tue, 27 Aug 2024 22:46:02 GMT
< content-type: text/plain;charset=UTF-8
< content-length: 23
< cache-control: private, no-store
< cf-placement: remote-SEA
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=cYbKoFuCk5Dnjg57XLc%2Bd%2F%2FZtpZOn%2FwmSg%2BvVCeXENxdBtfzpywCkmgGzjg57C3Pcf7L61v4FVSlBRx61wVe48xXFDBKN0tMegx9GZMrvGuRIkG0keTz2lT14pqi4Gbtbfa6MnqLFt3UBg%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 8b9f9867883150bf-AKL

< HTTP/2 404
< date: Tue, 27 Aug 2024 22:46:02 GMT
< content-type: text/plain;charset=UTF-8
< content-length: 23
< cache-control: private, no-store
< cf-placement: remote-SEA
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=cYbKoFuCk5Dnjg57XLc%2Bd%2F%2FZtpZOn%2FwmSg%2BvVCeXENxdBtfzpywCkmgGzjg57C3Pcf7L61v4FVSlBRx61wVe48xXFDBKN0tMegx9GZMrvGuRIkG0keTz2lT14pqi4Gbtbfa6MnqLFt3UBg%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 8b9f9867883150bf-AKL

WWalshy let me hit it from akl

C

Chaika•8/27/24, 10:46 PM

thx, he was saying it only happens when request header

Content-Length

Content-Length

:

as well

W

Walshy•8/27/24, 10:47 PM

oops

W

Walshy•8/27/24, 10:47 PM

< HTTP/2 520
< date: Tue, 27 Aug 2024 22:46:58 GMT
< content-type: text/plain;charset=UTF-8
< content-length: 11
< cf-placement: remote-SEA
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=TZj%2FYtiLyiuphFwv0vlGdJHaMT35Mdjt9I0668lEGpt27Kyp8QdBuiVZH1ntXG8OpNS8vGfGoB6xIoUTdeH%2FxIRct9fZsusTSeqtG9F2diC5g7DK6u2u4yBaPqOOAdcXJddYfYzCkjg37A%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 8b9f99c64a9f7257-AKL

< HTTP/2 520
< date: Tue, 27 Aug 2024 22:46:58 GMT
< content-type: text/plain;charset=UTF-8
< content-length: 11
< cf-placement: remote-SEA
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=TZj%2FYtiLyiuphFwv0vlGdJHaMT35Mdjt9I0668lEGpt27Kyp8QdBuiVZH1ntXG8OpNS8vGfGoB6xIoUTdeH%2FxIRct9fZsusTSeqtG9F2diC5g7DK6u2u4yBaPqOOAdcXJddYfYzCkjg37A%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 8b9f99c64a9f7257-AKL

< HTTP/2 520
< date: Tue, 27 Aug 2024 22:46:58 GMT
< content-type: text/plain;charset=UTF-8
< content-length: 11
< cf-placement: remote-SEA
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=TZj%2FYtiLyiuphFwv0vlGdJHaMT35Mdjt9I0668lEGpt27Kyp8QdBuiVZH1ntXG8OpNS8vGfGoB6xIoUTdeH%2FxIRct9fZsusTSeqtG9F2diC5g7DK6u2u4yBaPqOOAdcXJddYfYzCkjg37A%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 8b9f99c64a9f7257-AKL

< HTTP/2 520
< date: Tue, 27 Aug 2024 22:46:58 GMT
< content-type: text/plain;charset=UTF-8
< content-length: 11
< cf-placement: remote-SEA
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=TZj%2FYtiLyiuphFwv0vlGdJHaMT35Mdjt9I0668lEGpt27Kyp8QdBuiVZH1ntXG8OpNS8vGfGoB6xIoUTdeH%2FxIRct9fZsusTSeqtG9F2diC5g7DK6u2u4yBaPqOOAdcXJddYfYzCkjg37A%3D%3D"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< server: cloudflare
< cf-ray: 8b9f99c64a9f7257-AKL

yep hit 520

W

Walshy•8/27/24, 10:47 PM

let me take a little peek

C

Chaika•8/27/24, 10:47 PM

hmm yea weird but doesn't seem to happen from anywhere else. Curious it has the cf-placement header on the 520 as well

W

Walshy•8/27/24, 10:48 PM

oh this is weird...

W

Walshy•8/27/24, 10:49 PM

we're failing in smart placement

W

Walshy•8/27/24, 10:49 PM

let me ping the Onboarding team

W

Walshy•8/27/24, 11:15 PM

Such a weird situation haha, for now you can disable smart placement @Brett Willis

Onboarding will figure out exactly how this is going wrong and fix

WWalshy Such a weird situation haha, for now you can disable smart placement @Brett Will...

B

Brett WillisOP•8/27/24, 11:17 PM

Thanks for the support, let me know of any updates if you can

W

Walshy•8/27/24, 11:17 PM

will do!

M

moonchav•8/27/24, 11:25 PM

is there an easy way to see which of my worker's routes are getting requested and how much?

M

moonchav•8/27/24, 11:27 PM

or do I have to get down and dirty with workers analytics engine and log the requests myself

W

Walshy•8/27/24, 11:33 PM

there is not, AE would indeed be the best path

W

Walshy•8/27/24, 11:34 PM

or you could look at zone analytics but there's no Workers path, you'd need to do your own filtering there

M

moonchav•8/27/24, 11:39 PM

ah actually the zone analytics pretty much does the trick for me with some filtering - thank you!

E

esselungo•8/28/24, 8:06 AM

what happens if you have a cron job running every minute and another running every five minutes? do they merge into one? is it two separate calls?

Eesselungo what happens if you have a cron job running every minute and another running eve...

A

Ashley•8/28/24, 11:15 AM

https://developers.cloudflare.com/workers/examples/multiple-cron-triggers/ tl;dr you can use

event.cron

event.cron

event.cron

event.cron to determine which one is being executed

Cloudflare Docs

Multiple Cron Triggers | Cloudflare Workers docs

Set multiple Cron Triggers on three different schedules.

E

esselungo•8/28/24, 11:15 AM

yeah but i was wondering if it was smart enough to like merge if two events are triggered at the same itme

A

Ashley•8/28/24, 11:17 AM

My assumption would be it would just be triggered twice, otherwise you'd have no way to differentiate and event.cron is just a string

S

Sant9Iga(Никита)•8/28/24, 11:44 AM

hi guys! I have a small question. I wrote a worker. I work with it locally using wrangler-cli. Locally it works well except KV namespaces. I tried to run my worker in the --remote mode. My worker got a "Verifying you are human" page in that mode. any ideas?

FFayaz How do I do that?

C

Cyb3r-Jak3•8/28/24, 12:19 PM

https://developers.cloudflare.com/1.1.1.1/encryption/dns-over-https/make-api-requests/

Cloudflare Docs

Make API requests to 1.1.1.1 | Cloudflare 1.1.1.1 docs

Cloudflare offers a DNS over HTTPS resolver at:

I also had similar a couple of days ago, and it's back again. 100% of POST/PUT/PATCH requests agains

Similar Threads

Similar Threads

Similar Threads