HTTPS Requests being silently dropped?
tl;dr: CF is silently blocking incoming webhook/ipn requests (e.g. from paypal) - they never appear in logs, rules are correctly configured, endpoints reachable manually, payloads are not malformed. I'm looking for answers as to why this is happening now (after many years of it working fine).
inb4 "cloudflare doesn't silently block requests" - yeah, they do. please, read on.
I've been using cloudflare for about 5 or so years as my DNS provider and to proxy the connections to my webserver. It's always been pretty great and the features I used always worked more or less as I expected them to.
A few months ago though, suddenly, without any changes on my end, paypal was no longer able to reach my backend ipn script. I already had a rule for the url to prevent it being blocked which had been working fine for years.
I could still reach the script myself, and using cloudflares own trace tool, I could confirm that the script could be reached (matching the rule), even when closely matching the payload, user-agent etc. from paypal.
The weirdest part: cloudflare wasn't logging anything at all for the failed paypal connections. It was as though paypal were never sending the ipn payload - but they were - I confirmed it by changing the destination to a non-cloudflare host.
I spent a long time trying to diagnose the cloudflare issue but couldn't get to the bottom of it, so I churned from paypal and wrote a new implementation for a new payment provider. Everything worked, and I moved on with my life. (Note: this was with a 'Pro' account)
Recently a friend asked me to configure his webserver, cloudflare, etc. along with a payment system intergration - a webhook for mercadopago.
After getting everything set up and configured, having confirmed the webhook script on another non-cloudflare host, I deployed it on his backend, and lo and behold, cloudflare isn't letting the webhook through from mercadopago.
No log, nothing. Again, I confirmed the route, etc. All accessible. Everything fine. Except cloudflare isn't letting it through, somewhere upstream of this account. A fresh cloudflare account (free).
What the hell!
So this time I added a subdomain that wasn't proxied by cloudflare, gave it its own SSL (letsencrypt), re-wrote the webhook script as a standalone with its own database conn, and deployed it to the subdomain.
It works. The webhook arrives. Obviously I won't keep it like this because I am currently exposing the backend IP, i'll end up needing to put it elsewhere if I can't prevent cloudflare from blocking the requests upstream of my account.
So, to the question, what the hell is going on? Why are requests being silently blocked by cloudflare, and what can I do about it? What changed?!
Though I don't think any of this is actually relevant to the question, here are a few extra details:
- apache2 web server on ubuntu24.04
- cloudflare in full (strict) mode
- cloudflare origin certs on the web server
Trying to find other people experiencing this in the wild:
https://community.cloudflare.com/t/cloudflare-is-blocking-lemon-squeezy-webhook/807437/13
https://community.cloudflare.com/t/site-inaccessible-to-discord-crawler-silent-block-by-cloudflare-proxy/813140/7
https://community.cloudflare.com/t/suspected-network-block-for-live-paypal-ipns-all-local-cf-config-ruled-out/814435/3 (this guy solved it by changing his minimum TLS from 1.3 to 1.2 - but that doesn't seem to be my issue)
https://community.cloudflare.com/t/apex-push-notifications-blocked-websocket-handshake-failing-at-cloudflare-edge/825642
Frustrating to see in every one of these posts of people looking for help, there is some guy stating emphatically that cloudflare doesn't silently block requests.
inb4 "cloudflare doesn't silently block requests" - yeah, they do. please, read on.
I've been using cloudflare for about 5 or so years as my DNS provider and to proxy the connections to my webserver. It's always been pretty great and the features I used always worked more or less as I expected them to.
A few months ago though, suddenly, without any changes on my end, paypal was no longer able to reach my backend ipn script. I already had a rule for the url to prevent it being blocked which had been working fine for years.
I could still reach the script myself, and using cloudflares own trace tool, I could confirm that the script could be reached (matching the rule), even when closely matching the payload, user-agent etc. from paypal.
The weirdest part: cloudflare wasn't logging anything at all for the failed paypal connections. It was as though paypal were never sending the ipn payload - but they were - I confirmed it by changing the destination to a non-cloudflare host.
I spent a long time trying to diagnose the cloudflare issue but couldn't get to the bottom of it, so I churned from paypal and wrote a new implementation for a new payment provider. Everything worked, and I moved on with my life. (Note: this was with a 'Pro' account)
Recently a friend asked me to configure his webserver, cloudflare, etc. along with a payment system intergration - a webhook for mercadopago.
After getting everything set up and configured, having confirmed the webhook script on another non-cloudflare host, I deployed it on his backend, and lo and behold, cloudflare isn't letting the webhook through from mercadopago.
No log, nothing. Again, I confirmed the route, etc. All accessible. Everything fine. Except cloudflare isn't letting it through, somewhere upstream of this account. A fresh cloudflare account (free).
What the hell!
So this time I added a subdomain that wasn't proxied by cloudflare, gave it its own SSL (letsencrypt), re-wrote the webhook script as a standalone with its own database conn, and deployed it to the subdomain.
It works. The webhook arrives. Obviously I won't keep it like this because I am currently exposing the backend IP, i'll end up needing to put it elsewhere if I can't prevent cloudflare from blocking the requests upstream of my account.
So, to the question, what the hell is going on? Why are requests being silently blocked by cloudflare, and what can I do about it? What changed?!
Though I don't think any of this is actually relevant to the question, here are a few extra details:
- apache2 web server on ubuntu24.04
- cloudflare in full (strict) mode
- cloudflare origin certs on the web server
Trying to find other people experiencing this in the wild:
https://community.cloudflare.com/t/cloudflare-is-blocking-lemon-squeezy-webhook/807437/13
https://community.cloudflare.com/t/site-inaccessible-to-discord-crawler-silent-block-by-cloudflare-proxy/813140/7
https://community.cloudflare.com/t/suspected-network-block-for-live-paypal-ipns-all-local-cf-config-ruled-out/814435/3 (this guy solved it by changing his minimum TLS from 1.3 to 1.2 - but that doesn't seem to be my issue)
https://community.cloudflare.com/t/apex-push-notifications-blocked-websocket-handshake-failing-at-cloudflare-edge/825642
Frustrating to see in every one of these posts of people looking for help, there is some guy stating emphatically that cloudflare doesn't silently block requests.
Welcome to the official Cloudflare Developers server. Here you can ask for help and stay updated with the latest news
87,353Members
Resources
Was this page helpful?
