Cloudflare Developers•5mo ago

website getting ddosed?

hi, my website is facing 502 http error from a very long time (6 hours), its a pterodactyl panel, and the nginx error log shows something like connect() to unix:/run/php/php8.1-fpm.sock failed (11: Resource temporarily unavailable)... the thing is we are thinking its a ddos and have also enabled waf from cloudflare and blocked every country except India yet the website is down, is it really a ddos?

54 Replies

debargha•5mo ago

also this happened all of a sudden, everything was working fine

Chaika•5mo ago

To check if it's a DDoS, I would check: 1. your server's cpu usage/network 2. the overview graph in cloudflare for your site 3. Security -> Events for the number of events/requested blocked do you use Cloudflare Tunnels at all or no? Another common source of bad gateway if misconfigured

debargha•5mo ago

not using tunnel, but it is proxied, thinking to tunnel it

debargha•5mo ago

under attack mode is also active

Chaika•5mo ago

I would make sure they're not going around Cloudflare as well, only allow Cf IPs to ports 443/80 (assuming you're using default) also https://community.cloudflare.com/t/mitigating-an-http-ddos-attack-manually-with-cloudflare/302366

debargha•5mo ago

and this is the bandwidth

debargha•5mo ago

in the skip waf?

Chaika•5mo ago

no, this would be configuration on your origin https://developers.cloudflare.com/fundamentals/setup/allow-cloudflare-ip-addresses/

Allow Cloudflare IP addresses · Cloudflare Fundamentals docs

Because of how Cloudflare works, all traffic to proxied DNS records pass through Cloudflare before reaching your origin server. This means that your …

Chaika•5mo ago

that's way more use then Cloudflare shows. I assume you're running actual gameservers on that machine too, right?

debargha•5mo ago

yes, currently only minecraft okay let me check

debargha•5mo ago

do i have to follow these for each of the ips in the list?

Chaika•5mo ago

if you use a specific firewall already like ufw there's scripts out there as well, ex; https://github.com/Paul-Reed/cloudflare-ufw

GitHub

GitHub - Paul-Reed/cloudflare-ufw: Script to update UFW with Cloudf...

Script to update UFW with Cloudflare IPs. Contribute to Paul-Reed/cloudflare-ufw development by creating an account on GitHub.

Chaika•5mo ago

just be careful to not block everything/yourself out

debargha•5mo ago

i did this, also tunnelled the panel, yet its 502 this time its a little better at least the page opens and then again goes 502 and same going on

debargha•5mo ago

also, is this not for free? im not able to see it

debargha•5mo ago

@Chaika

Chaika•5mo ago

you only have events for free, not the graph still helpful to look over those and see if you can't identify a pattern possible on your origin as well your setup isn't optimized well/could be changed to handle requests better your description sounds like its just overloading it. Possible you have some limits you can raise, depends on your config

debargha•5mo ago

btw this isnt any problem right?

Chaika•5mo ago

I'm no linux expert but I don't see anything problemsome there. It's normal and expected for those /dev/loop | snap devices to be 100% full

debargha•5mo ago

ok, can u tell me a way to fix this issue completely its down for like 9 hours now NotLikeThis

the ddos seems to be out from india so I blocked every country yet its down coz the event doesnt show anything else

Chaika•5mo ago

I can't help much with the pterdaoctyl side of things, you have basically two options: Try to block more requests/identify a pattern you can block Increase origin resources/fix config to serve better

Chaika•5mo ago

If you google your error you can find some interesting info for how you can better configure your php fm pool: https://serverfault.com/questions/843460/php-fpm-sock-failed-11-resource-temporarily-unavailable-while-connecting-to-u

Server Fault

php-fpm.sock failed (11: Resource temporarily unavailable) while co...

I am stacked with following errors on my site when I test 200 hits per second. First I received 499 errors 2017-04-09 03:22:45 Error 162.158.79.219 499 GET / HTTP/1.1 0 nginx access
...

Chaika•5mo ago

https://serverfault.com/questions/867866/11-resource-temporarily-unavailable-while-connecting-to-upstream-bad-gateway

debargha•5mo ago

did, still the same... i even blocked every ip except mine still not up is it even ddos at this point nvm it turned out to be a hardware issue which the vps provider didnt check properly thanks for the help

Chaika•5mo ago

there was def a spike in the request graph you showed though

Chaika•5mo ago

perhaps it just revealed the underlying issue

debargha•5mo ago

but most of them was also blocked and when the ddos stopped the website still didnt come online coz i actually blocked every ip except mine lol

Chaika•5mo ago

I wouldn't be surpised if one caused the other, if the issue was heating or something, if you have never had that issue before and it started with the ddos seems too much to just be bad timing. There was posts about that issue mentioning the same, prob that either requests were stll queued or still consuming resources anyway you secured your origin as well and know how to protect yourself in the future, good thing to do anyway

debargha•5mo ago

the minecraft servers were running fine too, thats what makes it more confusing maybe only error in http and not tcp ? can it happen?

Chaika•5mo ago

I don't see how that would be a hardware issue related though

debargha•5mo ago

hmm, btw the nginx error logs showed like having trouble connecting to php-fpm socket the version was matching too like resource temporary unavailable

Akama Aka @ DoKomi•5mo ago

Have you tried restarting the fpm socket? Because if the fpm is down then you will get a bad gateway error back

debargha•5mo ago

yes obv i did, i even restarted the vps

Akama Aka @ DoKomi•5mo ago

Is the socket even active?

debargha•5mo ago

yes it was active like there were no errors in the status only the nginx had error connecing to the socket and my cpu was stuck at 66%

Akama Aka @ DoKomi•5mo ago

<:0010shiba_think:938055335397191690> Try to add a secondary socket as a fallback thats how I would do it then

debargha•5mo ago

how to? andd what secondary socket? like the same or some diff?

Akama Aka @ DoKomi•5mo ago

A different one a version lower or higher

debargha•5mo ago

i can use two php at th same time?

Akama Aka @ DoKomi•5mo ago

Not the same versions but two different versions

location ~ \.php$ {
  fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
  fastcgi_intercept_errors on;
  error_page 502 = @fallback;
}

location @fallback {
  fastcgi_pass 127.0.0.1:9000;
}

location ~ \.php$ {
  fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
  fastcgi_intercept_errors on;
  error_page 502 = @fallback;
}

location @fallback {
  fastcgi_pass 127.0.0.1:9000;
}