Stuck sending event
Hi there!
Yesterday our wordpress systems were under a denial of service attack from quite a large botnet. Not blocking this automatically was probably due to us not having CrowdSec properly configured, but this is something we will figure out.
However, upon looking through the logs I've found that quite often we get logs like this (a few dozen lines of the sort each time):
However, the identifier after
to is always different. Could you please tell me more about this log?
I assumed it was trying to send the event to our LAPI, but the identifier being different each time tells me I'm wrong.
Based on our graphs, I couldn't identify a CPU, RAM or network bottleneck either that would cause it to delay (neither on the LAPI neither on the client server).7 Replies
Important Information
This post has been marked as resolved. If this is a mistake please press the red button below or type
/unresolve© Created By WhyAydan for CrowdSec ❤️
This logs will happen if crowdsec takes too long to pour an event into a bucket (the id you see is the internal id of the bucket).
This can happen if crowdsec needs to handle much more logs it can (if you were under a big ddos with the default config, that's probably not surprising, by default we only use 1 CPU core).
For this particular error, you can increase the buckets_routines directive
CrowdSec Configuration | CrowdSec
CrowdSec has a main yaml configuration file, usually located in /etc/crowdsec/config.yaml.
just note you should increase
parser_routines and buckets_routines in unison so both should be the same value.Alright, that makes perfect sense, thank you!
So could it also be that our configurations regarding who to ban were right, it was just too overwhelmed because of this?
These logs happen before an alert is made, basically the manager that is monitoring the buckets was struggling to keep up cause so many events were being poured at the same time, increasing the routines means instead of one manager there are multiple working in unison.
Ah, I see!
Thank you very much for your help, we'll modify the configs, and hope for the best.
Have a great day!
Resolving Stuck sending event
This has now been resolved. If you think this is a mistake please run
/unresolve