Not sure if I understand difference between multi-server setup and log centralization.
I am new to Crowdsec and want to integrate this in my infrastructure. For testing purposes I set up two servers where I installed the Crowdsec security engine on both machines and configured one to be the central LAPI (Main Machine) and registered the other machine (Child Machine) to this central LAPI as shown in this instruction video: https://www.youtube.com/watch?v=V4rr2gcPfW0&t=2337s and this information from the documentation https://docs.crowdsec.net/u/user_guides/multiserver_setup/.
I also installed a firewall bouncer on the Child Machine and registered the bouncer to the Main Machine and this works and already started blocking some malicious IPs.
Now my question. In such a setup, where I installed the security engine on each node (not sure if installation of only bouncer is possible), would I still need to setup a central log processing via rsyslog? Would I need to install rsyslog on each child machine, as well as the main machine and then forward the logs as shown in this documentation article: https://docs.crowdsec.net/u/user_guides/log_centralization ?
Or is this not necessary if each child node has its own log processor, that they would then send alerts to the LAPI that they registered to on another machine to then get the decision to block back from the LAPI in case a match was found in any of the subscribed blocklists?
Because at the moment my understanding is following. When using Log Centralization, the crowdsec security engine is only installed on the central server. On the child nodes only the bouncers are installed and connected to the LAPI of the central server. And so that accurate decisions can be made, each node sends their logs via rsyslog to the central machines rsyslog and from there the data is analysed and decisions are made for the nodes to block any malicious IPs. While in multi-server architecture each node runs the security engine and parses their logs independently but shares it through the central LAPI.
CrowdSec
YouTube
Workshop: How to set up a CrowdSec multi server installation
In this video our support specialist walks through how to setup Crowdsec multi server installation! We have provided timestamps below to allow you to jump to specific sections.
TIMESTAMPS:
0:00 - Start
00:08 - Intro
01:13 - Stream Start
03:30 - Upgrade from sqlite to postgresql
17:50 - Nginx install and configuration
36:00 - Child agent setup
4...
About multi-server setup | CrowdSec
Introduction
Log Centralization | CrowdSec
Introduction
10 Replies
Important Information
This post has been marked as resolved. If this is a mistake please press the red button below or type
/unresolve© Created By WhyAydan for CrowdSec ❤️
Central rsyslog is not needed, because logs are processed locally on each machine. Then each node would send info to central local api, and other nodes could use messages from the api to apply secuity measures.
About logs in general. You can have it centralised for convenience, though. Processing it in central place is also possible but may introduce additional lag, but may also help in catching use cases where the same ip tries to probe nodes at once, thus may work better in some scenarios.
2. Rsyslog per host.
Yes you would need to configure log exporting to the central location.
So generally there are multiple options, which depend on the setup, such as computing power per node ( some devices may be too weak or not able to process logs such as routers)
Thank you very much for the fast response. But I am still not sure, if in a setup with rsyslog on each node still need the security engine installed or a bouncer is enough and work standalone as well.
But to make it more specific. I have the current setup. I have some servers in the cloud and some servers locally running. Via Port Mirroring I managed to receive all the traffic on one of my machines running locally with Suricata. The next step is to use the logs generated by suricata and block IPs on the node where malicious traffic was identified via Crowdsec.
The idea would be to run the security engine on the machine where suricata is running and parse the logs. When an malicious IP is found inside those logs a decision will be sent to cloud server bouncer from where the traffic is coming from to block the IP.
I am still unsure which architecture would be more suitable for this scenario.
If you centralise your logs, you just need a bouncer only.
The reason we made this article was so people who want to use enterprise account but don't want to pay for each node, they can have a central logging server so it will be just the cost of one.
Just note as well that rsyslog is a clear text protocol so it best not to send it over the wan unless you setup a VPN or something.
Ok, so both options would generate the same results. It is only a matter of setup and maintenance complexity, and processing power on each node?
You can also use encrypted messages over syslog. Or send them to something like s3 bucket and send sns notification when new file appears. Then another system for example on suricata host could fetch those logs and parse them. Antoher option is to install logging agent integrated with the cloud provide ( for example for AWS it would be Cloudeatch Logs) and also fetchc those logs from other.location over TLS
Or hub and spoke topology per location, such as each location has a local central logging agregation point which pushes logs further to central location for processing.
The remediation component may differ per location, for example in the cloud you would want to for example update Network ACL rules to drop traffic as soon as possible ( such as on security groups on loadbalancers etc), while on prem to drop it on the routers or border gateways.
Thank you both very very much. Your information helps a lot 😄
Of course in case internal attack you may still want to have additional protection on the hosts directly, thus to prevent of attack spread ( so tha for example you would not get ddosed from the internal networks)
Thank you again. I will mark this post as solved as you helped me with my questions.
Resolving Not sure if I understand difference between multi-server setup and log centralization.
This has now been resolved. If you think this is a mistake please run
/unresolve