Is it possible to parse multiple log lines combined?
Let's say I have this two log lines
The second line alone doesn't amount to much, and the first line also applies when successful logins occur.
I need a way to parse and evaluate two lines combined for a decision, since the New connection message contains the IP to block
and it appears also before other failed log messages.
11 Replies
Important Information
This post has been marked as resolved. If this is a mistake please press the red button below or type
/unresolve© Created By WhyAydan for CrowdSec ❤️
To parse multiline logs, crowdsec requires a common identifier between the lines (https://docs.crowdsec.net/docs/next/log_processor/parsers/format/#stash)
But in your case, there isn't anything, so you won't be able to do it
Format | CrowdSec
Parser configuration example
Wouldn't you be able to create a custom grok pattern for white space and/or enter?
if the first log line always ends with a dot. It should be possible, no?
No the problem is acquisitions read line by line, so each line is classed as its own entity so there is never another line. Hence why if there an identifier that connects the line together like a traceid or something they you can connect the lines using the caching system (its kinda hacky but it what we do for auditd).
Ahhh, I see
So the stash option that was presented by blotus would only work if both lines have something in common?
I can't use stash to store the IP value from line 1 and then use GetFromStash when parsing line 2? And then wait for line 2 to appear let's say 5 times and then do a blocking of the IP that I retrieve with GetFromStash?
Not really cause if the file has an influx of connections it could log out of order like
So it will guess client 2 as its the client that disconnecting not the server from the log.
Ok, then I guess my best choice would be creating a whitelist to which I add my own IPs. And just parse the New Connection Log. This way it should ignore my IPs and only detect foreign IPs. That should then work right? Since Whitelisting functionality is offered by Crowdsec e.g. to test configurations without blocking oneself.
At that point you might as well just firewall the port to your own IP?
Unless you have others connecting
It's a service hosted in the cloud I need to be able to connect from different machines to.
Thank you all very much for you help. Just started using Crowdsec a few days ago and it's really awesome.
Will mark this post as solved at this point. Thanks again.
Resolving Is it possible to parse multiple log lines combined?
This has now been resolved. If you think this is a mistake please run
/unresolve