CrowdSec•3mo ago

auth.log seems to not get parsed

It seems my crowdsec instance is not parsing auth.log file from ubuntu linux. I am running the crowdsec container, I have my auth.log mounted in the docker container. Here is the metrics output:

├──────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────┤
│ Source                           │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├──────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
│ file:/var/log/auth.log           │ 249        │ 1            │ 248            │ 4                      │ -                 │

├──────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────┤
│ Source                           │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├──────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
│ file:/var/log/auth.log           │ 249        │ 1            │ 248            │ 4                      │ -                 │

When I run on the auth.log cscli explain, it correctly detects scenarios:

    ├ Scenarios
        ├ 🟢 crowdsecurity/ssh-bf

    ├ Scenarios
        ├ 🟢 crowdsecurity/ssh-bf

It also fails to parse some lines:

    └-------- parser failure 🔴

    └-------- parser failure 🔴

No decisions are made for the IPs that trigger scenarios.

28 Replies

CrowdSec•3mo ago

Important Information

Thank you for getting in touch with your support request. To expedite a swift resolution, could you kindly provide the following information? Rest assured, we will respond promptly, and we greatly appreciate your patience. While you wait, please check the links below to see if this issue has been previously addressed. If you have managed to resolve it, please use run the command /resolve or press the green resolve button below.

Log Files

If you possess any log files that you believe could be beneficial, please include them at this time. By default, CrowdSec logs to /var/log/, where you will discover a corresponding log file for each component.

Guide Followed (CrowdSec Official)

If you have diligently followed one of our guides and hit a roadblock, please share the guide with us. This will help us assess if any adjustments are necessary to assist you further.

Screenshots

Please forward any screenshots depicting errors you encounter. Your visuals will provide us with a clear view of the issues you are facing.

CrowdSec•3mo ago

Resolving auth.log seems to not get parsed This has now been resolved. If you think this is a mistake please run /unresolve Unresolving auth.log seems to not get parsed This has now been unresolved.

atiOP•3mo ago

It seems more people are running into this issue as it is posted on some forums, but it seems nobody found a solution yet no those forums.

iiamloz•3mo ago

The issue is your missing s00 stage parsers, you need to also install cscli collections install crowdsecurity/linux collection oooh wait im an idiot and didnt read it properly need my ☕ do you have examples of the log lines that fail to parse? you can redact the IP addresses cause ssh scenario detect fail logins and the scenario only overflows if the IP login multiple times within a set timeframe, it doesnt overflow at the first sign of a failed login

atiOP•3mo ago

i can give you more logs sure so what I see is a flood of login attempts with multiple user names like 30-40 times with users like lenovo and ibm and so

atiOP•3mo ago

here

atiOP•3mo ago

as visible above, for some reason only 1 line of auth.log was parsed

iiamloz•3mo ago

I guess you cropped out the timestamps and other information like program?

atiOP•3mo ago

yep

iiamloz•3mo ago

cause like I said, it depends on timeframe so without that I cant understand if it an actual issue

atiOP•3mo ago

so this is mostly in time frame of a second

iiamloz•3mo ago

any chance you can DM the raw log lines as I need to test them,

atiOP•3mo ago

okay le me try

_KaszpiR_•3mo ago

Its probably because of the timestamp I had the same issue recently https://discord.com/channels/921520481163673640/922593826986672178/1381389460880494643 Maybe ubuntu switched from ISO to RFC3339 But it would mean they changed their default rsyslog config after years But then it would affect much more log parsers

m3tc0n•2w ago

I have the same problem. My logs dont get parsed because of the RFC3339 time stamps. Do you have a fix?

iiamloz•2w ago

I cant find my previous chats with OP, so if anyone can provide some examples logs I can investigate the parser

_KaszpiR_•2w ago

https://github.com/crowdsecurity/hub/issues/725#issuecomment-2954284896 some example and explanation related to other parser

GitHub

Bug found in a1ad/mikrotik-logs · Issue #725 · crowdsecurity/hub

@a1ad This one is for you. I troubleshooted the a1ad/mikrotik-logs parser on a ROS 7.9 VM. Nothing was parsing using the original parser: %{TIMESTAMP_ISO8601:timestamp} %{HOSTNAME} .* %{DATA:tag} i...

_KaszpiR_•2w ago

Generally ubuntu switched to different time format and jt becomes a real mess when doing upgrades I remember it causes various problems with other apps so i keep the old format and just adjusted only specific one, no time to fix it then (or now), maybe in the near future

iiamloz•2w ago

Okay just so I can get up to speed, ubuntu changed default timestamp but only for rsyslog remote? or all cases and this example in mikrotik is simply where you found it?

m3tc0n•2w ago

I have the problem with the firewall bouncer nog having any hits. So I investigated and discovered auth.log having different time stamps so the logs dont get parsed. Maybe syslog is also in the different time stamps. I wouldnt be surprised.

iiamloz•2w ago

Well you can test it via the below command:

grep sshd /var/log/auth.log | grep -Ei '(failed|invalid)' | tail -n 1 | cscli explain -f - --type syslog

grep sshd /var/log/auth.log | grep -Ei '(failed|invalid)' | tail -n 1 | cscli explain -f - --type syslog

or simply

grep -Ei 'sshd.*(failed|invalid)' /var/log/auth.log | tail -n 1 | cscli explain -f - --type syslog

grep -Ei 'sshd.*(failed|invalid)' /var/log/auth.log | tail -n 1 | cscli explain -f - --type syslog

_KaszpiR_•2w ago

Afair they changed defaults in the /etc/rsyslog.conf which affects everything And AFAIR it mainly affects systems without journald (especially Debian)

iiamloz•2w ago

you mean without rsyslog cause debian 12 doesnt come with it by default but journald should be there

m3tc0n•2w ago

I am getting this results when adjusting the command:

[ramon@docker-2: $~]$ sudo docker exec -t crowdsec sh -c "grep -Ei 'sshd.*(failed|invalid)' /var/log/auth.log | tail -n 1 | cscli explain -f - --type syslog"
line: 2025-08-14T12:53:59.215453+02:00 docker-2 sudo:    ramon : TTY=pts/3 ; PWD=/home/ramon ; USER=root ; COMMAND=/usr/bin/docker exec -t crowdsec sh -c 'grep -Ei \'sshd.*(failed|invalid)\' /var/log/auth.log | tail -n 1 | cscli explain -f - --type syslog'
        ├ s00-raw
        |       ├ :red_circle: crowdsecurity/cri-logs
        |       ├ :red_circle: crowdsecurity/docker-logs
        |       └ :green_circle: crowdsecurity/syslog-logs (+12 ~9)
        ├ s01-parse
        |       ├ :red_circle: crowdsecurity/appsec-logs
        |       ├ :red_circle: crowdsecurity/iptables-logs
        |       ├ :red_circle: fulljackz/proxmox-logs
        |       ├ :red_circle: crowdsecurity/sshd-logs
        |       └ :red_circle: crowdsecurity/traefik-logs
        └-------- parser failure :red_circle:

[ramon@docker-2: $~]$ sudo docker exec -t crowdsec sh -c "grep -Ei 'sshd.*(failed|invalid)' /var/log/auth.log | tail -n 1 | cscli explain -f - --type syslog"
line: 2025-08-14T12:53:59.215453+02:00 docker-2 sudo:    ramon : TTY=pts/3 ; PWD=/home/ramon ; USER=root ; COMMAND=/usr/bin/docker exec -t crowdsec sh -c 'grep -Ei \'sshd.*(failed|invalid)\' /var/log/auth.log | tail -n 1 | cscli explain -f - --type syslog'
        ├ s00-raw
        |       ├ :red_circle: crowdsecurity/cri-logs
        |       ├ :red_circle: crowdsecurity/docker-logs
        |       └ :green_circle: crowdsecurity/syslog-logs (+12 ~9)
        ├ s01-parse
        |       ├ :red_circle: crowdsecurity/appsec-logs
        |       ├ :red_circle: crowdsecurity/iptables-logs
        |       ├ :red_circle: fulljackz/proxmox-logs
        |       ├ :red_circle: crowdsecurity/sshd-logs
        |       └ :red_circle: crowdsecurity/traefik-logs
        └-------- parser failure :red_circle:

iiamloz•2w ago

Lol because your sudo its logging the docker exec command cause it matches itself, can you sudo docker -it exec crowdsec sh first then run it but it may now pick up on that command so you may want to add

grep -v sudo

grep -v sudo

before the explain

grep -Ei 'sshd.*(failed|invalid)' /var/log/auth.log | grep -v sudo | tail -n 1 | cscli explain -f - --type syslog

grep -Ei 'sshd.*(failed|invalid)' /var/log/auth.log | grep -v sudo | tail -n 1 | cscli explain -f - --type syslog

but you can see from that log line that it passed the syslog parser so the timestamp didnt matter

m3tc0n•2w ago

Thanks, it says the log file is empty

iiamloz•2w ago

then you most likely dont have any ssh failed logins.

m3tc0n•2w ago

Good to know. Thanks!

Gaming

Programming

auth.log seems to not get parsed

Did you find this page helpful?