CAPI whitelist
Hi!
Multiple of our customers complained that an external resource (YouTrack) couldn't reach our SMTP server.
After hours of research we found out that the whole /16 supernet of IPs - that they are using a small portion of - are in the CAPI blocklist.
Before that, we tried whitelisting. We have a postoverflow whitelist that reads from a file. It's been working great.
It was after this that I found out about CAPI blocklists.
I have two questions, one out of curiosity, and one about the solution.
1. Why was it necessary to create a different whitelist for CAPI decisions?
2. Is it possible to point the capi whitelist to the same txt file we use for our other whitelist? (It is just a simple text file with IP addresses in CIDR format on separate lines.)
Thanks in advance!
23 Replies
Important Information
This post has been marked as resolved. If this is a mistake please press the red button below or type
/unresolve© Created By WhyAydan for CrowdSec ❤️
Why was it necessary to create a different whitelist for CAPI decisions?This is legacy thing where we would allow you to create a parser whitelist for local events and another for CAPI decisions. Please use the new Allowlist feature as it applies to both: https://docs.crowdsec.net/docs/next/cscli/cscli_allowlists
Is it possible to point the capi whitelist to the same txt file we use for our other whitelist? (It is just a simple text file with IP addresses in CIDR format on separate lines.)This is the same answer as above.
cscli allowlists | CrowdSec
cscli allowlists
Thank you for the quick answer!
Have a great rest of your day.
Resolving CAPI whitelist
This has now been resolved. If you think this is a mistake please run
/unresolveit would also be interesting @alukas if you can provide subnet so we can do our own reasearch (DM me)
as we dont ban ranges from CAPI only individual ip's so unless their range is malicious it be hard
we found this resource if we are on right track https://www.jetbrains.com/help/youtrack/cloud/youtrack-cloud-instance-ip-ranges.html
YouTrack Cloud Help
YouTrack Cloud Instance IP Ranges | YouTrack Cloud
Yes, absolutely, those were the ones.
Based on our testing we mostly got requests from 18.201.1.131
From our side, we have recieved reports for this IP but we havent had enough to put within CAPI
so it seems the block may occured locally
On our machines it is part of the ipset crowdsec-blacklists-3
we also tried deleting any decisions with this IP, but the rule stayed in the ipset
18.201.0.0/16 timeout 853442
Can you check in the logs of the firewall bouncer which origin is for this ipset ?
Should have a log like
Using crowdsec-blacklists-3 as set for origin XXXtime="2025-03-12T14:24:03+01:00" level=info msg="Using crowdsec-blacklists-3 as set for origin cscli-import"
so local ban
someone imported a list of IPs to block with
cscli decisions importSo my original Allowlist option @alukas might not work as we didnt directly test it but now you know the origin and where you might need to prefilter before
cscli deicsions import is calledThanks for the help!
I'll try to find out where it came from exactly.
final ping @alukas we just dived into the code and can confirm allowlist wont work with
decisions import just so you are aware.I see, thanks @iiamloz !
I've been searching but it is easier to ask the professionals:
is it possible for me to somehow find out where the import came from?
if you run
cscli alerts list --all | grep "cscli-import"
hmm
https://github.com/crowdsecurity/crowdsec/blob/d64f196b3f3008cf39a538d6420012100044808a/cmd/crowdsec-cli/clidecision/import.go#L173
try cscli alerts list --all | grep "import"
it will tell you the generate alerts ids, which you may be able to inspect to see a machine id but unless you have multiple people running cscli around the place it wont tell you much.In our setup we have a central machine that is running the LAPI. (which just crashed on me when i asked it for a cscli machine list)
On the node where the IP range was in the ipset rules, there are only two alerts that come from cscli-import.
These both import a csv from the LAPI, and each of them ban 98 ranges, but none of them contain the range that we had an issue with.
- Do you have a trace for the crash ? (should be in
/var/lib/crowdsec/data/trace/ if it's LAPI that crashed, or if it was cscli, then the trace would hgave been shown to you)
- How did you check ? Based on the content of the file ? Based on the details of the alert ? CAPI does not send ranges (and if we did, we would not send a /16, that's way too big to ban in most cases), and by default crowdsec won't take decisions against range (we do have a scenario in the hub to ban a range if too many IPs from it are banned, but it's limited to a /24 and not installed by default)I think it crashed because we allocated a bit smaller amount of RAM than would be necessary at this size, it's been running for over a year, it really is not an issue and more than likely an error on our part. In the end we killed the process. The issue was that it stopped responding to cscli machine list and bouncer list, but I think it may have been because the VM didn't have enough resources, and in the meanwhile we got Icinga checks that were also calling the same list commands.
I do believe you that it wasn't a CAPI ban, I'm just having issues finding where it may have come from. I checked the file that was imported, I also checked with
cscli decisions list --range 18.201.0.0/16 which actually gave me an output of the 90+ IP ranges that were in the file, but it doesn't contain the 18.201.0.0/16 range.
This may remain a mystery.decisions list filter can be a bit buggy sometimes, especially with ranges :/
Best way would be to check directly in the db, assuming you are using the default sqlite:
Nope, nothing in the database.
The alert source ip is empty for imports, I guess you would need to join from decisions to get alert id