AppSec whitelist? Ignore vpatch-git-config when matching?
Hello, I am new to CrowdSec and have always worked with Fail2Ban before.
I secured my NPM with CrowdSec. Like many others, NPM acts as a bridge to underlying services. The
access.logs and error.logs are processed, and AppSec is also configured. Everything works. Today, I put my Nextcloud behind the NPM into operation. I was able to successfully configure the parser whitelist (s02-enrich) so that http-sensitive-files does not block my .git directories, etc.
However, I still occasionally got 403 and 404 errors during synchronization, which ultimately led to a ban for some directories (http-probing).
I didn't understand this at first, so as a newbie, I had to dig pretty deep and found out that it was due to the AppSec Rules vpatch-git-config. Due to the large number of .git directories with the usual subfiles, the large number of 403 errors triggered the http-probing alarm and ban.
Anyway, my problem now is that I don't want to simply and stupidly disable vpatch-git-config completely because of the other services behind it in NPM. I want to create an AppSec whitelist that responds to a URI including regex and does not execute any further AppSec or only vpatch-git-config.
I've googled, I've asked the AI, I've looked in the Crowdsec docs – without success. Can anyone help me?
For the sake of completeness, here is my s02-enrich whitelist, which does not work for AppSec InBand rules, as these are processed before the parser.
Thanks.34 Replies
Important Information
Thank you for getting in touch with your support request. To expedite a swift resolution, could you kindly provide the following information? Rest assured, we will respond promptly, and we greatly appreciate your patience. While you wait, please check the links below to see if this issue has been previously addressed. If you have managed to resolve it, please use run the command
/resolve or press the green resolve button below.Log Files
If you possess any log files that you believe could be beneficial, please include them at this time. By default, CrowdSec logs to /var/log/, where you will discover a corresponding log file for each component.
Guide Followed (CrowdSec Official)
If you have diligently followed one of our guides and hit a roadblock, please share the guide with us. This will help us assess if any adjustments are necessary to assist you further.
Screenshots
Please forward any screenshots depicting errors you encounter. Your visuals will provide us with a clear view of the issues you are facing.
© Created By WhyAydan for CrowdSec ❤️
For appsec, you'll need to create a custom appsec config in
/etc/crowdsec/appsec-configs/, for example whitelist.yaml with this content:
(I've taken the path from your example, feel free to change it or add more conditions)
And finally, reference this new appsec-config (custom/appsec-whitelist) in your acquisition config with the appsec-configs parameter (if you use appsec-config instead of appsec-configs, it's the same thing except it can take multiple config)
and because you also allow .env you'll also likely want to disable this rule as well
in the apply, add another item with RemoveInBandRuleByName("crowdsecurity/vpatch-env-access")Great. Thanks for the quick reply. I'll test it later or tomorrow and get back to you if necessary, or mark the post as solved.
So, the filters
req.URL.Host, req.URL.Path, etc. – where can I find them, or where can I see which ones are available and recognized?
Is there something similar to the following for AppSec detection:
Here you can already see which parameters are available for the filter.
By acquisition config, you mean the one under acquis.d in my case, npmplus.yaml.
Here, I add source or labels, then appsec-config: custom/appsec-whitelist? Correct?
Am I too stupid, or can't I find these options or possible parameters in the documentation?
https://docs.crowdsec.net/docs/next/log_processor/data_sources/introThe req object is documented here: https://docs.crowdsec.net/docs/next/appsec/hooks#req-object
AppSec Component Hooks | CrowdSec
The Application Security Component allows you to hook at different stages to change its behavior at runtime.
you'll want something like:
Hi @blotus, without filter
req.URL.Host == "nas.home.perrycox007.de" it works.
What am I doing wrong with req.URL.Host?
docker logs crowdsec -f
docker logs npmplus -f
can you add
debug: true in the appsec config ?
This will make crowdsec log more details about the expression eval, so you'll be able to see what is actually being comparedWait a moment, test briefly ' instead of "
Nope
not in the acquisition, directly in your custom appsec config
lol 😄
you can also put the acquisition in debug, but it will be way too verbose just to debug this
the same:
ah my bad
log_level: debug
it supports more granular logging configuration than just debug on/offIt's not easy to read...
I can't find anything in the output with nas.home or similar.
What should I look for?
Reminder: Without filter
req.URL.Host == "nas.home.perrycox007.de" it works.I just did a test locally, I think we have a small bug with the logging of expressions in the waf: they don't seem to properly use the logger (not sure why).
Thanks!! 😘
Its 1.16.11
@blotus You ping me if there is anything new or is there anything else I can do? Thanks a lot
for this issue, not much, for some reason the log level of the logger is set to
panic, regardless of the configuration, meaning it will never log anything
This also means that the helpers that can log by themselves (LogInfo for example) will not work :/
Can you try with req.Host instead ?Works.
Nice.
cool
We need to fix the documentation then
and clean up a bit the content of
req I guessDo you know if
req.URL.Path is correct? I can only test it at home tonight.it should be yes
Nope.
Try it:
-
req.Path = type http.Request has no field Path
- req.URL = invalid operation: contains (mismatched types url.URL and string)req.URL.Path
is not working.
🙁
try
req.URI contains "/remote.php/dav/"Nope.
sorry friday 😄
req.RequestURII feel like we're looking for a needle in a haystack.
I think it's a German proverb. It might sound strange to you.
we have the same in french 🙂
I think it works, but now I have to adjust my s02 parser again. 😄 It's ridiculous.
Thanks for your hard work!
you don't need to change anything in the s02 parser I think ?
It's not the same objects that are exposed
No, that's not what I mean. Because AppSec now lets me through, http-sensitive-files triggers the access.log. So now I have to adjust the s02 parser.
aah ok 🙂
Works! Thanks. Solved!