Direct manipulation of the allowlist in the database.
If a record with an IP is inserted into the
allow_list_items table directly in the DB, is it going to be picked up by the LAPI immediately or on a restart of the LAPI only?7 Replies
Important Information
This post has been marked as resolved. If this is a mistake please press the red button below or type
/unresolve© Created By WhyAydan for CrowdSec ❤️
If you are inserting it into the database, then LAPI picks it up as the table is queried when incoming requests go to
/v1/allowlists, no need to restart anything.
just note that if you are manipulating the database directly if there is a decision active it wont be removed as per the new feature (in #📣・announcements ) as the deletion of the decision is via cscli allowlist and papi (premium feature) components.If I make the same instance to process logs from Loki, is it going to check the DB each time, or is there some internal caching of the allowlists?
Allowlists are checked by LAPI when an alert is received (ie, we don't check allowlists each time we parse a log line, we let the log processor handle it as usual, and drop the alert in LAPI if an allowlist match)
LAPI queries the DB directly, so there's no delay between updating an allowlist and it working
we only cache the allowlists in the WAF, as it's running in the log processor, which does not have access to the DB.
In this case, they are refreshed every minute
Also
What's your use case for writing directly to the DB ?
cscli should be used to manage the content.
If you absolutely need to directly write to the DB, at least try to use the same functions as crowdsec (with https://pkg.go.dev/github.com/crowdsecurity/crowdsec@v1.6.9/pkg/database#Client.AddToAllowlist for example), but keep in mind we do not expect anyone to call those functions, so the signature might change at any time but at least everything will be created properly in the DBI have a database of users with IPs in another application. I can't track changes there one-by-one, I can only export the entire DB of IPs once an hour. The operations like
cscli allowlist add and cscli allowlist remove won't work in this case (unless I do a diff with the previous version of the exported DB, which I want to avoid). Having something like cscli allowlist import -f file would have been helpful, but these commands need to run on the LAPI and I run an LAPI inside a container in a cluster and doing exec into a running container in a cluster is cumbersome. An ideal solution would be a CLI that can import an entire list of IPs from a file/stdin, but that can be run in a separate job and interact with LAPI through HTTP, given a URL and an API key.I have created a Python script to import whitelists into the allowlist in the DB. https://github.com/AlexeyDemidov/crowdsec-allowlist
GitHub
GitHub - AlexeyDemidov/crowdsec-allowlist: A script to import a YAM...
A script to import a YAML whitelist into CrowdSec MySQL/MariaDB allowlist - AlexeyDemidov/crowdsec-allowlist
Resolving Direct manipulation of the allowlist in the database.
This has now been resolved. If you think this is a mistake please run
/unresolve