CrowdSec•2mo ago

firewall bouncer stops grabbing new decisions after a while

I've been having some intermittent issues with the CrowdSec iptables bouncer where it'll stop bouncing after a few days. When I restart the bouncer everything works fine, but after a while it just stops bouncing. I don't see any errors in the log files that give a hint as to what might be the problem and I can clearly see that it's querying the LAPI with no problem, so the issue has to be with the bouncer itself. this is my config file:

mode: iptables
update_frequency: 1s
log_mode: file
log_dir: /var/log/
log_level: info
log_compression: true
log_max_size: 100
log_max_backups: 3
log_max_age: 30
api_url: <redacted>
api_key: <redacted>
insecure_skip_verify: false
disable_ipv6: false
deny_action: DROP
deny_log: false
supported_decisions_types:
  - ban
#to change log prefix
#deny_log_prefix: "crowdsec: "
#to change the blacklists name
blacklists_ipv4: crowdsec-blacklists
blacklists_ipv6: crowdsec6-blacklists
#type of ipset to use
ipset_type: nethash
#if present, insert rule in those chains
iptables_chains:
  - INPUT
  - FORWARD
#  - DOCKER-USER

## nftables
nftables:
  ipv4:
    enabled: true
    set-only: false
    table: crowdsec
    chain: crowdsec-chain
    priority: -10
  ipv6:
    enabled: true
    set-only: false
    table: crowdsec6
    chain: crowdsec6-chain
    priority: -10

nftables_hooks:
  - input
  - forward

# packet filter
pf:
  # an empty string disables the anchor
  anchor_name: ""

prometheus:
  enabled: false
  listen_addr: 127.0.0.1
  listen_port: 60601

mode: iptables
update_frequency: 1s
log_mode: file
log_dir: /var/log/
log_level: info
log_compression: true
log_max_size: 100
log_max_backups: 3
log_max_age: 30
api_url: <redacted>
api_key: <redacted>
insecure_skip_verify: false
disable_ipv6: false
deny_action: DROP
deny_log: false
supported_decisions_types:
  - ban
#to change log prefix
#deny_log_prefix: "crowdsec: "
#to change the blacklists name
blacklists_ipv4: crowdsec-blacklists
blacklists_ipv6: crowdsec6-blacklists
#type of ipset to use
ipset_type: nethash
#if present, insert rule in those chains
iptables_chains:
  - INPUT
  - FORWARD
#  - DOCKER-USER

## nftables
nftables:
  ipv4:
    enabled: true
    set-only: false
    table: crowdsec
    chain: crowdsec-chain
    priority: -10
  ipv6:
    enabled: true
    set-only: false
    table: crowdsec6
    chain: crowdsec6-chain
    priority: -10

nftables_hooks:
  - input
  - forward

# packet filter
pf:
  # an empty string disables the anchor
  anchor_name: ""

prometheus:
  enabled: false
  listen_addr: 127.0.0.1
  listen_port: 60601

8 Replies

CrowdSec•2mo ago

Important Information

Thank you for getting in touch with your support request. To expedite a swift resolution, could you kindly provide the following information? Rest assured, we will respond promptly, and we greatly appreciate your patience. While you wait, please check the links below to see if this issue has been previously addressed. If you have managed to resolve it, please use run the command /resolve or press the green resolve button below.

Log Files

If you possess any log files that you believe could be beneficial, please include them at this time. By default, CrowdSec logs to /var/log/, where you will discover a corresponding log file for each component.

Guide Followed (CrowdSec Official)

If you have diligently followed one of our guides and hit a roadblock, please share the guide with us. This will help us assess if any adjustments are necessary to assist you further.

Screenshots

Please forward any screenshots depicting errors you encounter. Your visuals will provide us with a clear view of the issues you are facing.

GNU Plus Windows UserOP•2mo ago

CrowdSec LAPI version:

version: v1.6.11-debian-pragmatic-amd64-d64ee2ae
Codename: alphaga
BuildDate: 2025-07-22_13:17:08
GoVersion: 1.24.4
Platform: linux
libre2: C++
User-Agent: crowdsec/v1.6.11-debian-pragmatic-amd64-d64ee2ae-linux
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog
Excluded components: cscli_setup

version: v1.6.11-debian-pragmatic-amd64-d64ee2ae
Codename: alphaga
BuildDate: 2025-07-22_13:17:08
GoVersion: 1.24.4
Platform: linux
libre2: C++
User-Agent: crowdsec/v1.6.11-debian-pragmatic-amd64-d64ee2ae-linux
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0
Built-in optional components: datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog
Excluded components: cscli_setup

Bouncer version:

version: v0.0.34-debian-pragmatic-amd64-4144555453620958398aee64253dfd90bbc1f698
BuildDate: 2025-08-04_10:04:33
GoVersion: 1.24.5
Platform: linux

version: v0.0.34-debian-pragmatic-amd64-4144555453620958398aee64253dfd90bbc1f698
BuildDate: 2025-08-04_10:04:33
GoVersion: 1.24.5
Platform: linux

OS: Ubuntu 24.04 LTS I do have some hardened kernel parameters but I doubt this is the issue as I've had them hardened for a long time, the firewall bouncer used to work fine but I guess something has changed. Posting them here just in case:

kernel.io_uring_disabled = 1
kernel.io_uring_group    = 1001
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.*.send_redirects = 0
net.ipv4.conf.*.accept_redirects = 0
net.ipv6.conf.*.accept_redirects = 0
net.ipv6.conf.all.use_tempaddr=2
net.ipv6.conf.default.use_tempaddr=2
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
net.core.bpf_jit_harden = 2
kernel.unprivileged_bpf_disabled = 1
kernel.sysrq = 0
kernel.perf_event_paranoid = 3
kernel.kptr_restrict = 2
kernel.dmesg_restrict = 1
fs.binfmt_misc.status = 0
fs.suid_dumpable = 0
fs.protected_regular = 2
fs.protected_fifos = 2
dev.tty.ldisc_autoload = 0
kernel.printk = 3 3 3 3
kernel.core_pattern = |/bin/false
kernel.core_uses_pid                      = 1
net.ipv4.conf.default.accept_source_route = 0
vm.swappiness = 1
kernel.randomize_va_space = 2
net.ipv4.conf.all.secure_redirects     = 0
net.ipv4.conf.default.secure_redirects = 0
net.ipv6.conf.all.accept_ra     = 0
net.ipv6.conf.default.accept_ra = 0
kernel.yama.ptrace_scope                         = 3
vm.unprivileged_userfaultfd                      = 0
vm.mmap_rnd_bits                                 = 32
vm.mmap_rnd_compat_bits                          = 16
kernel.kexec_load_disabled                       = 1
kernel.io_uring_disabled                         = 2
kernel.apparmor_restrict_unprivileged_unconfined = 1

kernel.io_uring_disabled = 1
kernel.io_uring_group    = 1001
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.*.send_redirects = 0
net.ipv4.conf.*.accept_redirects = 0
net.ipv6.conf.*.accept_redirects = 0
net.ipv6.conf.all.use_tempaddr=2
net.ipv6.conf.default.use_tempaddr=2
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
net.core.bpf_jit_harden = 2
kernel.unprivileged_bpf_disabled = 1
kernel.sysrq = 0
kernel.perf_event_paranoid = 3
kernel.kptr_restrict = 2
kernel.dmesg_restrict = 1
fs.binfmt_misc.status = 0
fs.suid_dumpable = 0
fs.protected_regular = 2
fs.protected_fifos = 2
dev.tty.ldisc_autoload = 0
kernel.printk = 3 3 3 3
kernel.core_pattern = |/bin/false
kernel.core_uses_pid                      = 1
net.ipv4.conf.default.accept_source_route = 0
vm.swappiness = 1
kernel.randomize_va_space = 2
net.ipv4.conf.all.secure_redirects     = 0
net.ipv4.conf.default.secure_redirects = 0
net.ipv6.conf.all.accept_ra     = 0
net.ipv6.conf.default.accept_ra = 0
kernel.yama.ptrace_scope                         = 3
vm.unprivileged_userfaultfd                      = 0
vm.mmap_rnd_bits                                 = 32
vm.mmap_rnd_compat_bits                          = 16
kernel.kexec_load_disabled                       = 1
kernel.io_uring_disabled                         = 2
kernel.apparmor_restrict_unprivileged_unconfined = 1

_KaszpiR_•2mo ago

Any messages in kernel log or dmesg?

GNU Plus Windows UserOP•2mo ago

blotus•2mo ago

Do you have anything that would flush/remove/update/.... the firewall rules ? Do you still see the rules when the bouncer seems to be broken ?

GNU Plus Windows UserOP•2mo ago

no, the only thing touching iptables is the firewall bouncer. I manage my custom firewall rules within the hypervisor which doesn't touch anything inside the VM I'm pretty sure it's still there but I'll have to double check. I think the issue is somehow related to adding new decisions if I go by what's in the logs

blotus•2mo ago

If we cannot add an ip to the set, you should have errors in your logs (the most common way this can happen is if you have more decisions than the configured set size). But If you don't see anything in your logs, I fear you'll need to run the bouncer in debug mode until it happens again

GNU Plus Windows UserOP•3w ago

ugh, couldn't reproduce for 5 days. I think it's fixed and then this issue pops up a bit later After a bit of testing, and reproducing the issue. I can find absolutely nothing of interest in the debug log, I can see the bouncer is able to query the LAPI just fine. I'm wondering if this could happen if the LAPI is being overloaded, I'm currently experiencing issues because of the performance issues when importing too many blocklists. I'm looking at ways to reduce the load atm

Gaming

Programming

firewall bouncer stops grabbing new decisions after a while

Did you find this page helpful?