Distribution mode cross region
I set up distribution mode with a single LAPI (server) and multiple Openresty bouncer + Appsec (agents). If the agent was in the same AZ region as server there is no issue. But if the agent was in difference AZ regions e.g. US-SG there was issue as bellow:
-crowsec.log (agent) :
time="2025-12-03T08:03:34Z" level=error msg="Error checking auth for API key: Head "http://51.xxx.xxx.xxx/v1/decisions/stream\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" name=CDN-WAF type=appsec
time="2025-12-03T08:03:34Z" level=error msg="Unauthorized request from '127.0.0.1:38654' (real IP = xx.xxx.xxx) invalid API key" name=CDN-WAF type=appsec
- bouncer log: 2025/12/03 08:03:34 [error] 9726#9726: *2 [lua] crowdsec.lua:560: AppSecCheck(): Unauthenticated request to APPSEC, client: xxx.xxx.xxx I tried to check any firewall or network restriction but there is no. Could you please help guide? Thank
- bouncer log: 2025/12/03 08:03:34 [error] 9726#9726: *2 [lua] crowdsec.lua:560: AppSecCheck(): Unauthenticated request to APPSEC, client: xxx.xxx.xxx I tried to check any firewall or network restriction but there is no. Could you please help guide? Thank
8 Replies
Important Information
This post has been marked as resolved. If this is a mistake please press the red button below or type
/unresolve© Created By WhyAydan for CrowdSec ❤️
its most likely the timeout we put in the request from appsec -> lapi to check if the api key used provided by the bouncer is valid (we must do this to ensure the request from bouncer is actually valid and not just a random http request if exposed to the internet).
but by default we only allow for 200ms https://github.com/crowdsecurity/crowdsec/blob/65b7465ef1c738ab4776e1f762323657bb429248/pkg/acquisition/modules/appsec/config.go#L230-L232
so if the request will take longer than it will timeout and prevent it from communicating.
Oh I see thank you.
Is there any possible to change this value?
Not currently
Do you recomnent for cross-region setup. I've tried with cloud local network (vrack) but timeout still happen.
noted issue here for tracking https://github.com/crowdsecurity/crowdsec/issues/4102
ideally you should create clusters per region, yes each cluster will have their own remediations. Or you used a shared database between regions and each region gets their own LAPI thats connected to the same database. https://docs.crowdsec.net/docs/next/local_api/database
but if you go the per cluster remediations, thats why we created the remediation sync feature in the console to ease these types of deployments.
Thank you. I'll try.
Resolving Distribution mode cross region
This has now been resolved. If you think this is a mistake please run
/unresolve