C
CrowdSec•2w ago
alendroit

data directive in scenario do not load local file in memory

Hello, I created a dummy scenario to import local data file (with a list of IPs) but it seems to not be loaded in memory. I use the imported file with the File() function in filters section of profiles.yaml: 
filters:
 -  Alert.Remediation == true && Alert.GetScope() == "Ip" && Alert.GetValue() in File("importtrustedip.txt")
filters:
 -  Alert.Remediation == true && Alert.GetScope() == "Ip" && Alert.GetValue() in File("importtrustedip.txt")
It worked for a couple of months but I realized it do not work anymore. Here is the dummy scenario: 
name: gcustom/importtrustedip
description: "Mandatory to notify when when trusted ip get banned"
data:
  - dest_file: importtrustedip.txt
    type: string
name: gcustom/importtrustedip
description: "Mandatory to notify when when trusted ip get banned"
data:
  - dest_file: importtrustedip.txt
    type: string
Here is the debug log: 
time="2025-08-12T12:50:06+02:00" level=error msg="file 'importtrustedip.txt' (type:string) not found in expr library"
time="2025-08-12T12:50:06+02:00" level=error msg="expr library : (map[string][]string) {\n}\n"
time="2025-08-12T12:50:06+02:00" level=debug msg="dbg(result=false): Alert.Remediation == true && Alert.GetScope() == \"Ip\" && Alert.GetValue() in File(\"importtrustedip.txt\")" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [true] " name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [== true] true == true -> [true]" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [&&] AND -> true" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [== \"Ip\"] \"Ip\" == \"Ip\" -> [true]" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [&&] AND -> true" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [File(\"importtrustedip.txt\")] File(\"importtrustedip.txt\") = []" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [in File(\"importtrustedip.txt\")] \"135.233.112.24\" in [] -> false" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg="Profile ban_meraki filter is unsuccessful" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=error msg="file 'importtrustedip.txt' (type:string) not found in expr library"
time="2025-08-12T12:50:06+02:00" level=error msg="expr library : (map[string][]string) {\n}\n"
time="2025-08-12T12:50:06+02:00" level=debug msg="dbg(result=false): Alert.Remediation == true && Alert.GetScope() == \"Ip\" && Alert.GetValue() in File(\"importtrustedip.txt\")" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [true] " name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [== true] true == true -> [true]" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [&&] AND -> true" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [== \"Ip\"] \"Ip\" == \"Ip\" -> [true]" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [&&] AND -> true" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [File(\"importtrustedip.txt\")] File(\"importtrustedip.txt\") = []" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg=" [in File(\"importtrustedip.txt\")] \"135.233.112.24\" in [] -> false" name=ban_meraki type=profile
time="2025-08-12T12:50:06+02:00" level=debug msg="Profile ban_meraki filter is unsuccessful" name=ban_meraki type=profile
13 Replies
CrowdSec
CrowdSec•2w ago
Important Information
This post has been marked as resolved. If this is a mistake please press the red button below or type /unresolve
© Created By WhyAydan for CrowdSec ❤️
iiamloz
iiamloz•2w ago
the File at profiles level was really a hack workaround, in the latest version we introduced cscli allowlist function you may want to switch over to this for better support.
iiamloz
iiamloz•2w ago
alendroit
alendroitOP•2w ago
Actually I use this filter in a profile to monitor IPs in the file. If one of IPs in this file get a decision, it gets ban and send a mail notification. That way I can receive an alert if trusted ip get banned and I can remediate (I don't wand to allow blindly these IPs) So cscli allowlist seems not to be helpful in that case .. but thank you i didn't know it has been released !
iiamloz
iiamloz•2w ago
Okay makes sense, so importtrustedip.txt exists in the data directory and has content?
alendroit
alendroitOP•2w ago
yep, present in /var/lib/crowdsec/data/ and has list of ips (one per line)
iiamloz
iiamloz•2w ago
Okay can you add debug: true to your scenario and restart crowdsec to see if it loads into memory?
alendroit
alendroitOP•2w ago
Here it is

message.txt

iiamloz
iiamloz•2w ago
hmmm there is nothing best probbaly to switch the whole of crowdsec into debug
alendroit
alendroitOP•2w ago
I set crowdsec in debug, the "Returning list of scenarios :" do not return the scenario.. (but return crowdsecurity and other custom scenarios) Maybe I should add some dummy section in the import data scenario ? Or add in an existing working scenario ?
iiamloz
iiamloz•2w ago
most likely because it doesnt have any properties its not a "full scenario" but yeah you can add it to the bottom of another scenario
alendroit
alendroitOP•2w ago
I find out .. I hardened my LAPI weeks ago, by commenting: #crowdsec_service: # #console_context_path: /etc/crowdsec/console/context.yaml #acquisition_path: /etc/crowdsec/acquis.yaml #acquisition_dir: /etc/crowdsec/acquis.d #parser_routines: 1 The server running LAPI do not need to parse logs, so I deactivate this section. But, side effect it do not load scenario/parsers etc After uncomment those lines, it works again.
CrowdSec
CrowdSec•2w ago
Resolving data directive in scenario do not load local file in memory This has now been resolved. If you think this is a mistake please run /unresolve

Did you find this page helpful?