npmplus parsing

i recently migrated from npm to npmplus and it appears the logs arent being parsed?

33 Replies

Important Information

Thank you for getting in touch with your support request. To expedite a swift resolution, could you kindly provide the following information? Rest assured, we will respond promptly, and we greatly appreciate your patience. While you wait, please check the links below to see if this issue has been previously addressed. If you have managed to resolve it, please use run the command /resolve or press the green resolve button below.

Log Files

If you possess any log files that you believe could be beneficial, please include them at this time. By default, CrowdSec logs to /var/log/, where you will discover a corresponding log file for each component.

Guide Followed (CrowdSec Official)

If you have diligently followed one of our guides and hit a roadblock, please share the guide with us. This will help us assess if any adjustments are necessary to assist you further.

Screenshots

Please forward any screenshots depicting errors you encounter. Your visuals will provide us with a clear view of the issues you are facing.

b_d0nOP•8mo ago

Shows the NPM is tainted and i can’t figure out why it’s not liking it

iiamloz•8mo ago

if you run cscli collections inspect ZoeyVid/npmplus it should inform why it is tainted I can see from the current files it only see the base nginx access log, did you configure all paths like zoeyvid stated in their repo?

b_d0nOP•8mo ago

Yes. Also it’s saying tainted by base-http-scenario

b_d0nOP•8mo ago

iiamloz•8mo ago

okay did you modify or remove some scanerios? then you can follow the chain cscli collections inspect crowdsecurity/base-http-scenarios

b_d0nOP•8mo ago

No but I am coming from LePresidente NPM is there something I need to change

iiamloz•8mo ago

remediation and scenario are not tied so no.

b_d0nOP•8mo ago

/ # cscli collections inspect crowdsecurity/base-http-scenarios
type: collections
name: crowdsecurity/base-http-scenarios
file_name: base-http-scenarios.yaml
description: 'http common : scanners detection'
author: crowdsecurity
path: collections/crowdsecurity/base-http-scenarios.yaml
version: "1.0"
parsers:
  - crowdsecurity/http-logs
scenarios:
  - crowdsecurity/http-crawl-non_statics
  - crowdsecurity/http-probing
  - crowdsecurity/http-bad-user-agent
  - crowdsecurity/http-path-traversal-probing
  - crowdsecurity/http-sensitive-files
  - crowdsecurity/http-sqli-probing
  - crowdsecurity/http-xss-probing
  - crowdsecurity/http-backdoors-attempts
  - ltsich/http-w00tw00t
  - crowdsecurity/http-generic-bf
  - crowdsecurity/http-open-proxy
  - crowdsecurity/http-admin-interface-probing
  - crowdsecurity/http-wordpress-scan
  - crowdsecurity/http-cve-probing
collections:
  - crowdsecurity/http-cve
contexts:
  - crowdsecurity/http_base
local_version: "1.0"
local_hash: b0c860f48e5d24ba5e278523e5b1652ae370228eaadcc809db1f5b3463c8ce46
installed: false
downloaded: true
uptodate: true
tainted: false
belongs_to_collections:
  - crowdsecurity/apache2
  - crowdsecurity/apiscp
  - crowdsecurity/apiscp
  - crowdsecurity/aws-cloudfront
  - crowdsecurity/caddy
  - crowdsecurity/exchange
  - crowdsecurity/fastly
  - crowdsecurity/haproxy
  - crowdsecurity/iis
  - crowdsecurity/litespeed
  - crowdsecurity/nginx
  - crowdsecurity/nginx-proxy-manager
  - crowdsecurity/pfsense
  - crowdsecurity/supabase-compose
  - crowdsecurity/traefik
  - crowdsecurity/whm
  - ZoeyVid/npmplus
local: false

/ # cscli collections inspect crowdsecurity/base-http-scenarios
type: collections
name: crowdsecurity/base-http-scenarios
file_name: base-http-scenarios.yaml
description: 'http common : scanners detection'
author: crowdsecurity
path: collections/crowdsecurity/base-http-scenarios.yaml
version: "1.0"
parsers:
  - crowdsecurity/http-logs
scenarios:
  - crowdsecurity/http-crawl-non_statics
  - crowdsecurity/http-probing
  - crowdsecurity/http-bad-user-agent
  - crowdsecurity/http-path-traversal-probing
  - crowdsecurity/http-sensitive-files
  - crowdsecurity/http-sqli-probing
  - crowdsecurity/http-xss-probing
  - crowdsecurity/http-backdoors-attempts
  - ltsich/http-w00tw00t
  - crowdsecurity/http-generic-bf
  - crowdsecurity/http-open-proxy
  - crowdsecurity/http-admin-interface-probing
  - crowdsecurity/http-wordpress-scan
  - crowdsecurity/http-cve-probing
collections:
  - crowdsecurity/http-cve
contexts:
  - crowdsecurity/http_base
local_version: "1.0"
local_hash: b0c860f48e5d24ba5e278523e5b1652ae370228eaadcc809db1f5b3463c8ce46
installed: false
downloaded: true
uptodate: true
tainted: false
belongs_to_collections:
  - crowdsecurity/apache2
  - crowdsecurity/apiscp
  - crowdsecurity/apiscp
  - crowdsecurity/aws-cloudfront
  - crowdsecurity/caddy
  - crowdsecurity/exchange
  - crowdsecurity/fastly
  - crowdsecurity/haproxy
  - crowdsecurity/iis
  - crowdsecurity/litespeed
  - crowdsecurity/nginx
  - crowdsecurity/nginx-proxy-manager
  - crowdsecurity/pfsense
  - crowdsecurity/supabase-compose
  - crowdsecurity/traefik
  - crowdsecurity/whm
  - ZoeyVid/npmplus
local: false

iiamloz•8mo ago

😕

b_d0nOP•8mo ago

...why

iiamloz•8mo ago

you can run cscli collections update ZoeyVid/npmplus --force and it will force up the collection, but it may updat esome files you may want to keep so run cscli parsers list to see if any tained ones you want such as whitelists

b_d0nOP•8mo ago

unknown flag --force would it be -f

iiamloz•8mo ago

https://tenor.com/view/fall-stairs-gif-12539685

Tenor

iiamloz•8mo ago

okay use install instead of update

b_d0nOP•8mo ago

😂 ok its no longer tainted but now does it work lol

b_d0nOP•8mo ago

iiamloz•8mo ago

what the configuration set? the acquis.d

b_d0nOP•8mo ago

yes acquis.d/npmplus.yaml

filenames:
  - /data/nginx/*.log
labels:
  type: npmplus
---
source: docker
container_name:
 - npmplus
labels:
  type: npmplus
---
source: docker
container_name:
 - npmplus
labels:
  type: modsecurity
---
listen_addr: 0.0.0.0:7422
appsec_config: crowdsecurity/appsec-default
name: appsec
source: appsec
labels:
  type: appsec
# if you use openappsec you can enable this
#---
#source: docker
#container_name:
# - openappsec-agent
#labels:
#  type: openappsec

filenames:
  - /data/nginx/*.log
labels:
  type: npmplus
---
source: docker
container_name:
 - npmplus
labels:
  type: npmplus
---
source: docker
container_name:
 - npmplus
labels:
  type: modsecurity
---
listen_addr: 0.0.0.0:7422
appsec_config: crowdsecurity/appsec-default
name: appsec
source: appsec
labels:
  type: appsec
# if you use openappsec you can enable this
#---
#source: docker
#container_name:
# - openappsec-agent
#labels:
#  type: openappsec

but i also have this in the root crowdsec foler from the lepresedente appdata/crowdsec/acquis.yaml f

ilenames:
  - /var/log/nginx/*.log
#this is not a syslog log, indicate which kind of logs it is
labels:
  type: nginx-proxy-manager
---
filenames:
  - /var/log/authentik.log
labels:
  type: authentik
---
source: docker
container_name:
 - authentik
labels:
  type: authentik

ilenames:
  - /var/log/nginx/*.log
#this is not a syslog log, indicate which kind of logs it is
labels:
  type: nginx-proxy-manager
---
filenames:
  - /var/log/authentik.log
labels:
  type: authentik
---
source: docker
container_name:
 - authentik
labels:
  type: authentik

iiamloz•8mo ago

hmm but from the metrics, I dont see /data/nginx/ in it?

b_d0nOP•8mo ago

is the path wrong...

iiamloz•8mo ago

is crowdsec in a container or bare metal?

b_d0nOP•8mo ago

container

iiamloz•8mo ago

and you mounted the npmplus volume / path to crowdsec?

b_d0nOP•8mo ago

iiamloz•8mo ago

#    volumes:
#      - "/opt/crowdsec/conf:/etc/crowdsec"
#      - "/opt/crowdsec/data:/var/lib/crowdsec/data"
#      - "/opt/npmplus/nginx:/opt/npmplus/nginx:ro"
#      - "/var/run/docker.sock:/var/run/docker.sock:ro"

#    volumes:
#      - "/opt/crowdsec/conf:/etc/crowdsec"
#      - "/opt/crowdsec/data:/var/lib/crowdsec/data"
#      - "/opt/npmplus/nginx:/opt/npmplus/nginx:ro"
#      - "/var/run/docker.sock:/var/run/docker.sock:ro"

b_d0nOP•8mo ago

i have docker_host set in crowdsec

iiamloz•8mo ago

and you no longer mounting any other files to /var/log/nginx? so i would remove:

filenames:
  - /var/log/nginx/*.log
#this is not a syslog log, indicate which kind of logs it is
labels:
  type: nginx-proxy-manager
---

filenames:
  - /var/log/nginx/*.log
#this is not a syslog log, indicate which kind of logs it is
labels:
  type: nginx-proxy-manager
---

and then update npmplus

b_d0nOP•8mo ago

you think its interfering?

iiamloz•8mo ago

I think the type wrong then in the acquis.d/npmplus.yaml add /var/log/nginx/*.log to the filenames

b_d0nOP•8mo ago

so the acquis.yaml was interfering with the npmplus.yaml

iiamloz•8mo ago

kind off, the type set in the acquis.yaml is the default nginx-proxy-manager but npmplus has it own format

b_d0nOP•8mo ago

ok it appears to be parsing the logs file:/var/log/nginx/access.log but trying bf on radarr from a non local network isnt invoking a ban 😦 also if im giving access to the docker socket then why doesnt metrics show all the containers? do i need too manually list them in the acquis.yaml?

Gaming

Programming

npmplus parsing

Did you find this page helpful?