CrowdSec 1.7.3 parser: evt.StrTime not being set when using microsecond timestamps
Hi everyone,
I'm currently working on a custom parser and I'm stuck on an issue.
The logs unfortunately contain timestamps with microseconds.
I'm running CrowdSec in Docker, version 1.7.3.
Here’s my current parser code:
name: zoraxy-logs
description: "Zoraxy HTTP router logs (robust for hostnames + microseconds)"
filter: "true"
onsuccess: next_stage
nodes:
- grok:
pattern: '[(?P<timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}(?:.%{INT})?)]\s+[router:(?:host-http|whitelist|blacklist|subdomain-http|host-websocket|vdir-http|vdir-websocket|redirect|root-no_resp)]\s+[origin:(?P<target_server>%{IPORHOST}|[A-Za-z0-9.-]+)(?::%{NUMBER:target_port})?]\s+[client:\s%{IPORHOST:remote_addr}]\s+[useragent:\s%{GREEDYDATA:http_user_agent}]\s*%{WORD:verb}\s+%{DATA:request}\s+%{NUMBER:status}'
apply_on: message
statics:
# Literal value, not expression – works in 1.7.3
- target: evt.StrTime
value: "{{.Parsed.timestamp}}"
- meta: log_type
value: http_access-log
- meta: service
value: http
- meta: source
value: zoraxy
- meta: program
value: zoraxy
- meta: source_ip
expression: evt.Parsed.remote_addr
- meta: http_status
expression: evt.Parsed.status
- meta: http_path
expression: evt.Parsed.request
- meta: http_verb
expression: evt.Parsed.verb
- meta: http_user_agent
expression: evt.Parsed.http_user_agent
- meta: target_fqdn
expression: evt.Parsed.target_server
The error I get is:
update evt.StrTime : -> {{.Parsed.timestamp}}
Has anyone run into this before or found a clean way to handle it?
9 Replies
Important Information
Thank you for getting in touch with your support request. To expedite a swift resolution, could you kindly provide the following information? Rest assured, we will respond promptly, and we greatly appreciate your patience. While you wait, please check the links below to see if this issue has been previously addressed. If you have managed to resolve it, please use run the command
/resolve or press the green resolve button below.Log Files
If you possess any log files that you believe could be beneficial, please include them at this time. By default, CrowdSec logs to /var/log/, where you will discover a corresponding log file for each component.
Guide Followed (CrowdSec Official)
If you have diligently followed one of our guides and hit a roadblock, please share the guide with us. This will help us assess if any adjustments are necessary to assist you further.
Screenshots
Please forward any screenshots depicting errors you encounter. Your visuals will provide us with a clear view of the issues you are facing.
© Created By WhyAydan for CrowdSec ❤️
thats because
value means set it to this string, you should use expression instead as shown in the other variables you pasted.
might also be useful if your using AI to also say it in the message as then we know if your actually reading the documentation.Thanks for the hint — I actually already tried using expression: instead of value: for evt.StrTime.
However, in CrowdSec 1.7.3 this doesn’t work when the parser runs in the s01-parse stage, because evt.StrTime is read-only there.
CrowdSec ignores the assignment silently — you won’t get an error, but the field remains unset.
That’s why the log still shows:
update evt.StrTime : -> {{.Parsed.timestamp}}
The only way to correctly set evt.StrTime in 1.7.x is to do it in the s00-raw stage, before the event time is finalized.
That’s also how the built-in non-syslog parser handles timestamps.
So it’s not about the syntax (value vs expression),
but about where in the pipeline the timestamp is assigned.
if CrowdSec silently ignore it, it because
.Parsed.timestamp is empty
I didnt actually see your pattern is not using grok syntax please ask your AI to review other implentations in other parsers to understand how to do it correctly, otherwise it doesnt know unless you pre prompt it by saying its grok instead of regex.
please please actually try instead of just using an AI, and replying with an AI come on.....
im not agaisnt using AI and we updated our contrib guide to allow it, but its only as smart as the prompt you give it.
so if you dont know how crowdsec works or at least ask it to go looking how to do it, it will just give you crappy results.
I use it regularly for small changes on the Hub, and recently the local memory on my machine (cursor) has managed to actually get quite good at generating useful items. Plus we are already working heavily on the MCP so once that also has parser support hopefully AI will stop generating bad stuff 😄Sorry, CrowdSec is still new territory for me. Without AI, I wouldn’t have come this far.
You seem to know your stuff – could you maybe show me what the parser for my config should look like?
Yeah, I’ve noticed that too – the AI isn’t exactly the brightest. It keeps making the same mistakes, even though it’s already been told which CrowdSec version is being used.
Can you paste an example log line or send it to me via DM's? (you can redact the IP by changing it to an internal one)
Yeah, just a moment.
also to ask I guess you tried this first? https://app.crowdsec.net/hub/author/Raithmir/log-parsers/zoraxy-logs via the collection: https://app.crowdsec.net/hub/author/Raithmir/collections/zoraxy
yes
The AI used it and customized it for me.