Skip to main content

Heres my attempt at parser extension:

 

filter {mutate {
replace => {
"principal.user.user_display_name" => "UNKNOWN"
}
}
grok {
match => { "message" => ["ruser=(?<user>[^ ]+)","user (?<user>[^:]+)"] }
overwrite => ["principal.user.user_display_name"]
on_error=> "User update failed"
}

statedump {}
mutate {merge => {"@output" => "event"}}
}

 

I get the following error

 

generic::unknown: pipeline.ParseLogEntry failed: LOG_PARSING_CBN_ERROR: "generic::invalid_argument: pipeline failed: filter mutate (3) failed: merge failure: merge source field \\"event\\" must not be empty (try using replace to provide the value before calling merge)"

 

 Can someone help me understand whats wrong? A sample log entry would be as follows:

 

<85>Nov 4 12:07:00 system-name sudo[2700721]: pam_sss(sudo:auth): authentication failure; logname=johnf uid=5354 euid=0 tty=/dev/pts/1 ruser=johnf rhost= user=johnf

 

 

I'd recommend using KV extraction for logs that have key/value pairs (much easier than Grok). I updated the parser extension to use that, and also included some missing data from the target field that was causing the issue. 


New Parser


filter {
# initialize the token
mutate {
replace => {
"user" => ""
}
}

kv {
source => "message"
field_split => " "
value_split => "="
}
#statedump{}

if [user] != "" {
mutate {
replace => {
"event.idm.read_only_udm.principal.user.user_display_name" => "%{user}"
}
}
}

mutate {
merge => {
"@output" => "event"
}
}
}

Will yield this UDM event


metadata.event_timestamp : "2024-11-04T12:07:00Z"
metadata.event_type : "STATUS_UPDATE"
metadata.vendor_name : "Linux"
metadata.product_name : "AuditD"
metadata.product_event_type : "sudo"
metadata.description : "pam_sss(sudo:auth): authentication failure; logname=johnf uid=5354 euid=0 tty=/dev/pts/1 ruser=johnf rhost= user=johnf"
metadata.log_type : "AUDITD"
principal.hostname : "system-name"
principal.user.user_display_name : "johnf"
intermediary[0].hostname : "system-name"

Hope this helps!


-mike

Thanks Mike for taking a look into this.
In hind sight I should have shared that there were other log entries from the smae source that doesn't follow the KV pattern.

Is there a way to just look at sudo events

 

<86>Mar 5 12:25:43 system-name sshd[91732]: Did not receive identification string from 192.168.1.1 port 34876
<85>Nov 4 13:13:28 system-name sudo[3891200]: pam_sss(sudo:auth): received for user johnf: 7 (Authentication failure)

 



But the parser works perfectly well for the given log. 


I ended up with 

filter {
# initialize the token
# Decode Unicode escape sequences (e.g., \\u003c85\\u003e -> <85>)

# statedump{}
if ([message] =~ /sudo:auth/ and [message] =~ /=/)
{
mutate {
replace => {
"user" => ""
}
}
kv {
source => "message"
field_split => " "
value_split => "="
}

if [user] != "" {
mutate {
replace => {
"event.idm.read_only_udm.principal.user.user_display_name" => "%{user}"
}
}
}
mutate {
merge => {
"@output" => "event"
}
}
}

# statedump{}
}

Reply


Terms & ConditionsCookie settings
Privacy Policy Terms of Service Gen AI Policy Community Guidelines Cookie Settings

© 2025 Google. All rights reserved.