Kubernetes textPayload Split

Hi @ad9001 

There is no target.description field in UDM schema, you may use metadata.description instead.

Split function might may give you output in textPayload_array and considering no keys are given, consider checking by accessing values using fields like textPayload_array.0, textPayload_array.1 and so on. 

Hi @s_shubh 

Thank you for the suggestion, I tried adding path but had no luck. I tried adding json with array function for split column as well but this time I get an error message:

CBN Snippet:

 Based on the statedump, I do see the following:

In the first mutate block under if condition, I observed letter "a". Could you please try to remove it and run, if it's not a typo ? 

Also try using replace for description instead merge in mutate. 

Hi @s_shubh 

Adding a replace instead of a merge no longer shows error message however only shows the field.

UDM Output:

Is there a better way to split and add all the arrays into their own fields or field of arrays? I've include more info which may help from the UDM Output Error, Statedump, and Raw log (sanitized).

UDM Output Error:

 State-dump (sanitized):

Raw Log (Sanitized):

Parser Extension:

Hi,

Based on previous comments, I can see that "textPayload" contains multiple values; it's a list. Lists need to be mapped a to a repeated UDM field assuming you'd like to map all the contents of the list and you must use a for loop. I would suggest adding this data to the "additional" UDM field.

Hi @Rene_Figueroa and @s_shubh 

Thank you for the the suggestions, looks like the following parser below worked when setting the with a replace and "%{textPayload_array.1}" However now I run into issue where I need to split again from the equal sign and quote (=") to setup a KV where the "type" is the key and the value is "container_app_firewall_audit"

Also where sha'll I place the "For" loop within the parser I have below?

Output

Parser

Since all of this data is JSON, the JSON function should split up everything all the values in key-value pairs. Also, do you want to map all the content of "textPayload_array" to the UDM event? If so, then metadata.description won't do since it does not hold arrays. That's why I suggested using "additional" instead. 

Do you have a whole sample raw log we can review?

Hi @Rene_Figueroa 

Yes, I would like to map the entire array to UDM events with Key/Value pairs so I can generate a rule detection to create an alert based on a specific value on a key label (field).

I've tried adding "additional" based on the following below, but get the following error. I followed the guide here based on the documentation but none of the combinations work.

Parser Error: 

Parser Update with Additional:

Raw Log (Sanitized):

Hi, 

Our syntax supports "additional.string_value". You would need to map every key-value pair as following:

mutate {
replace => {
"textproto_array.key" => "%{keyvalue}"
"textproto_array.value.string_value" => "%{value}"
}
merge => {
"event.idm.read_only_udm.additional.fields" => "textproto"
}
}

You can enclosed the above in a for loop for all of your key-value pairs. Note that this will only work when the "textPayload" meets your parser criteria. 

I reviewed our default KUBERNETES_NODE parser and I see we already map "textpayload" in "about.labels", so you can write your rule to look for these UDM field already.