Skip to main content

I want to build a Logstash parser to handle CEF-format logs where the field keys resemble nested structures with array indices. The CEF format includes key-value pairs, which I can extract using:

 

kv {   source => "cef_event_attributes"   field_split => "|"   value_split => "="   target => "cef_fields" }

Here are a few example key-value pairs:

 

mitre1.enterprise.10.techniques.5.techniqueID = T1047 mitre1.enterprise.10.techniques.4.techniqueID = T1021 mitre1.enterprise.9.techniques.3.techniqueID = T1059 user = abc123

These are flattened field names. For example,
mitre1.enterprise.10.techniques.5.techniqueID represents a path-like structure and can be interpreted (conceptually) as:
mitre1[0].enterprise[10].techniques[5].techniqueID — though I don't need to construct a nested object, just treat the key as a full string.

✅ What I’m Looking For:

Is there any Logstash-native way to:

  • Dynamically iterate over all key-value pairs in cef_fields?

  • Skip a known field like user (which I can map directly as:
    "event.idm.read_only_udm.principal.user.first_name" => "%{[cef_fields][user]}")

  • For every other dynamic field, add it as an additional UDM field, for example:
    Key = mitre1.enterprise.10.techniques.5.techniqueID
    Value = T1047
    → Stored as:

 

{   "key": "mitre1.enterprise.10.techniques.5.techniqueID",   "value": {     "string_value": "T1047"   } }

Can this be done using only native Logstash plugins inside the filter {} block ?

Be the first to reply!

Reply