Parser extension Multiple Raw field to same UDM Field

Question

In a predefined parser, 2 different raw fields are parsed to same UDM field:

"var_target.resource.resource_subtype" => "%{protoPayload.request.machineType}"

"var_target.resource.resource_subtype" => "%{resource.type}"

When raw log has both the values (resource.type and protoPayload.request.machineType) target.resource.resource_subtype contains value of resource.type because it’s mapped later on in the parser.

How can we extract both the fields?

DanHansen · Accepted Answer

You should be able to write "%{protoPayload.request.machineType}" to a different UDM field in an extension. That should extract both fields to their assigned UDM identities.

ion_ · Answer

Dan is 100% right about how you can fix it, however, I think thats the wrong optimisation. If this is a bug in the default parser then you can get it fixed for all of us by making a support ticket and walking them through the issue.

If you are referring to GCP_CLOUDAUDIT, it looks like the checks happen quite close to each other and the fix would pretty much just be copy pasting them next to eachother and adding an if or elif. Heres what i'm seeing in my version of the GCP_CLOUDAUDIT parser:

  if [protoPayload][request][machineType] != "" {
    mutate {
      replace => {
        "var_target.resource.resource_subtype"
        => "%{protoPayload.request.machineType}"
      }
      on_error => "resource_type_not_present"
    }
  }
...
//Unrelated for loop
...
  if [resource][type] != "" {
    mutate {
      replace => {
        "var_target.resource.resource_subtype" => "%{resource.type}"
      }
      on_error => "resource_type_not_present"
    }
  }

There are tons of issues like this throughout the default parsers and the more responsibility we take for getting fixes integrated, the better the product gets for everyone.

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded