Skip to main content
Solved

Concatinating field from nested json by extending the parser

  • September 6, 2024
  • 4 replies
  • 40 views
S
Forum|alt.badge.img+2

Hi All,  Need to implement this urgently - tried all options including ruby / split etc but none of that works within google chronicle parser extension. 

I have a simple requirement to concatenate field value from recurring field as per below. Please suggest how this can be implemented in parser extension.  Input and expected output is as per below.. Thanks in advance.

Input:

{
"folders": [
{"resourceFolderDisplayName": "L1"},
{"resourceFolderDisplayName": "L2"},
{"resourceFolderDisplayName": "L3"}
]
}

Expected Output:

UDM.METADATA.RESOURCEPATH = L1/L2/L3

Best answer by AbdElHafez

This is a valid version, I could not find a resourcepath under metadata so I put in metadata.description

 

filter { json { source => "message" array_function => "split_columns" } statedump { "label" => "1"} mutate {replace => {"temp"=>""}} #Init the concat token for index1,item1 in folders { mutate {convert => {"index1"=>"string"}} mutate {replace => {"label1"=>""}} #Init the collector token mutate {replace => {"obj.val"=>"%{item1.resourceFolderDisplayName}"}} mutate {replace => {"label1"=>"%{obj.val}"}} mutate {merge => {"labels"=>"label1"}} #Init the collector token # Concat if [index1]=='0' {mutate {replace => {"temp"=>"%{label1}"}}} else {mutate {replace => {"temp"=>"%{temp}|%{label1}"}}} statedump { "label" => "2"} } mutate { replace => { "event.idm.read_only_udm.metadata.product_name" => "Upstream" "event.idm.read_only_udm.metadata.vendor_name" => "Upstream" "event.idm.read_only_udm.metadata.event_type" => "GENERIC_EVENT" "event.idm.read_only_udm.metadata.description" => "%{temp}" #Concat in Temp } } mutate { merge => { "@output" => "event" } } statedump { "label" => "3"} }

 

4 replies

AbdElHafez
Staff
Forum|alt.badge.img+12
  • AbdElHafez
  • Staff
  • September 9, 2024

Is the number of items in the folders list fixed as 3 items or is it variable ? and are the keys names "resourceFolderDisplayName" fixed ?

AbdElHafez
Staff
Forum|alt.badge.img+12
  • AbdElHafez
  • Staff
  • Answer
  • September 9, 2024

This is a valid version, I could not find a resourcepath under metadata so I put in metadata.description

 

filter { json { source => "message" array_function => "split_columns" } statedump { "label" => "1"} mutate {replace => {"temp"=>""}} #Init the concat token for index1,item1 in folders { mutate {convert => {"index1"=>"string"}} mutate {replace => {"label1"=>""}} #Init the collector token mutate {replace => {"obj.val"=>"%{item1.resourceFolderDisplayName}"}} mutate {replace => {"label1"=>"%{obj.val}"}} mutate {merge => {"labels"=>"label1"}} #Init the collector token # Concat if [index1]=='0' {mutate {replace => {"temp"=>"%{label1}"}}} else {mutate {replace => {"temp"=>"%{temp}|%{label1}"}}} statedump { "label" => "2"} } mutate { replace => { "event.idm.read_only_udm.metadata.product_name" => "Upstream" "event.idm.read_only_udm.metadata.vendor_name" => "Upstream" "event.idm.read_only_udm.metadata.event_type" => "GENERIC_EVENT" "event.idm.read_only_udm.metadata.description" => "%{temp}" #Concat in Temp } } mutate { merge => { "@output" => "event" } } statedump { "label" => "3"} }

 

    S
    Forum|alt.badge.img+2
    • swanand
    • Author
    • New Member
    • September 11, 2024

    This worked like a charm... Thanks a ton. 

    I came across one more issue - when trying to map the concatenation  result to the UDM field (which is mapped to metadata.description in the code you shared). Seems array's are overwritten and not appended in the extension - and I was not able to add this in any of the existing array fields.

    The parser i m extending is GCP SCC threat and since i can't use any array fields, ended up using this to store hierarchy folders for GCP project: 

        "event.idm.read_only_udm.src.process.file.full_path" => "%{temp}" #Concat in Temp
     
    Any thoughts on whether this is the right field or should i consider using any other please ?
     
    Thanks again !
    AbdElHafez
    Staff
    Forum|alt.badge.img+12
    • AbdElHafez
    • Staff
    • September 11, 2024

    Happy to know it worked.
    The concatenation token "temp" is a string field so it must be assigned to any string field, if you want to add/concatenate to an existing string field that is populated by the parser already, then you would to add the code block to the parser and you will have to maintain it with updates, as the token values do not persist between the parser and the parser extension unfortunately.

    Same goes for arrays, but you would need to properly format "temp" as an array or add a key value to it and append it to the labels in the main parser block.

    Terms & ConditionsCookie settingsAccessibility statement
    Privacy Policy Terms of Service Gen AI Policy Community Guidelines Cookie Settings

    © 2025 Google. All rights reserved.