+4I have logs in JSON format. Single log line has multiple single json objects delimited by new line. here each json object is one log. This is for AWS_ROUTE_53 parser.
{
"field1" : "value",
"filed2" : [ { "field3": "value" } ]
}
{
"field1" : "value",
"filed2" : [ { "field3": "value" } ]
}
+5Hello,
I was actually working on this with someone just last week. I'm not sure why AWS is formatting logs in this new and odd way, but the work around is a quick addition to the AWS_ROUTE_53 parser.
You will need to make a custom version of the parser to do this.
Near the top of the parser, inside of the filter block, probably after the first mutate replace, you will want to add a gsub (global substitute) block. In this block you will look for every occurrence of }{ and replace it with },{
It should look something like the below.
mutate {
gsub => ["message", "}{", "},{"]
}
Logic further down in the parser will convert this into a proper array and should parse the logs successfully. Please test this since this is just an off the cuff response and I am not certain that you're seeing the same thing I was working on previously.
I would also strongly encourage you to open a support ticket requesting a parser update for this log type if you are receiving logs in the format you specified.
+4I have already made this change to the parser. It is able to parse all the fields but only for one random event out of multiple events present in the log line. Remaining events are not converting to udm events. I have compiled it with cbn tool and output has multiple UDM events. We are able to see only one udm event when we submit the same parser in chronicle instance.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.