+2Hi,
I'm looking to create a parser extension for the default GCP Loadbalancer one, since it currently doesn't parse Cloud Armor policy events if they're in preview mode. I've tried to copy what is used in the parser for the enforcedSecurityPolicy data, changing the values to previewSecurityPolicy to append it as an additional security_result.outcome value, with no success. Any help would be appreciated.
+2I have this - but it appears to overwrite the enforced policy - not append to to the security_result like I'd want.
filter {
json {
source => "message"
array_function => "split_columns"
on_error => "_not_json"
}
if [jsonPayload][previewSecurityPolicy][name] != "" {
mutate {
replace => {
"security_result.rule_name" =>
"%{jsonPayload.previewSecurityPolicy.name}"
}
}
}
mutate {
rename => {
"jsonPayload.previewSecurityPolicy.priority" =>
"security_result.priority_details"
}
}
mutate {
convert => {
"security_result.priority_details" => "string"
}
on_error => "already_a_string"
}
if [jsonPayload][previewSecurityPolicy][configuredAction] == "DENY" {
mutate {
replace => {
"_action" => "BLOCK"
}
}
}
else if [jsonPayload][previewSecurityPolicy][configuredAction] != "" {
mutate {
replace => {
"_action" => "ALLOW"
}
}
}
if [_action] != "" {
mutate {
replace => {
"security_result.action_details" =>
"%{jsonPayload.previewSecurityPolicy.configuredAction}"
}
}
mutate {
merge => {
"security_result.action" => "_action"
}
}
}
for i, threat_category in
jsonPayload.previewSecurityPolicy.threatIntelligence.categories {
mutate {
replace => {
"var_threat_intelligence_category" => ""
}
}
mutate {
replace => {
"var_threat_intelligence_category.value" =>
"%{threat_category}"
}
on_error => "no_threat_intelligence_category"
}
if ![no_threat_intelligence_category] and
[var_threat_intelligence_category][value] != "" {
mutate {
replace => {
"var_threat_intelligence_category.key" =>
"threat_intelligence_category"
}
}
mutate {
merge => {
"security_result.rule_labels" =>
"var_threat_intelligence_category"
}
}
}
}
mutate {
replace => {
"var_jsonpayload_previewsecuritypolicy_outcome.value" =>
"%{jsonPayload.previewSecurityPolicy.outcome}"
}
on_error => "no_jsonpayload_previewsecuritypolicy_outcome"
}
if ![no_jsonpayload_previewsecuritypolicy_outcome] and
[var_jsonpayload_previewsecuritypolicy_outcome][value] != "" {
mutate {
replace => {
"var_jsonpayload_previewsecuritypolicy_outcome.key" =>
"jsonpayload_previewsecuritypolicy_outcome"
}
}
mutate {
merge => {
"security_result.outcomes" =>
"var_jsonpayload_previewsecuritypolicy_outcome"
}
}
}
mutate {
replace => {
"var_matched_field_name.value" =>
"%{jsonPayload.previewSecurityPolicy.matchedFieldName}"
}
on_error => "no_matched_field_name"
}
if ![no_matched_field_name] and [var_matched_field_name][value] != "" {
mutate {
replace => {
"var_matched_field_name.key" => "matched_field_name"
}
}
mutate {
merge => {
"security_result.rule_labels" => "var_matched_field_name"
}
}
}
mutate {
replace => {
"var_matched_field_type.value" =>
"%{jsonPayload.previewSecurityPolicy.matchedFieldType}"
}
on_error => "no_matched_field_type"
}
if ![no_matched_field_type] and [var_matched_field_type][value] != "" {
mutate {
replace => {
"var_matched_field_type.key" => "matched_field_type"
}
}
mutate {
merge => {
"security_result.rule_labels" => "var_matched_field_type"
}
}
}
mutate {
replace => {
"var_matched_field_value.value" =>
"%{jsonPayload.previewSecurityPolicy.matchedFieldValue}"
}
on_error => "no_matched_field_value"
}
if ![no_matched_field_value] and [var_matched_field_value][value] != "" {
mutate {
replace => {
"var_matched_field_value.key" => "matched_field_value"
}
}
mutate {
merge => {
"security_result.rule_labels" => "var_matched_field_value"
}
}
}
mutate {
convert => {
"jsonPayload.previewSecurityPolicy.matchedLength" => "string"
}
on_error => "already_a_string"
}
mutate {
replace => {
"var_matched_length.value" =>
"%{jsonPayload.previewSecurityPolicy.matchedLength}"
}
on_error => "no_matched_length"
}
if ![no_matched_length] and [var_matched_length][value] != "" {
mutate {
replace => {
"var_matched_length.key" => "matched_length"
}
}
mutate {
merge => {
"security_result.rule_labels" => "var_matched_length"
}
}
}
for index, preconfiguredExprId in jsonPayload.previewSecurityPolicy.preconfiguredExprIds {
mutate {
replace => {
"var_preconfigured_expr_ids" => ""
}
}
mutate {
replace => {
"var_preconfigured_expr_ids.key" => "preconfigured_expr_ids"
"var_preconfigured_expr_ids.value" => "%{preconfiguredExprId}"
}
}
mutate {
merge => {
"security_result.rule_labels" => "var_preconfigured_expr_ids"
}
}
if [security_result] != "" {
mutate {
merge => {
"event.idm.read_only_udm.security_result" => "security_result"
}
}
}
mutate {
merge => {
"@output" => "event"
}
}
}
}
+2Got it working by using a custom parser instead of an extension and just adding the content above. Think the first way doesn't work due to this which says -
A parser extension can map data to a repeated field only when the repeated field is at the lowest level of the hierarchy. For example, mapping values to udm.principal.ip is supported because the repeated ip field is at the lowest level of the hierarchy and principal is not a repeated field. Mapping values to udm.intermediary.hostname is not supported because intermediary is a repeated field and is not at the lowest level of the hierarchy.
It means parser updates probably will be more awkward if I have to use a custom parser instead of just an extension - but I don't think there's a cleaner way unless someone can correct me.
+6Got it working by using a custom parser instead of an extension and just adding the content above. Think the first way doesn't work due to this which says -
A parser extension can map data to a repeated field only when the repeated field is at the lowest level of the hierarchy. For example, mapping values to udm.principal.ip is supported because the repeated ip field is at the lowest level of the hierarchy and principal is not a repeated field. Mapping values to udm.intermediary.hostname is not supported because intermediary is a repeated field and is not at the lowest level of the hierarchy.
It means parser updates probably will be more awkward if I have to use a custom parser instead of just an extension - but I don't think there's a cleaner way unless someone can correct me.
This seems like a regression, i've got the default loadbalancing parser dated "2023-06-30" and it does pull preview values through. Does anyone from Chronicle know why this is no longer there?
Its worth noting that we do maintain a number of custom parsers because we've frequently run into issues where logs don't look exactly how we want them. For example, in our loadbalancing parser we've also pulled priority through to a label for both enforced and preview.
+3This seems like a regression, i've got the default loadbalancing parser dated "2023-06-30" and it does pull preview values through. Does anyone from Chronicle know why this is no longer there?
Its worth noting that we do maintain a number of custom parsers because we've frequently run into issues where logs don't look exactly how we want them. For example, in our loadbalancing parser we've also pulled priority through to a label for both enforced and preview.
We rebuilt many of the GCP parsers, including Load Balancing. We didn't test it against the preview version of Cloud Armor policies, so it probably got missed.
@Nathan-Scott can you open a support case with this issue and the unparsed logs so we can fix it in the default parser?
Thanks.
+2We rebuilt many of the GCP parsers, including Load Balancing. We didn't test it against the preview version of Cloud Armor policies, so it probably got missed.
@Nathan-Scott can you open a support case with this issue and the unparsed logs so we can fix it in the default parser?
Thanks.
Independently of me discovering this, I've found out another user from my org created a support ticket 2-3 weeks ago with the issue outlined, with no progress made since.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
