Alerting on log ingestion anomalies using UDM Events

Interestingly, in the BigQuery export, these data are correctly typed as floats:

Have you tried using the cast.as_float in the outcome section when you are applying the aggregation function?  I am pulling the task from additional fields but it is a text string that is storing a number.

$event.additional.fields["Task"] != "0"
$event.additional.fields["Task"] != ""

outcome:
$task_array = array_distinct($event.additional.fields["Task"])
$task_max = max(cast.as_float($event.additional.fields["Task"]))
$task_min = min(cast.as_float($event.additional.fields["Task"]))
$task_count = count($event.additional.fields["Task"])

 

 

Have you tried using the cast.as_float in the outcome section when you are applying the aggregation function?  I am pulling the task from additional fields but it is a text string that is storing a number.

For my testing I found that I needed both of these lines in my events section to get my minimum value to be not 0, so try neither, one or both and see what you get.

 

$event.additional.fields["Task"] != "0"
$event.additional.fields["Task"] != ""

outcome:
$task_array = array_distinct($event.additional.fields["Task"])
$task_max = max(cast.as_float($event.additional.fields["Task"]))
$task_min = min(cast.as_float($event.additional.fields["Task"]))
$task_count = count($event.additional.fields["Task"])

 

 

Hi,

Thank you very much for your detailed answer and for sharing your testing approach! I tried your suggestions, but unfortunately, the issue seems to be deeper.

Based on what you shared, I realized that all the strings in my case are actually equal to "", even though when I display these values, they appear different.

So, it looks like the problem is not just with the aggregation or casting, but rather with how the solution is handling or reading these values.

If you have any idea why this could happen, I would really appreciate your help. Thanks again for your time!

Best regards,

I don't have a solution to your problem inside SecOps but I just wanted to make sure you were aware the ingestion metrics are also inside GCP in metrics viewer.

You can write promql against the metrics data then use an alerting rule to send alerts (in GCP). I presume you are using cloud-functions run against the bigquery data because it's much easier to do stats calculations in general programming language but just wanted to make sure you (and future vieweres) were aware of the alternative approach. Good luck!

I find it odd that these values are showing up but you are specifying value being null. Is it possible that you have more than one key of the same name? If there is anything more to your rule than the screenshot snippets you provided that might provide a clue but understand if you can't share it.

Hi everyone,

For those who might find this helpful, I managed to work around this issue: It seems the additionals fields don't handle float casting correctly.
I was able to get my log type monitoring rule to work by using the detection_fields instead, as they do handle the cast to float properly.

Hope this helps someone!