Skip to main content

Is there a way to account for null/zero values that show up when you run a match window? 

For example this query - 

ingestion.log_type != ""

$log_type = ingestion.log_type

match:

$log_type

outcome:

$event_count = sum(ingestion.event_count)

$event_type_count = count_distinct(ingestion.event_type)

$event_types = array_distinct(ingestion.event_type)

order:

$event_count desc

 

Gives results back like the following

log_type - Test

event_count - 4

event_type_count = 3

event_types - USER,,EDR

as you can see it accounted for the zero value but I do not want it to account for the empty value in that count for event_type_count

the match section should by default excluding the null events.

https://cloud.google.com/chronicle/docs/detection/yara-l-2-0-syntax#options_section_syntax

https://cloud.google.com/chronicle/docs/detection/yara-l-2-0-syntax#allow_zero_values

 

I guess the log_type are all unique, because it could be normal behavior to have a count of 4 but distinct count of 3.

You can add an IF condition in the outcome section to count only if the field is populated


When I try to do that it gets an error - 

compilation error parsing query: parser error: mismatched input 'options' expecting <EOF> line: 9 column: 1-2 : invalid argument
 


my bad, I was thinking you were working in detection, rules editor.

in search/dashboard, you can either

  • remove empty value in the condition

ingestion.event_type != ""

 

  • replace the empty value by “NULL”, which is the most accurate if you want to have the true count but all the possible eventype.

ingestion.log_type != ""

$log_type = ingestion.log_type

$event_type_temp = if(ingestion.event_type = "", "NULL",ingestion.event_type)

match:

$log_type

outcome:

$event_count = count(ingestion.event_count)

$event_type_count = count_distinct(ingestion.event_type)

$event_types = array_distinct(ingestion.event_type)

$event_type = array_distinct($event_type_temp)

 

To rmove the empty value from the array, I didn’t find yet how to achieve that.


I get the outcomes correctly but I am not sure how to count based off of $event_type_temp as it will include the null values as well which. I would want to exclude it for the overall count


I’m not sure if this is possible, since you’re not excluding the empty value in the condition.

 

ingestion.log_type != ""

ingestion.event_type != ""

$log_type = ingestion.log_type

$event_type = ingestion.event_type

match:

$log_type

outcome:

$event_count = count(ingestion.event_count)

$event_type_unique_count = count_distinct($event_type)

$event_types = array_distinct($event_type)

order:

$event_count desc

 

But if you find how to exclude empty value only from the array_distinct or count_disctinct, I’ll be interested to know.


Reply