Skip to main content

I want to calculate the storage volume used by each log type in the Google Secops . 

 

This is a query I used for this. This gives me the log volume values for each ingestion component (Ingestion API, Normalizer, Out-of-Band Processor) used by each log type.

I want to know when it comes to calculating storage volume for each log type , which ingestion component/s can be used? 

 

$comp = ingestion.component

ingestion.log_type != ""

$Log_Type = ingestion.log_type

$Date = timestamp.get_date(ingestion.end_time)

 

match:

    $Log_Type, $comp

 

outcome:

    $Count = math.round(sum(ingestion.log_volume) / (1000 * 1000 * 1000), 2)

    $eventc = math.round(sum(ingestion.log_count))

 

order:

    $Count desc

From https://cloud.google.com/chronicle/docs/reference/ingestion-metrics-schema 

 

All log flows use the Ingestion API. Therefore, to get an accurate measure of the total volume ingested, refer to the component="Ingestion API".


Try this.

Query 1: Log Volume (GB) by Log Type - Last 30 Days

ingestion.component = "Ingestion API"

ingestion.log_type = $logType

match:

    $logType

outcome:

    $total_gb = math.round(sum(ingestion.log_volume) / math.pow(1000, 3), 4)

    $logCount = sum(ingestion.log_count)    

order:

    $total_gb desc



Query 2: Ingested GB (Last 1 Day)

ingestion.component = "Ingestion API"

outcome:

    // It takes a big number and makes it smaller so we can call it gigabytes (1000^3)

    $total_gb = math.round(sum(ingestion.log_volume) / math.pow(1000, 3), 4)



query 3: Ingested TB (Last 7 Days)

ingestion.component = "Ingestion API"

outcome:

    $total_tb = math.round(sum(ingestion.log_volume) / math.pow(1000, 4), 4)

*** You can change query 3 timestamp to weekly, monthly, and yearly