Skip to main content

Hello everyone, i want to calculate the storage volume used by each log type in Google SecOps.

 

For this, i have used the below query to create a dashboard, and this gives me the volume for each ingestion component (Ingestion API, Normalizer, Out-of-Band Processor)

I want to know, if i want to calculate the storage volume used, which ingestion component/s should be considered?

 

$comp = ingestion.component
ingestion.log_type != ""
$Log_Type = ingestion.log_type
$Date = timestamp.get_date(ingestion.end_time)

match:
    $Log_Type, $comp

outcome:
    $Count = math.round(sum(ingestion.log_volume) / (1000 * 1000 * 1000), 2)
    $eventc = math.round(sum(ingestion.log_count))

order:
    $Count desc

Hi, You can use the "Data Ingestion and Health" dashboard to view this information.

But if you prefer something more custom, you can use the following query:

ingestion.component = "Ingestion API"
ingestion.log_type != ""
ingestion.log_type != "FORWARDER_HEARTBEAT"
$log_type = ingestion.log_type
match:
$log_type
outcome:
$thoughput = math.round(sum(ingestion.log_volume) / (1000 * 1000 * 1000), 1)
order:
$thoughput desc

Personally, for this scenario, I prefer using Chris Martin’s dashboard — highly recommended!:
https://medium.com/@thatsiemguy/monitoring-your-google-secops-license-utilization-revisited-ef2d067a9c16
 


Hi,
You can use the "Data Ingestion and Health" dashboard to view this information.
But if you prefer something more custom, you can use the following query:

ingestion.component = "Ingestion API"
ingestion.log_type != ""
ingestion.log_type != "FORWARDER_HEARTBEAT"
$log_type = ingestion.log_type
match:
    $log_type
outcome:
    $thoughput = math.round(sum(ingestion.log_volume) / (1000 * 1000 * 1000), 1)
order:
    $thoughput desc
 

Personally, for this scenario, I prefer using the following dashboard — highly recommended!
https://medium.com/@thatsiemguy/monitoring-your-google-secops-license-utilization-revisited-ef2d067a9c16


@Eoved Hi, regarding Google SecOps license storage, will this be calculated based on the log volume from ingestion API only?