Google SecOps Unified Data Model (UDM) Downstream Adoption Guide

Author: 

Darren Davis, Senior Technical Solutions Consultant

Part 1: UDM Events

Normalization is the heart of the Google SecOps platform, and standardizing security logs into UDM is what makes downstream detection rules, indexing, and fast-paced investigations possible.

In this guide, we are going to explore the UDM Event model from a downstream consumer's perspective. We will unpack the core nouns, understand field rules, examine how UDM behaves in Search and Rules, and walk through concrete, applied examples of how to query and alert on UDM Events.

Section 1: What is the Unified Data Model (UDM) Event?

Before we construct YARA-L rules or Looker dashboards, let's align on what a UDM Event actually is.

Point-in-Time Telemetry

A UDM Event represents a single, point-in-time activity or security telemetry record. Examples include a process starting on an endpoint, a network connection passing through a firewall, a user logging into an identity provider, or a file being deleted on a server.

Contrast this with the UDM Entity Model (which we will cover in Part 2 of this series), which represents a state-in-time contextual view of assets, users, or resources (like Active Directory user details or CMDB asset criticality). For Part 1, we are focusing exclusively on point-in-time events.

Downstream Velocity: Schema-on-Write vs. Schema-on-Read

Traditional SIEM solutions often rely on Schema-on-Read, meaning logs are stored as raw text, and expensive regular expressions are evaluated on-the-fly when a query is run or a rule is evaluated. This leads to slow searches, rule timeouts, and significant analyst fatigue when querying data.

Google SecOps uses Schema-on-Write. Normalization occurs directly within the ingestion pipeline at the parser level. Disparate vendor log strings are transformed into a single, standardized, and strictly-typed UDM structure before they are written to the database. Once written, these standardized fields are indexed and optimized for search and YARA-L rule evaluation at lightning-fast speeds.

Structural Strictness and Type Safety

Raw log files are ingested as simple text strings. However, the Google SecOps UDM schema enforces strict data typing. Downstream, this type safety ensures that you can perform accurate mathematical operations, range queries, and logical evaluations without worrying about string mismatches.

The standard datatypes you will interact with in Search and Rules are:

• string: Standard UTF-8 encoded text.

• Integer (int32/int64): Whole numbers. Standard numerical fields (like ports or process IDs) are represented as integers.

• uinteger (uint32/uint64): Unsigned integers that must be non-negative (0 or greater). Commonly used for non-negative metrics like byte counts, packet counts, or file sizes.

• Float: Decimal numbers used for precise measurements, coordinates, or risk scores.

• Boolean: A binary value representing true or false. Perfect for status flags (like executed or is_admin).

• ipaddress: A specialized UDM format for validating IPv4 and IPv6 structures.

Section 2: Navigating UDM Across the SecOps Ecosystem

As an analyst or detection engineer, you will interact with UDM across different stages of the Google SecOps platform. It is vital to recognize that the prefix for UDM fields changes depending on where you are working.
 

A Sneak Peek at Entities:

When we get to Part 2 and begin working with UDM Entity data (for contextual AD, DLP, or CMDB data), the prefixes shift:

• In Rules (Detect Engine): The prefix is $entity (e.g., $entity.graph.entity.hostname).
• In Search: Directly query using the entity prefix.

Section 3: UDM Event Model Overview

The fundamental architecture of a UDM Event centers on Nouns and Extensions. Rather than mapping logs to custom vendor-defined fields, UDM organizes the event around the key participants in the transaction.

By translating vendor-specific log fields (like source IP, destination IP, device name, or user) into specific UDM participant blocks downstream, you gain a standard taxonomy.

This unified taxonomy is why a single YARA-L detection rule can detect a threat across Windows event logs, Linux syslogs, CrowdStrike alerts, and Palo Alto firewall connections simultaneously. All IP addresses, hostnames, and user IDs are located in the exact same structured fields, regardless of the security tool that generated the log.

Section 4: Deep Dive into Event Blocks and Nouns

Let’s unpack the primary components of a UDM Event record. Each block has a specific operational purpose, strict rules, and corresponding target formats.

1. Metadata Block

The metadata section stores general background details of the event.

• metadata.event_type (Enum): Specifies the type of event. This must be a predefined enumerated type (e.g., PROCESS_LAUNCH, NETWORK_CONNECTION, USER_LOGIN). In YARA-L, filtering on this field is critical for indexing and rule performance.

• metadata.event_timestamp (Timestamp): The GMT timestamp when the event was generated.

• metadata.product_event_type: The original, product-specific event name (e.g., "EventID 4625", "ProcessRollUp").

• metadata.product_log_id: Vendor-specific unique event GUID.

• metadata.product_name / metadata.vendor_name: Standardized product (e.g., "Falcon", "ASA") and vendor (e.g., "CrowdStrike", "Cisco").
   

Operational Downstream Textproto Example (Metadata):
 

metadata: {
  event_type: PROCESS_LAUNCH
  event_timestamp: {
    seconds: 1765852715
    nanos: 0
  }
  vendor_name: "CrowdStrike"
  product_name: "Falcon"
  product_event_type: "ProcessRollUp"
  product_log_id: "ABcd1234-98766"
}

2. Principal vs. Src

These two nouns are often confused, but they have completely different operational contexts.

Principal (The Actor)

The principal represents the acting entity that originates the activity described in the event.

• Structural Rules: The principal must include at least one machine identifier (hostname, MAC, IP, EDR asset ID) or user identifier (username). Optionally, it can contain process details.

• Banned Fields: To maintain schema integrity, the principal block must NOT contain the following fields: email, files, registry keys, or registry values.

Src (The Object Being Acted Upon)

The src represents the source entity being acted upon by the participant, along with the device or process context for the source object (the machine where the source object physically resides).

• Operational Example: If a user on Workstation-A copies a file located on FileServer-1 over to USB-Drive, the src block represents FileServer-1 and the source file itself, while the principal remains the user on Workstation-A.

Downstream Textproto Representation:

# Principal: The workstation where the admin user ran the command
principal: {
  hostname: "workstation-01"
  user: {
    userid: "admin_user"
  }
}
# Src: The remote file share where the copied file originally resided
src: {
  hostname: "fileserver-99"
  file: {
    full_path: "\\\\fileserver-99\\share\\sensitive_data.txt"
    size: 2048
  }
}

3. Target

The target represents the target entity being referenced by the event, or an object residing on the target entity.

• For a network connection: principal is the source machine, and target is the destination web server or database [227].

• For process injection: principal is the parent process initiating the action, and target is the victim process being injected [231, 297].

• For Windows registry modification: target stores the affected registry key and its value data [236, 237].
   

Downstream Textproto Representation (Target File):
 

target: {
  file: {
    full_path: "C:\\Windows\\System32\\shady.exe"
    sha256: "d7173c568b8985e61b4050f81b3fd8e75bc922d2a0843d7079c81ca4b6e36417"
    size: 512000
  }
}

4. Intermediary vs. Observer

These components handle middle-tier nodes in network or system transactions.

Intermediary (The Router/Relay)

An intermediary represents one or more intermediate entities that actively process, route, or potentially modify the activity. Examples include web proxy servers, SMTP mail relays, load balancers, or single sign-on (SSO) identity servers.

• Rule of Thumb: The principal (originator), target (intended destination), and initial action description remain exactly the same, regardless of intermediary actions. For instance, a successful web connection from Client-A to Server-B, and a connection from Client-A to Server-B that gets blocked by Proxy-C, will both maintain principal: Client-A and target: Server-B. Proxy-C is mapped as the intermediary.

Observer (The Passive Listener)

An observer represents an entity that passively observes and reports on the event but does not route, modify, or sit directly in the active network path. Examples include packet sniffers, TAP devices, or network-based vulnerability scanners.
 

Downstream Textproto Representation (Proxy Intermediary):

principal: { ip: "192.168.1.50" }
target: { ip: "8.8.8.8" }
intermediary: {
  hostname: "corporate-web-proxy"
  ip: "192.168.1.1"
}

5. securityResult

The security_result block is critical. It standardizes security risks, threat intelligence hits, and actions taken by endpoints, firewalls, and security products.

Normalizing Actions: ALLOW vs. BLOCK

Google SecOps classifies product actions strictly:

• Successful Actions:ALLOW and ALLOW_WITH_MODIFICATION.

• Failed Actions:BLOCK, QUARANTINE, FAIL, and CHALLENGE.

The Two Types of securityResult

A UDM event can contain multiple repeated security results. They are split into two logical types:

1. Entire Event Security Result: The result applies to the entire event as a whole (e.g., a mailbox receiving a SPAM or phishing email). In this scenario, the security_result.about field must remain empty.

2. Specific Object Security Result: The result applies to a specific object referenced within the event (e.g., a local file scan finding a Trojan in a specific file on a host). In this scenario, the security_resultmust populate the about field with the details of the implicated noun (such as the process, file, IP, or email details).

Downstream Textproto Representation (Specific Object Trojan Detection):

security_result: {
  action: BLOCK
  category: SOFTWARE_MALICIOUS
  threat_name: "W32/File-A"
  # This result is specifically about this malicious file
  about: {
    file: {
      full_path: "C:\\Users\\victim\\Downloads\\malware.exe"
      md5: "35bf623e7db9bf0d68d0dda764fd9e8c"
    }
  }
}

6. About (Standalone Noun Block)

The about field at the event root serves as a standalone block. It represents entities referenced by the event that are not otherwise described as participants.

• Examples: Email file attachments, embedded domains/URLs/IPs inside an email body, or DLL modules loaded during a PROCESS_LAUNCH event.

7. Extensions, Extracted, and Grouped

• Extensions (extensions.auth, extensions.vulns, etc.): Event-specific sub-messages designed to capture specialized metadata. For example, the auth extension captures login mechanics.

• Extracted (extracted): Stored as a raw, flattened JSON structure of fields that don't have direct schema mappings. Useful for conserving context, but try to avoid relying heavily on this as it is not first-party UDM

• Grouped (grouped): An optional block containing repeated list arrays for standard aliases (domain, email, file_path, hash, hostname, ip, process_id, user) to assist in high-performance correlation. Note: this is handled by SecOps on the backend.
   

Section 5: Understanding and Querying Enumerated Fields (Enums) Downstream

One of the most powerful, yet occasionally misunderstood, features of the Google SecOps Unified Data Model is its extensive use of Enumerated Fields (Enums).

What is a UDM Enum?

An enum is a unique data type where a human-readable, uppercase string constant corresponds directly to an underlying numerical value in the database.

Upstream, when logs are ingested, parsers map raw, messy vendor values into these normalized enums. Downstream,when you are writing rules, searching for threats, or constructing dashboards, you only deal with the standardized, uppercase string constants.

For example, whether a firewall log says "permit", "allowed", "pass", or "success", Google SecOps maps it to the standard ALLOW enum. Similarly, "drop", "deny", "block", or "reject" are mapped to the standard BLOCK enum.
 

Case Conventions & Naming Rules

Google SecOps employs strict styling rules to differentiate field names, field types, and enum values:

• Field Type Values: Expressed using camelCase characters in documentation (e.g., platform or eventType).

• Field Names: Expressed in pure lowercase_with_underscores (e.g., metadata.event_type or security_result.action).

• Enum Values: Strictly written as UPPERCASE_WITH_UNDERSCORES (e.g., PROCESS_LAUNCH, BLOCK, TCP, WINDOWS).

How Enums Behave Downstream
 

1. In UDM Search & Dashboards

When searching, you filter fields using their standard uppercase string names. Since the indexing layer resolves enums automatically, your searches are incredibly fast and consistent.

• Correct: metadata.event_type = "NETWORK_CONNECTION"

• Incorrect: metadata.event_type = "network_connection" (this will return zero results due to strict case validation).

2. In YARA-L Rules

In the detection engine, enums are evaluated using their uppercase string names. This allows you to write rules that target multiple vendors simultaneously.

• YARA-L Clause: $event.security_result.action = "BLOCK"

• Detection Advantage: This single line will catch blocks from Palo Alto firewalls, Cisco ASA gateways, CrowdStrike Falcon endpoint agents, and Windows Defender simultaneously, without needing separate OR conditions for every vendor's terminology.

3. In Legacy Dashboards & BigQuery SQL

In BigQuery, enums are represented as integers, and an enum mapping is required for event_type. When viewing Looker dashboards, enums are typically exposed as readable strings (e.g., 'ALLOW' or 'BLOCK'), which allows Looker to perform high-performance aggregations and filtering without full-text search overhead.

Core Downstream Enums Quick Reference

Here is a master table of the most critical enums that every analyst and detection engineer should know:
 

Section 6: Structural Requirements for Common Log Families

Downstream query velocity and rule accuracy depend on mapping logs to the correct UDM structures. When querying, alert hunting, or designing dashboards for different log classes, you must understand the mandatory fields, participant relationships, and technical design rules governing each family.

1. Network Telemetry Profile (Firewalls, Web Gateways, Routers)

Network telemetry represents communication sessions between devices. It is primarily normalized to NETWORK_CONNECTION or NETWORK_HTTP.

• Core Nouns & Participants:

  • Principal (The Source): Represents the device that initiated the network session. Must contain at least one machine identifier (such as ip, hostname, mac, or EDR asset_id).

  • Target (The Destination): Represents the intended receiving machine. Must include destination identifiers.

  • Intermediary (Optional Proxy/Relay): Represents proxy servers, SMTP relays, or firewalls that actively process, route, or block the connection. Note that the principal (initiator) and target (destination) remain constant; the proxy is mapped as the intermediary.

  • Observer (Optional passive node): Represents packet sniffers or passive network monitors that observe the transaction without routing or modifying it.
     

• Key Fields for Network Search & Rules:

  • network.ip_protocol: Standardized protocol enum (e.g., TCP, UDP, ICMP).

  • network.direction: Traffic direction relative to the principal device (INBOUND, OUTBOUND, BROADCAST).

  • principal.port and target.port: Network port integers.

  • network.sent_bytes / network.received_bytes / network.total_bytes: Unsigned integers tracking traffic volume.

  • network.session_duration: Tracks connection length in seconds and nanoseconds.
     

• The Downstream "Single-IP Port" Rule:

  • This is a strict UDM constraint. If either the principal or target block contains a specified port, that noun must contain one and only one IP address. This IP represents the exact interface bound to that port during the connection.

  • If no ports are specified (such as in bulk asset reports or non-connection telemetry), you are permitted to list multiple IP addresses under a single host.

  • Rule of Thumb: Never attempt to evaluate or filter multiple IPs in a YARA-L match or search if ports are specified, it violates UDM validation.

2. EDR & Endpoint Telemetry Profile (Host Telemetry)

Endpoint Detection and Response (EDR) and host logs track process activities, filesystem access, registry changes, and system configurations on assets.

• Process Launch (PROCESS_LAUNCH)

  • Used to track binary executions on endpoints.

  • Required downstream structure:

    • principal: Represents the endpoint machine where the launch occurred. Must contain an asset identifier (such as hostname or EDR asset_id).

    • principal.process: Represents the parent process that spawned the action. Requires the parent's pid and command_line.

    • target.process: Represents the newly spawned child process. It must contain the child's pid, command_line, and file.full_path.

    • Query Tip: To hunt for malicious execution, always pivot on $event.target.process.command_line or $event.target.process.file.sha256.
       

• File Activity (FILE_CREATION, FILE_DELETION, FILE_MODIFICATION, FILE_READ, FILE_OPEN)

  •  Used to track filesystem operations.

  • Required downstream structure:

    • principal: Represents the device and the acting process or user performing the operation. Must include a machine identifier and should populate principal.user.userid and principal.process.pid.

    • target.file: Represents the file being acted upon. Must include the full path (target.file.full_path) and file metadata (such as size or cryptographic hashes like sha256 or md5).

    • src.file (Specifically for FILE_COPY): Represents the original source file being copied (e.g., on a remote file share), while target.file represents the newly created copy.
       

• Registry Modification (REGISTRY_MODIFICATION, REGISTRY_CREATION)

  • Used to track Windows registry alterations (typically associated with persistence).

  • Required downstream structure:

    • principal: Represents the host device and process performing the modification.

    • target.registry: Tracks the affected registry key. Must populate target.registry.registry_key (e.g., HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services...).

    • Optional Context: Downstream analysis benefits significantly when target.registry.registry_value_name and target.registry.registry_value_data are present to evaluate exact persistence scripts or settings.

3. Security Alert & Threat Detection Profile (Logs with Security Results)

This profile covers telemetry from any security device (firewalls, web proxies, EDR agents, antivirus software, email security gateways) that has evaluated a transaction and rendered a security verdict or alert.

• The security_result Block:

  • Every detection or alert must populate the repeated security_result block at the event root.

  • Key standardized downstream fields:

    • security_result.action: The verdict rendered by the vendor tool. Classified into Successful (ALLOW, ALLOW_WITH_MODIFICATION) or Failed (BLOCK, QUARANTINE, FAIL, CHALLENGE) actions.

    • security_result.category: Standardized threat category (e.g., SOFTWARE_MALICIOUS, ACL_VIOLATION, AUTH_VIOLATION, EXPLOIT, POLICY_VIOLATION).

    • security_result.severity: Standardized product severity (e.g., INFORMATIONAL, LOW, MEDIUM, HIGH, CRITICAL).

    • security_result.threat_name: Standardized malware or threat name (e.g., W32/File-A, Slammer).

    • security_result.rule_name / security_result.rule_id: Name and identifier of the signature or rule that triggered.

• The Crucial Design Distinction: Entire Event vs. Specific Object

  • When searching or writing rules, you must pay attention to how security_result is bounded:

1. Entire Event Security Result: The security verdict applies to the transaction as a whole (e.g., an entire email is flagged as SPAM or phishing). In this scenario, the security_result.about field must remain empty.

2. Specific Object Security Result: The security verdict applies to a specific component within a larger event (e.g., an endpoint process launch occurred, but the antivirus module detected that the spawned executable binary itself contains a Trojan). In this scenario, the security_result block must populate the about field with details of the implicated noun (such as the specific file or process details).

• Why this matters downstream: If you are searching for malicious hashes, searching bare fields might fail if the vendor's AV engine logged the malware under the security_result.about.file.sha256 block instead of the root target.file.sha256 block. Always structure your rules and search queries to inspect both locations!

Section 7: Downstream Use Cases & Applied Examples

UDM field requirements are strictly dependent on the specified event_type. Below are three common security use cases demonstrating how to search and build YARA-L detection rules using normalized UDM fields.

Use Case 1: Process Launch (PROCESS_LAUNCH)

This event type represents a process execution on an endpoint.

Structural Rules:

• principal: Represents the host asset where the execution occurred.

• principal.process: Represents the parent process that initiated the execution.

• target.process: Represents the newly created process being launched.

Downstream UDM Search Query:

metadata.event_type = "PROCESS_LAUNCH" AND target.process.file.full_path = "C:\\Windows\\System32\\cmd.exe"

YARA-L Rule Example (Suspicious Process Spawn):

rule suspicious_process_spawn {
  meta:
    author = "Darren Davis"
    description = "Detects cmd.exe spawned by a non-standard parent process"

  events:
    $launch.metadata.event_type = "PROCESS_LAUNCH"
    $launch.principal.process.file.full_path != "C:\\Windows\\explorer.exe"
    $launch.target.process.file.full_path = "C:\\Windows\\System32\\cmd.exe"

  condition:
    $launch
}

Use Case 2: Network Connection (NETWORK_CONNECTION)

This represents network telemetry, usually originating from firewalls, web gateways, or routers.

Structural Rules:

• principal: The device that initiated the network connection.

• target: The destination machine.

• network: Capture connection details (ip_protocol, ports, and byte counts).

• The Single-IP Port Rule: If either the principal or target has a port specified, you must populate one and only one IP address in that noun (representing the exact IP bound to that port during the connection).

Downstream UDM Search Query:

metadata.event_type = "NETWORK_CONNECTION" AND target.port = 443 AND network.ip_protocol = "TCP"

YARA-L Rule Example (Beaconing Destination Port 443):

rule beaconing_detected {
  meta:
    author = "Darren Davis"
    description = "Detects outbound network connection on port 443 to a single external IP"

  events:
    $net.metadata.event_type = "NETWORK_CONNECTION"
    $net.principal.ip = $src_ip
    $net.target.ip = $dst_ip
    $net.target.port = 443
    $net.network.direction = "OUTBOUND"

  match:
    $src_ip over 10m

  condition:
    #net >= 50
}

Use Case 3: User Login (USER_LOGIN)

This covers authentication telemetry, such as endpoint logins or identity provider audits.

Structural Rules:

• principal: For remote logins, map the machine where the user is logging in from. For local console logins, do not set the principal.

• target: Map the user that is logging in (target.user.userid).

• intermediary: For SSO or Identity Provider logins, map the SSO authentication server details here.

• extensions.auth: Capture the auth_type (e.g., SSO, VPN) and the specific mechanism used (e.g., USERNAME_PASSWORD, OTP).

• security_result: If the authentication fails, the security_result block contains the category AUTH_VIOLATION and action FAIL or BLOCK.

Downstream UDM Search Query:

metadata.event_type = "USER_LOGIN" AND security_result.category = "AUTH_VIOLATION" AND extensions.auth.type = "SSO"

YARA-L Rule Example (Brute Force Authentication):

rule sso_brute_force {
  meta:
    author = "Darren Davis"
    description = "Detects multiple failed login attempts against a single target user"

  events:
    $login.metadata.event_type = "USER_LOGIN"
    $login.target.user.userid = $user
    $login.security_result.category = "AUTH_VIOLATION"
    $login.security_result.action = "BLOCK"

  match:
    $user over 5m

  condition:
    #login >= 10
}

Section 8: Downstream Best Practices

To wrap up Part 1, here are three design commandments that will ensure your searches, rules, and dashboards perform flawlessly downstream:

1. Pivot on Normalized Actions (ALLOW vs. BLOCK): Different firewall and EDR vendors use wild terminologies like "drop", "quarantine", "permit", or "terminate". Downstream, do not hardcode vendor actions. Always leverage the standardized, normalized values in security_result.action.

2. Filter by Event Type First: When writing YARA-L rules or constructing heavy search queries, always specify metadata.event_type as your first filter. This narrows down the database partitions instantly, preventing timeouts and saving massive index computing resources.

3. Know Your Prefixes: Remember that bare fields or the udm. prefixes are used in Search, $event is the rule execution standard, and parsing uses CBN normalizer mappings. Aligning your queries to the appropriate platform interface will save you hours of debugging!

With this foundation, querying and alerting on UDM Events will become second nature! Get ready for Part 2, where we will tackle the Entity Model and relationship graphing.

Summary & Conclusion 

Standardizing security logs under the Unified Data Model (UDM) is a force multiplier for your security operations. By enforcing a Schema-on-Write architecture, Google SecOps standardizes diverse logs upon ingestion, ensuring downstream searches, rules, and dashboards execute at lightning speed and preventing the "UDM fatigue" of legacy platforms.

Crucial Downstream Takeaways: Standardization Over Customization: Leverage UDM's unified taxonomy. Standardizing actions (like ALLOW and BLOCK) and participant nouns (like Principal vs. Src) allows a single YARA-L rule to detect threats across Windows, Linux, and multi-vendor firewall logs simultaneously. Filter by Event Type First: Always specify metadata.event_type as your first filter in search queries and YARA-L rules to instantly partition database indexing, optimize performance, and prevent rule timeouts. SOC Mindset Shift: Pivot from vendor-specific syntax to the structural "UDM Event Story"—determining who the principal, target, and intermediary participants are in any given transaction. By mastering these point-in-time Event telemetry mechanics, normalization downstream becomes second nature.

With this foundation, querying and alerting on UDM Events will become second nature! Get ready for Part 2, where we will tackle the Entity Model and relationship graphing.