Tuesday's Tip of the Week - When Logs Don't Parse: Writing CBN Parsers for Custom Sources

Why Parsers Matter

Every log source that flows into Google SecOps must be transformed from its raw format into UDM before it becomes searchable, huntable, or detectable. When a log source has no parser, or the default parser misses critical fields, those fields simply do not exist in UDM. Your detection rules cannot match on data that was never mapped. Your UDM Search queries return nothing. The raw data is technically stored, but it is invisible to your detection program until a parser maps it correctly.

How CBN Parsers Work

Chronicle uses Code-Based Normalizer (CBN) parsers. Each parser handles a specific log source and follows a consistent workflow:

1. Ingest raw log: The parser receives a raw log entry (JSON, syslog, CSV, XML, or other format).
2. Extract fields: The parser uses extraction logic (Grok patterns, regex, JSON path expressions) to pull individual values from the raw log.
3. Map to UDM: Each extracted value is mapped to the appropriate UDM field. A source IP becomes principal.ip. A username becomes principal.user.userid. An action outcome becomes security_result.action.
4. Output UDM event: The parser emits a fully structured UDM event with metadata.event_type, noun groups populated, and all relevant fields in place.

The parser's core job is translating vendor-specific field names and structures into the universal UDM schema. A Palo Alto firewall log and a Check Point firewall log both become NETWORK_CONNECTION events with principal.ip, target.ip, and security_result.action fields.

The Parser Editor Workflow

Google SecOps provides a parser editor in the UI where you build and test custom parsers:

1. Paste raw log samples: Grab representative log entries from your source. Include normal events, error events, and edge cases.
2. Write mapping logic: Define how each raw field maps to a UDM field. The parser editor provides the authoring environment for this logic.
3. Validate output: The editor shows you the resulting UDM event for each sample log. Verify that metadata.event_type is correct, that principal and target fields are populated, and that no critical data is dropped.
4. Iterate: Adjust mappings, add additional field extractions, and re-validate until the output matches your expectations.
5. Deploy: Once validated, deploy the parser to process incoming logs from that source.

For detailed CBN parser syntax and authoring specifics, refer to the Google SecOps parser documentation. The syntax is specific to the platform and is best learned from the official reference alongside the editor's built-in guidance.

When You Need a Custom Parser

Common scenarios that require custom parser work:

• New log source with no default parser: A niche vendor product or a custom application generating its own log format.
• Default parser missing fields: The built-in parser maps the basics but drops fields your detections need, such as a custom risk score or a session identifier.
• Log format changes: A vendor updates their log format in a new software version, and the existing parser no longer extracts fields correctly.

When logs from your ingestion pipeline don't parse to UDM, this is where you fix it. A log source that ingests successfully but parses poorly gives you a false sense of coverage.