Tuesday's Tip of the Week - Finding and Fixing Unparsed Logs

July 7, 2026

Blind Spots: Finding and Fixing Unparsed Logs

The Hidden Risk of Unparsed Logs

Unparsed logs are the silent failure mode of any SIEM deployment. When a log fails to parse, the raw data is still stored in Google SecOps and can be found using Raw Log Scan. However, that data is NOT available for UDM Search or YARA-L detection rules. If your firewall logs stop parsing correctly after a firmware update, your network detection rules keep running without errors. They just stop matching anything. No alerts fire, and you might mistake silence for safety.

The Data Ingestion and Health Dashboard

Google SecOps provides a Data Ingestion and Health dashboard that gives you visibility into the state of your log pipeline. Monitor these indicators for every log source:

• Volume by source: How many events per hour/day are arriving from each log source? Establish baselines so you can spot drops.
• Parse rate: What percentage of ingested logs successfully parse to UDM? Your target should be 99% or higher across all sources.
• Ingestion latency: How long between event generation and availability in SecOps? High latency means your detections fire late.

Make reviewing this dashboard part of your daily SOC routine, at least once per shift.

Common Causes of Parse Failures

When you see parse rates drop, investigate these common causes:

• Log format changes: A vendor pushes a software update that changes field names, adds new fields, or restructures the log output. This is the most frequent cause.
• Missing parsers for new sources: A new tool or service starts sending logs, but no parser exists for that format yet.
• Malformed log entries: Application errors, truncated messages, or encoding issues produce logs that don't match the expected structure.
• Parser version mismatches: A parser update is deployed that doesn't handle older log format variants that are still in the pipeline.

The Fix Workflow

When you identify a failing source:

1. Get the Broken Logs

Don't waste time looking at healthy data. You need the exact strings causing the failures.

• Go into SIEM Search and pull the raw logs for that specific device.

• You're looking for the ones where fields are missing, or where the whole payload is just shoved into a generic description or unparsed block because the system choked on it.

• Copy 3 to 5 different examples of those broken logs into a notepad. You need a few variations to make sure your fix doesn't just work for one single event.

2. Hunt Down the Broken Block (The Cut & Drop Method)

Open the parser editor and paste one of your broken logs into the test pane. Since you can’t just comment lines out without breaking the compiler, do this instead:

• Scroll past the very first extraction block (the initial json or grok at the top that breaks the raw log into basic pieces).

• Select everything below that initial block all the way to the bottom of the script, cut it, and paste it into a separate blank text document.

• Hit Preview/Test. If the top block passes on its own, your baseline is good.

• Paste a chunk of the code back in, section by section (like the next conditional or mutate block), and test again.

• Keep pasting code back in until the test fails. The last section you pasted back in is your culprit.

3. Fix the Logic

Once you pinpoint the failing block, it's usually one of three things:

• Empty fields breaking regex: The parser expects a number or string (like a port or username), but the vendor sent a dash - or left it blank because the field was empty. You need to adjust the grok pattern or regex to allow for nulls or optional characters.

• Timestamp shifts: The vendor updated their software and changed the log time format (like adding a timezone offset or switching to ISO). Your date {} filter will fail if that exact new pattern isn't added to the matching array.

• Variable collisions: Someone used a reserved system keyword like event, message, or timestamp as a temporary variable name, which causes a conflict in the backend engine. Rename them to something unique like tmp_data.

4. Test Against Past Data

Do not deploy a fix based on just your few samples, or you risk breaking the logs that were actually working fine.

• Use the Validate tool in the editor to run your new code against the last 30 days of historical logs for that source.

• Check the error rate. If your fix cleared your specific error but caused a spike in failures for other log types from that vendor, pull the change back. Wrap your fix in an if/else block so it only triggers on the specific event type or ID you are trying to repair.

5. Deploy and Confirm

When the validation looks clean, push it live.

• Hit Submit or Activate. Keep in mind it takes 20 to 30 minutes for the change to actually compile and distribute across Google's pipeline.

• Wait a bit, then run a UDM search on new incoming traffic to verify the fields are finally populating correctly instead of dumping into the unparsed bucket.

Alert on Ingestion Drops

Do not rely on manual dashboard checks alone. Set up monitoring so that if a critical log source's volume drops below 50% of its established baseline, your team is notified immediately. A firewall that normally sends 100,000 events per hour suddenly sending 10,000 could mean a configuration change, a network issue, or a compromised device suppressing its own logs.