Skip to main content

Locating Unparsed Logs in SIEM?

  • October 2, 2023
  • 2 replies
  • 59 views
M
Forum|alt.badge.img+2

I ingested 911K log lines and I can see in the "Data Ingestion and Health" dashboard that 8.2K failed, but I don't know how to find those so that I can update the custom parser to handle whatever issue they have. I was able to also create a custom dashboard to access the Drop Reason Code via BigQuery and see:

LOG_PARSING_DROPPED_BY_FILTER: TAG_MALFORMED_ENCODING

However, I can't do a Raw Log Search for that. I tried just doing a Raw Log Search for "." hoping that I could use the Procedural Filter with Event Type = Unparsed Log but since it caps out at 10,000 log lines, setting it to the window that I Ingested the 911K logs for quickly hits that limit. The only thing I can think to do is cut the Raw Log Search window down until it's <10,000 log lines (15-30min windows) and walking through the entire day until I can find the Event Type = Unparsed Logs. But there's gotta be an easier way, yah?

2 replies

I
Forum|alt.badge.img+6
  • ion_Bronze 2
  • Bronze 2
  • October 12, 2023

My workflow for this is to pull the logs using cbn_cli. You can then find something unique and see them in raw log search if you want, but realistically if a log is failing you usually need to do the following (all of which happens outside of the UI)
* Identify the bit thats failing parsing
* Grab the parser and fix the issue
* Generate a bunch of logs and see if you broke anything else
* Upload the updated parser and wait

Happy to explain in more detail if you're going to go down this path.

M
Forum|alt.badge.img+2
  • MNS
  • Author
  • New Member
  • October 12, 2023

My workflow for this is to pull the logs using cbn_cli. You can then find something unique and see them in raw log search if you want, but realistically if a log is failing you usually need to do the following (all of which happens outside of the UI)
* Identify the bit thats failing parsing
* Grab the parser and fix the issue
* Generate a bunch of logs and see if you broke anything else
* Upload the updated parser and wait

Happy to explain in more detail if you're going to go down this path.


Thanks for the response, I started going down a similar path. I think just being able to specify my own log file to validate against would be useful, which it seems can be done with the command:

chronicle_cli parsers run   

I'll try incorporating that into my workflow.

Terms & ConditionsCookie settingsAccessibility statement
Privacy Policy Terms of Service Gen AI Policy Community Guidelines Cookie Settings

© 2025 Google. All rights reserved.