Skip to main content

I ingested 911K log lines and I can see in the "Data Ingestion and Health" dashboard that 8.2K failed, but I don't know how to find those so that I can update the custom parser to handle whatever issue they have. I was able to also create a custom dashboard to access the Drop Reason Code via BigQuery and see:

LOG_PARSING_DROPPED_BY_FILTER: TAG_MALFORMED_ENCODING

However, I can't do a Raw Log Search for that. I tried just doing a Raw Log Search for "." hoping that I could use the Procedural Filter with Event Type = Unparsed Log but since it caps out at 10,000 log lines, setting it to the window that I Ingested the 911K logs for quickly hits that limit. The only thing I can think to do is cut the Raw Log Search window down until it's <10,000 log lines (15-30min windows) and walking through the entire day until I can find the Event Type = Unparsed Logs. But there's gotta be an easier way, yah?

My workflow for this is to pull the logs using cbn_cli. You can then find something unique and see them in raw log search if you want, but realistically if a log is failing you usually need to do the following (all of which happens outside of the UI)
* Identify the bit thats failing parsing
* Grab the parser and fix the issue
* Generate a bunch of logs and see if you broke anything else
* Upload the updated parser and wait

Happy to explain in more detail if you're going to go down this path.

My workflow for this is to pull the logs using cbn_cli. You can then find something unique and see them in raw log search if you want, but realistically if a log is failing you usually need to do the following (all of which happens outside of the UI)
* Identify the bit thats failing parsing
* Grab the parser and fix the issue
* Generate a bunch of logs and see if you broke anything else
* Upload the updated parser and wait

Happy to explain in more detail if you're going to go down this path.


Thanks for the response, I started going down a similar path. I think just being able to specify my own log file to validate against would be useful, which it seems can be done with the command:

chronicle_cli parsers run   

I'll try incorporating that into my workflow.

Reply


Terms & ConditionsCookie settings
Privacy Policy Terms of Service Gen AI Policy Community Guidelines Cookie Settings

© 2025 Google. All rights reserved.