Finding Unparsed events

You can now run raw searches through udm search page. 

Hello @srijankafle, the raw scan is your best approach. A few pointers that might help narrow things down a little:

• .* is a nice broad wildcard to leverage
• A dropdown of log sources (types) can be selected to narrow the search to a specific type of data, e.g.

• A time boundary is always applied to raw search, could this be narrowed further from whatever you're using?

If the above doesn't get you further ahead, you might find some more answers here: https://cloud.google.com/chronicle/docs/investigation/search-raw-log or feel free to follow up here with further questions.

Hi @chrisproudley ,

We are currently using the same method. However in an MSSP environment where there are thousands of events ingested every few seconds, finding the unparsed log is like searching for a needle in a haystack.

We have reached where a point where we have to reduce the timeframe to only include few minutes of logs (as the maximum result is capped) and identify unparsed logs and repeat this until we assume there are no other logs missing.
We are searching for a way to search just for unparsed log. As a engineer this is a very crucial part of the task that we would need and I do not see this being discussed anywhere else.

I’m still stuck with the same issue. The solution to apply a procedural filter (or the post-search filter suggested above) doesn’t work because the results cap at 10k which are all parsed fine, leading to a very misleading “No results found”. This solution above only works in tiny environments.

This is a fundamental SOC/SIEM maintenance task so it should be supported directly in the UI with a method that works in high volume environments.

We recently released this functionality in UDM Search:

raw = /.+/ parsed = false

That’s the ticket, thanks!