Raw logs (ingestion API) + custom Parser

Question

Good morning,

I have a question about log ingestion via the ingestion API. Initially, my logs contained only 13 fields for testing purposes. However, I later expanded them to include 22 fields. I built a custom parser, and when I preview it, the UDM output appears correct.

The problem I am having now is when I go to validate the parser it errors out and indicates that the older raw log with 13 fields is the cause which makes sense as the columns/fields are missing.

I was wondering is there a way to remove an individual log or even just remove all entries and start fresh?

Thanks in advance

bsalvatore · Accepted Answer

If the old logs contains a specific pattern (for example a label named 'test' or same fields contains a test value) you can define an if condition at the top of the parser to drop {} all logs contains a specific pattern.

SoarMike · Answer

If the old logs contains a specific pattern (for example a label named 'test' or same fields contains a test value) you can define an if condition at the top of the parser to drop {} all logs contains a specific pattern.

Just wanted to touch on this again I managed to implement pattern dropping based on time for the logs we do not need but I am still getting one more validation error. The issue is with this one it does not even specify use log so I can test on it just curious if you may have any ideas?

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded