Hello Everyone,

While working with custom parsers and entity ingestion in Google SecOps, I have come across two related issues around Entity visibility and UDM field availability. I am outlining them below:

1. Delay in Entity Visibility

• I created a custom parser and ingested entities.

• The logs appear immediately in Raw Log Search, but the corresponding entities only become visible in UDM Search and Native Dashboards after almost a full day.

• I ensured that timestamp fields such as graph.metadata.interval.start_time and graph.metadata.collected_timestampare set to the current time.

Interestingly:

• Entities ingested using the v2/entities:batchCreate endpoint (USER/ASSET) appear within minutes.

• However, when trying to ingest FILE type entities via the same endpoint, I encounter a 400 error.
  400

{

  "error": {

    "code": 400,

    "message": "Request contains an invalid argument.",

    "status": "INVALID_ARGUMENT"

  }

}

👉 Question: Why is there such a long delay for entities from custom parsers, and are there any best practices to ensure they appear in UDM Search and dashboards in near real-time?

Not sure why i am getting 400 while ingesting the `FILE` entitites using the v2/entities:batchCreate endpoint.

2. Missing UDM Fields in Raw Log Search

• For logs where we generate both Events and Entities, the parsed UDM fields are not visible in Raw Log Search.

• These UDM fields, however, appear correctly in Native Dashboards.

• For other logs where only Entities are parsed, UDM fields display as expected in Raw Log Search.

👉 Question: Is this the expected behavior, or is there a way to configure things so that parsed UDM fields are also visible in Raw Log Search when both Events and Entities are created?

<UDM parsing not showing>

<UDM parsing showing>