Skip to main content
Solved

Ingestion API

  • September 22, 2024
  • 3 replies
  • 52 views

vishnu_manu
Forum|alt.badge.img+4

Community,

I recently integrated a log source using the Ingestion API. Specifically, I have a script that makes API calls, and the responses are sent to Chronicle via the Ingestion API.

The first time I ran the script, the logs were ingested successfully, and I could see them. However, after running the script 2-3 more times, even though there were no errors, the logs are not refelected in the instance and its been like 48hrs.

I'm wondering what the issue could be. If the logs were ingested successfully the first time, why are they not appearing on subsequent attempts? For your info, I'm trying to ingest the same logs again. Could it be that the Ingestion API prevents sending the same logs multiple times, or something like that?

Best answer by suzhuang

The Ingestion API uses batch IDs to track groups of logs. If you send a batch of logs with the same batch ID as a previous batch, the new logs will be discarded and not appear in your SecOps instance. Were the second/third logs you were trying to send from the same batch ID?

3 replies

suzhuang
Staff
Forum|alt.badge.img+6
  • Staff
  • Answer
  • September 23, 2024

The Ingestion API uses batch IDs to track groups of logs. If you send a batch of logs with the same batch ID as a previous batch, the new logs will be discarded and not appear in your SecOps instance. Were the second/third logs you were trying to send from the same batch ID?


vishnu_manu
Forum|alt.badge.img+4
  • Author
  • Bronze 2
  • September 24, 2024

The Ingestion API uses batch IDs to track groups of logs. If you send a batch of logs with the same batch ID as a previous batch, the new logs will be discarded and not appear in your SecOps instance. Were the second/third logs you were trying to send from the same batch ID?


How can I check the batch ID, and how do I ensure that each time I run my script, the batch ID is unique?


Rene_Figueroa
Staff
Forum|alt.badge.img+10

How can I check the batch ID, and how do I ensure that each time I run my script, the batch ID is unique?


Hi @vishnu_manu The batch IDs are internal information only available inside Google. The batch ID will be unique as long as duplicate logs are not sent.