Skip to main content

Building a connector which will ingest data from a source where there is no explicit unique event id

  • February 22, 2022
  • 1 reply
  • 0 views

Forum|alt.badge.img+4

Hi,
I'm currently building a connector which will ingest data from a source where there is no explicit unique event id. What's the best strategy for storing and evaluating whether an event has been seen before? I'm thinking I should concat a couple of unique values such as timestamp, IP url etc, and then hash them. The use the hash value to store and check to see if the event has been processed before, Though I'm not sure if this is a bit overkill? Any recommendations would be appreciated.

1 reply

Forum|alt.badge.img+4
  • Author
  • New Member
  • February 24, 2022

thanks for the input, went with an MD5 of the strings.