Connector health monitorin

Hi @yadavmanjeet65 can you share a little more with us. what type of connector(s) are you referring to? Does this have to do with feeds going down? We find that sometime this happens at the source, or an intermediary (network connection) rather than the target (SecOps). If you can share a little more with me I can try to help point you in the right direction. Thanks

Its was a M365 connector used to feed in the M365 incidents to Chronicle.

M365 Connector Pulls alerts and associated info ----> Creates an Alert --> Followed by Case creation. 

This isn't going to address the silent log source, but it does provide a method to alert when there is latency between the time between an event occurs within, in this case, O365/Entra ID, and when it is collected. I've found that sometimes there is a lag of several hours for some of the different events and while this relies of me getting the data, at least I know that the data is laggy.

Not sure if that helps but hopefully provides some food for thought.

rule latency_cloud_collection {
meta:
 author = "Google Cloud Security"
 description = "Detect Latency from Cloud Based Event Sources"
 mitre_attack_tactic = "Collection"
 mitre_attack_technique = "Data from Local System"
 mitre_attack_url = "https://attack.mitre.org/techniques/T1005/"
 mitre_attack_version = "v13.1"
 type = "alert"
 data_source = "o365"
 severity = "Low"
 priority = "Low"
events:
$event.metadata.vendor_name = "Microsoft"
(
$event.metadata.product_name = "Azure AD" or
$event.metadata.product_name = "Office 365" or
$event.metadata.product_name = "Azure AD Directory Audit"
)
$event.metadata.product_name = $productName
$latency = math.abs($event.metadata.ingested_timestamp.seconds - $event.metadata.event_timestamp.seconds)
$latency > 1800

match:
$productName over 1h

outcome:
$risk_score = 65
$latency_seconds = max($latency)
$event_count = count_distinct($event.metadata.id)
$target_application = array_distinct($event.target.application)
$security_description = array_distinct($event.security_result.description)
$security_summary = array_distinct($event.security_result.summary)

condition:
$event
}

Hi @yadavmanjeet65,  is this a standalone legacy SOAR instance (*.siemplify-soar.com) or Google SecOps with unified SIEM and SOAR (*.chronicle.security)?

 Hi @yadavmanjeet65 ,

If you have a tenant with BYOP enabled, your ingestion log data is now available in the project. We have a document that explains how you can set up a new Cloud Monitoring project using the Chronicle Collector. For what you want, I'd use the "Total Ingested Log Count" and then create a log_filter equal to the log type (I think that's OFFICE_365 for what you're describing?). Then you can use Observability Monitoring to check status and alert when the feed drops below a threshold.

The page that explains how to do this is here: https://cloud.google.com/chronicle/docs/ingestion/ingestion-notifications-for-health-metrics

If you're not using BYOP (you don't have a GCP project), you're probably using Chronicle SIEM and have Federated Auth. In that case, you can also get ingestion data as collected in the Chronicle Data Lake, assuming your license provides access to that?  There's a nice blog post from a colleague of mine that explains how to use this:

https://medium.com/@thatsiemguy/chronicle-ingestion-stats-metrics-4fa14386b9fc

HTH

One more approach for posterity - 

You can also retrieve connector statistics, including connection status, from an external monitoring system or from a SOAR playbook using the SOAR API as shown in the examples here:

https://github.com/jacobm3/google-secops-utils/tree/main/connector-health