Skip to main content

Hi 
I am trying to convert the below KQL query to yara L but it does not seem to be working , can someone suggest where i am going wrong 

 

EmailEvents
| where TimeGenerated > ago(30d)
| where SenderFromDomain <> "XXXXX.com" or SenderFromDomain <> "XXXXXX.com"
| where LatestDeliveryLocation == "Inbox/folder"
| summarize EmailCount = count() by RecipientEmailAddress, bin(TimeGenerated, 180m)
| where EmailCount > 500
| project TimeGenerated, RecipientEmailAddress, EmailCount

 

YARA 

 

rule WHS_High_Volume_of_Emails_from_Sender_Identifying_Potential_Spam_Activity
{
meta:
author = "ABC"
description = "Use-case detects when a user sends emails to an abnormally high amount of recipients."
severity = "Medium"
priority = "Medium"
mitre_tactic = "TA0042"
mitre_technique = "T1586.002"

events:
$event.metadata.log_type = "OFFICE_365"
$event.metadata.event_type = "EMAIL_UNCATEGORIZED"
$event.target.resource.name = "\\\\Inbox"
$event.network.email.mail_id != ""
re.capture ($event.network.email.from,"@(.*)") = $Domain
$sender = $event.network.email.from
$recipient= $event.network.email.to
//not $Domain in %Allowed_Domains

$threat_signature = strings.coalesce("High_Volume_of_Emails_from_Sender_Identifying_Potential_Spam_Activity", $event.security_result.threat_name, $event.additional.fields["signature"])
$targeted_host = strings.to_upper(strings.coalesce($event.target.hostname, $event.principal.hostname))


match:
$recipient over 3h


outcome:

$total_recipients = count($recipient)
$risk_score = max(60)
$threat_signature_list = array_distinct($threat_signature)
$activity_outcome_list = max(
if(any $event.security_result.action = "ALLOW" or any $event.security_result.action = "ALLOW_WITH_MODIFICATION" or any $event.security_result.action = "FAIL" or any $event.security_result.action = "UNKNOWN_ACTION" , 1, 0)
+ if(any $event.security_result.action = "BLOCK" or any $event.security_result.action = "QUARANTINE" or any $event.security_result.action = "CHALLENGE" , 2, 0)
)
$total_log_count = count_distinct($event.metadata.id)


condition:
#event > 500
}

 

@jstoner : Any assistance would be of great help 

Could you provide more information about what isn't working or what output/errors you're getting?


One thing to check is to ensure your UDM events have values in the expected fields that you're building around. For example, you may find that the value of `network.email.to` is null.


Can you do a UDM Search and find logs that match the conditions?  If you start with the Office365 logs (metadata.log_type) for a few days worth of events, can you apply filters via the Aggregation panel in Search until you have a pool of matching candidates?  If so, can you get results for a count of 1 ($event rather than #event>500)?   


 


Shameless plug, but I wrote a blog on troubleshooting YARA-L rules so here it is: https://www.googlecloudcommunity.com/gc/Community-Blog/New-to-Google-Secops-Top-Ten-YARA-L-Rules-Troubleshooting-Tips/ba-p/848769 


To @gkush point, stepping through via search, emulating the conditions (as near as possible) and then using comments and the test rule to isolate on troublesome parts of the rule is the way to go...


I boiled the rule down to this and removed things that did not appear to have any bearing on the results of the rule triggering and ended up with this. Note the 2 additional outcome variables I added so I can see the receipient count per event as well as the names so I can do a quick spot check on the condition (i did a smaller time and smaller threshold to validate but the concept applies).


 



rule WHS_High_Volume_of_Emails_from_Sender_Identifying_Potential_Spam_Activity {

meta:

events:
$event.metadata.log_type = "OFFICE_365"
$event.metadata.event_type = "EMAIL_UNCATEGORIZED"
$event.target.resource.name = "\\\\Inbox"
$event.network.email.mail_id != ""
re.capture ($event.network.email.from,"@(.*)") = $Domain
$sender = $event.network.email.from
$recipient= $event.network.email.to
//not $Domain in %Allowed_Domains

match:
$recipient over 3h

outcome:
$risk_score = 60
$recipient_name = array_distinct($recipient)
$recipient_count = count($recipient)

condition:
#event > 500
}


If all of this is looking good, then I would look to incrementally add the other components you had in your rule to round it out but based on what you are showing, this appears to be what the KQL is focused on. To @t-martin point, make sure that the data exists as such, i found some o365 logs using MipLabel in product_event_type which worked for me, but if to/from are populating correctly then that's cool, just introspect your data first...


 


 


Shameless plug, but I wrote a blog on troubleshooting YARA-L rules so here it is: https://www.googlecloudcommunity.com/gc/Community-Blog/New-to-Google-Secops-Top-Ten-YARA-L-Rules-Troubleshooting-Tips/ba-p/848769 


To @gkush point, stepping through via search, emulating the conditions (as near as possible) and then using comments and the test rule to isolate on troublesome parts of the rule is the way to go...


I boiled the rule down to this and removed things that did not appear to have any bearing on the results of the rule triggering and ended up with this. Note the 2 additional outcome variables I added so I can see the receipient count per event as well as the names so I can do a quick spot check on the condition (i did a smaller time and smaller threshold to validate but the concept applies).


 



rule WHS_High_Volume_of_Emails_from_Sender_Identifying_Potential_Spam_Activity {

meta:

events:
$event.metadata.log_type = "OFFICE_365"
$event.metadata.event_type = "EMAIL_UNCATEGORIZED"
$event.target.resource.name = "\\\\Inbox"
$event.network.email.mail_id != ""
re.capture ($event.network.email.from,"@(.*)") = $Domain
$sender = $event.network.email.from
$recipient= $event.network.email.to
//not $Domain in %Allowed_Domains

match:
$recipient over 3h

outcome:
$risk_score = 60
$recipient_name = array_distinct($recipient)
$recipient_count = count($recipient)

condition:
#event > 500
}


If all of this is looking good, then I would look to incrementally add the other components you had in your rule to round it out but based on what you are showing, this appears to be what the KQL is focused on. To @t-martin point, make sure that the data exists as such, i found some o365 logs using MipLabel in product_event_type which worked for me, but if to/from are populating correctly then that's cool, just introspect your data first...


 


 


Thanks @jstoner and @gkush : i have onboarded 0365 exchange audit logs and was trying to build this alert . When running above logic makes me wonder are they actually email events that i should be using. Should i build a spam alert (like above) using 0365 exchange audit?  The even type i am seeing does not seem to email traffic logs 

 

I believe logs i should use o365 message trace but i dont see that integration supported in google chronicle, is my understanding correct here? 


As I looked at my dataset it seemed like my MipLabel stuff worked well but i didn't dig into it hard so some testing would be in order to determine if that is the best. I can say the MailItemsAccessed are probably not a great choice mainly because those get generated not at a 1:1 with mail items and doing stuff like enumerating a mailbox via the graph api will result in a bunch of those events. The message trace is another option you could look at using but it is not available via O365 API and has a separate method for collection, that's just how MS has structured it. Message trace is nice in that it does give you receipient, sender, subject and attachment name/size if I recall and might be the best solution for what you are trying to do but i don't know if miplabel is a terrible answer either, it's just something i would suggest doing a little more testing on to determine how those events get created.


As I looked at my dataset it seemed like my MipLabel stuff worked well but i didn't dig into it hard so some testing would be in order to determine if that is the best. I can say the MailItemsAccessed are probably not a great choice mainly because those get generated not at a 1:1 with mail items and doing stuff like enumerating a mailbox via the graph api will result in a bunch of those events. The message trace is another option you could look at using but it is not available via O365 API and has a separate method for collection, that's just how MS has structured it. Message trace is nice in that it does give you receipient, sender, subject and attachment name/size if I recall and might be the best solution for what you are trying to do but i don't know if miplabel is a terrible answer either, it's just something i would suggest doing a little more testing on to determine how those events get created.


Thanks @jstoner , i did check miplabel it appears it is triggered when mail contains sensitivity label. Since spam email may not contain these i dont think it fits the bill. 

In Splunk we could pull message trace directly via API, in chronicle i was informed we need to export it to a storage bucket and then use feed API mechanism to pull. Any reason why it is not a direct API. 


i don't have any direct insight into why we are dumping it into a bucket and then pulling from there. I played with message trace a few years back with splunk and part of the challenge, at least at the time, was that you had those different output options in terms of reporting and at the time it was like a scheduled report that would run and output. Perhaps we need to take a fresh look at it and see if there is an opportunity to collect it directly rather than a bucket. I would suggest a feature request to that end might be a way to kickstart that.


Reply