help with Query

Some additional context on top of @AymanC comment above...The concept of ANY/ALL is now available within UDM search like it previously was in rules. Until recently, there was an offset between the search behavior on repeated fields from rules so this capability was aligned.

We have had a recent ticket opened about this behavior with enum fields (like action) that our team is looking into, but the general rule of thumb here is that you would use ANY with = and ALL with != for folks who have not used this before.

@AymanC  and @jstoner  : Lets says there are multiple hash values 

security_result[0].about.file.sha256 = abcd

security_result[1].about.file.sha256 = xyz

 How would i assign an event variable here for the hash so i can compare it with the threat feeds . 
when i write as below it throws error message . 

I brought this up yesterday in another post, but you can use any/all when performing direct comparisons, but not for use with a placeholder variable or against a reference list. 

If you do something like this in you rule and test it, is it not working and matching on both of those hashes?

$event.security_result.about.file.sha256 = $hash
$ioc.graph.entity.file.sha256 = $hash (don't hold me to that exact field name)

$event.security_result.about.file.sha256 = $hash
$ioc.graph.entity.file.sha256 = $hash (don't hold me to that exact field name)

@jstoner : Since there are multiple hash values in a single event and they are represented in the array format  like this 

security_result[0].about.file.sha256

 If i write my query like below , wouldn't all the values be stored in one variable  and in that case when comparing it with threat feeds , it will never match right ? currently i am not seeing any results when i run it for last 2 weeks but i can see there are some hash which is malicious in VT.

security_result.about.file.sha256 = $hash

security_result[0].about.file.sha256

security_result.about.file.sha256 = $hash

Hi @rahul7514,

If the above solution that @jstoner is recommended is not working, does the below work at all in your use case (this may be onerous or result in lengthy alert generation delay depending on the amount)?

Kind Regards,

Ayman

@AymanC Yes this is what i am thinking to do ,however i can see an email which has 30 attachment(30 hashes) so it will be difficult for me to guess uptill how much i have to write in above format ,so was checking if there is any easy way 

I did a quick and dirty test to better illustrate what I would expect to occur when searching against an repeated field. I don't have multiple hash values in a security result like your example but i do have multiple rule_names that I will use for the example, the concept should be the same. The other thing I will mention is that I am using a reference list and a search as well but again same concepts should apply.

The initial search returns rows like this one with 2 rule names, the attack technique and the sysmon rule id, same concept where the repeated value is security_results[n]. I get 125 events, just standard query matching field and value.

In this second example, same time range same criteria except i have a reference list that has two values in it EventID: 1 (which all 125 events have in it) and the File and Directory Discovery technique that is in the event highlighted above.

This time we get 152 events. What's changed. When we added our criteria via the reference list you essentially are doing an unnest of that repeated field and matching against all value in the repeated. Notice in this image you get two events highlighted. They are the same event but because they match on both values in the list, they are represented twice.

So, I feel like you are safe with the join as I mentioned above, but I also feel strongly that all detections built should be tested against your data so you understand what is happening in the detection before fielding it.

Hope this helps.

@AymanC / @jstoner : I used both any and all still i can see those values getting populated ,where am i going wrong 

I've opened a ticket around this to get some additional insight. I tried a few additional things to mirror your search and found some similar challenges so I want to get some additional opinions on this one and see if there is something that needs to be fixed. 

Sure thanks @jstoner , please post here if any update on that.