I took a quick look through the github repo and the only one that I found that matched the names on the list is community/microsoft/windows/port_proxy_forwarding_T1090_cisa_report.yaral.
That one is built based on the reference material cited from the CISA report and sections of the rule are commented that cover opportunities to tune and tweak the rule. The community rules are intended to be a starting point and it is assumed that they will require tuning for the specific environment they are deployed in.
I pulled this example rule together for excessive port scanning that I think is a decent start for that use case though additional tuning may be required.
This example has IP netblocks for 10.0.0.0/8 defined as start and end so this would be internal scan activities but could be adapted to external to internal by adding a NOT to the principal.ip netblock (assuming the 10 block is the internal block of interest).
It's worth mentioning that if you are looking for ext to int scans that principal hostname likely won't exist for those systems so using IP address for external scans might be better. I mocked up an IP to IP example and commented it out as well as a match statement. You could also mix and match ip and hostname if you want.
Finally, we are using the outcome section to generate a count of the distinct ports seen in our 5 min match window and then using a threshold in the condition section to look for more than 50 distinct ports in that 5 min window. That number is something you will need to tune based on the kind of scan you are looking to detect. An nmap generic scan gave me 1001 ports in that window but that could vary.
Depending upon the data sets you have, there may be other rules that could pick this up too. For example my suricata rules fired when i ran the nmap for this example and I could write a rule against that to detect scanning activity perhaps. I also had a bunch of NETWORK_UNCATEGORIZED activity that touched a large number of ports as well so that might be another event type i might throw into this rule (separated with an OR) to broaden my rule if needed.
rule excessive_port_activity {
meta:
author = "Google Cloud Security"
description = "Detects excessive (defined by number of distinct ports logged) between the same hosts within an internal network within a designated time window"
type = "alert"
data_source = "gcp, corelight"
severity = "Low"
priority = "Low"
events:
$net.metadata.event_type = "NETWORK_CONNECTION"
//following two lines focus that this scan is happening internal to internal
net.ip_in_range_cidr($net.target.ip, "10.0.0.0/8")
net.ip_in_range_cidr($net.principal.ip, "10.0.0.0/8")
//looking for specific combination of target and principal hostname for scan
$net.principal.hostname = $p_host
$net.target.hostname = $t_host
//alternatively, looking for specific combination of target and principal IP for scan - if external scan, principal will not have a hostname
//$net.principal.ip = $p_ip
//$net.target.ip = $t_ip
match:
$p_host, $t_host over 5m
//$p_ip, $t_ip over 5m
outcome:
$port_count = count_distinct($net.target.port)
condition:
$net and $port_count > 50
}
I took a quick look through the github repo and the only one that I found that matched the names on the list is community/microsoft/windows/port_proxy_forwarding_T1090_cisa_report.yaral.
That one is built based on the reference material cited from the CISA report and sections of the rule are commented that cover opportunities to tune and tweak the rule. The community rules are intended to be a starting point and it is assumed that they will require tuning for the specific environment they are deployed in.
I pulled this example rule together for excessive port scanning that I think is a decent start for that use case though additional tuning may be required.
This example has IP netblocks for 10.0.0.0/8 defined as start and end so this would be internal scan activities but could be adapted to external to internal by adding a NOT to the principal.ip netblock (assuming the 10 block is the internal block of interest).
It's worth mentioning that if you are looking for ext to int scans that principal hostname likely won't exist for those systems so using IP address for external scans might be better. I mocked up an IP to IP example and commented it out as well as a match statement. You could also mix and match ip and hostname if you want.
Finally, we are using the outcome section to generate a count of the distinct ports seen in our 5 min match window and then using a threshold in the condition section to look for more than 50 distinct ports in that 5 min window. That number is something you will need to tune based on the kind of scan you are looking to detect. An nmap generic scan gave me 1001 ports in that window but that could vary.
Depending upon the data sets you have, there may be other rules that could pick this up too. For example my suricata rules fired when i ran the nmap for this example and I could write a rule against that to detect scanning activity perhaps. I also had a bunch of NETWORK_UNCATEGORIZED activity that touched a large number of ports as well so that might be another event type i might throw into this rule (separated with an OR) to broaden my rule if needed.
rule excessive_port_activity {
meta:
author = "Google Cloud Security"
description = "Detects excessive (defined by number of distinct ports logged) between the same hosts within an internal network within a designated time window"
type = "alert"
data_source = "gcp, corelight"
severity = "Low"
priority = "Low"
events:
$net.metadata.event_type = "NETWORK_CONNECTION"
//following two lines focus that this scan is happening internal to internal
net.ip_in_range_cidr($net.target.ip, "10.0.0.0/8")
net.ip_in_range_cidr($net.principal.ip, "10.0.0.0/8")
//looking for specific combination of target and principal hostname for scan
$net.principal.hostname = $p_host
$net.target.hostname = $t_host
//alternatively, looking for specific combination of target and principal IP for scan - if external scan, principal will not have a hostname
//$net.principal.ip = $p_ip
//$net.target.ip = $t_ip
match:
$p_host, $t_host over 5m
//$p_ip, $t_ip over 5m
outcome:
$port_count = count_distinct($net.target.port)
condition:
$net and $port_count > 50
}
Hi @jstoner ,
The above rule will definitely help. Thank you for your quick response.
Thanks,
Neha.H
Neha, Once you get this rule enabled and are ready to test, I'd highly recommend if you are able to run your own nmap scans to test the rule. nmap has many options to modify the rate and speed of the scans as well as implement decoys. I'd use these not only to test your new rule but also assess any changes in your IPS / FW alerts. The man page and this link is a good place to start trying evasive tactics.: https://nmap.org/book/man-bypass-firewalls-ids.html
Neha, Once you get this rule enabled and are ready to test, I'd highly recommend if you are able to run your own nmap scans to test the rule. nmap has many options to modify the rate and speed of the scans as well as implement decoys. I'd use these not only to test your new rule but also assess any changes in your IPS / FW alerts. The man page and this link is a good place to start trying evasive tactics.: https://nmap.org/book/man-bypass-firewalls-ids.html
Hi @ScottieJ ,
Sure i will do it and make it a practice so that we can efficiently check whether the rule is working perfectly.
Thanks,
Neha.H
Thank you very much @jstoner for this reply. I've modified your string for my environment's use case (and used your external scan option) but am receiving an error now.
events:
$net.metadata.event_type = "NETWORK_CONNECTION"
AND principal.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
AND target.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
$net.principal.ip = $p_ip
$net.target.ip = $t_ip
match:
$p_ip over 5m
outcome:
$port_count = count_distinct($net.target.port)
condition:
$net and $port_count > 50
Error: compilation error parsing query: parser error: no viable alternative at input 'count_distinct' line: 10 column: 19-20 : invalid argument
It seems to not like this line:
$port_count = count_distinct($net.target.port)
I've been messing with it for the past few hours without luck.
You are missing the event variables for the principal.ip and target.ip in the events section.
AND $net.principal.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
AND $net.target.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
While I've got you here, while this looks a little longer, this is a bit more efficient since we don't need to regex each IP address and might be easier to troubleshoot (or not) but figured I would mention it. With the function we would put the NOT and parenthesis around the values we are looking to negate.
NOT(net.ip_in_range_cidr($net.principal.ip, "10.0.0.0/8") or net.ip_in_range_cidr($net.principal.ip, "172.16.31.0/12") or net.ip_in_range_cidr($net.principal.ip, "192.168.0.0/16"))
NOT(net.ip_in_range_cidr($net.target.ip, "10.0.0.0/8") or net.ip_in_range_cidr($net.target.ip, "172.16.31.0/12") or net.ip_in_range_cidr($net.target.ip, "192.168.0.0/16"))
Thanks @jstoner for the additional guidance on the IP search.
I've added the event variables you mentioned above as well as an event variable for target port ($net.target.port = $t_po) but I'm still getting the same error
"compilation error parsing query: parser error: no viable alternative at input 'count_distinct' line: 11 column: 19-20 : invalid argument"
events:
$net.metadata.event_type = "NETWORK_CONNECTION"
$net.principal.ip = $p_ip
$net.target.ip = $t_ip
$net.target.port = $t_po
AND $net.principal.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
AND $net.target.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
match:
$p_ip over 5m
outcome:
$port_count = count_distinct($net.target.port)
condition:
$net and $port_count > 50
Based on what you are showing me, I'm not sure what to tell you. I pasted it into my editor and it compiles nicely. The meta section should be at the top and the brackets at the beginning and end of the rule but those are not the reasons you are getting that error message on line 11. do you have other stuff there that isn't included that is causing this? Not sure what to tell you based on what I can see.
@jstoner Yea not sure what the problem is either. I have nothing else in my query except for what is copied above. A colleague ran the same query in their instance of Chronicle with the same error. Still working on it -- will let you know if I figure it out. Thanks for the support!
The things that jump to mind if you pasted it in would be looking for stray characters, for example the double quotes in gdocs versus the quotes you get in a rule editor are different and would create a syntax error, making sure all quotes are closed and the like.
The things that jump to mind if you pasted it in would be looking for stray characters, for example the double quotes in gdocs versus the quotes you get in a rule editor are different and would create a syntax error, making sure all quotes are closed and the like.
OK so to your point - when I open Chronicle and type in the query manually (as opposed to copying and pasting from my text doc) it works all the way up until the last line:
condition:
$net and $port_count > 50
As soon as I even type the word "condition" in the next line I get the same error
You used the term query earlier but i was looking at the subject of rules...This will work in rules, but will not work as constructed in a search because at the moment the concept of a condition exist for a search.
events:
metadata.event_type = "NETWORK_CONNECTION"
AND principal.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
AND target.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
principal.ip = $p_ip
match:
$p_ip over 5m
outcome:
$port_count = count_distinct(target.port)
order:
$port_count desc
Here is the search that I believe you are trying to do. There are a few differences in the structure of a search and a rule in YARA-L, condition being one of them, but the concept of event variables for now don't exist in search either.
We have a bunch of content including blogs and videos on building searches and building rules that I can point you to to help out with this as well if you'd like.
Ahh yes sorry I was referring to search. Your query worked in our environment -- good to know that "condition" only works in rules, not in search.
So is there any way within search to say "show me only IP's that have scanned over 50 ports in the last 5 minutes?"
If you wouldn't mind pointing me to those training resources I'd appreciate it.
All good, glad we were able to clarify. At the moment within search, you'd need to sort by that count and get the whole list and manually cut it off or add back in a limit statement for a row count. I realize that isn't optimal at the moment, but we have a request to be able to set a threshold similar to a condition for search that I am hoping becomes available soon.
Regarding resources, here are some that may be helpful
Expanding on @jstoner response, you have the ability to use an IF statement, so if the distinct count of target.port is a specific value, output 'YES', else 'NO' as an example. You can then add a filter to only show the outcome of the IF section to allow for easy export. As opposed to ordering it by the port count, exporting it to CSV and then cutting it to where for example the $p_ip (principal ip) port count is less than 5.
events:
metadata.event_type = "NETWORK_CONNECTION"
AND principal.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
AND target.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
principal.ip = $p_ip
match:
$p_ip over 5m
outcome:
$port_count = count_distinct(target.port)
$MoreThen5Ports = if(count_distinct(target.port) >= 5, "YES", "NO")
order:
$port_count desc
Expanding on @jstoner response, you have the ability to use an IF statement, so if the distinct count of target.port is a specific value, output 'YES', else 'NO' as an example. You can then add a filter to only show the outcome of the IF section to allow for easy export. As opposed to ordering it by the port count, exporting it to CSV and then cutting it to where for example the $p_ip (principal ip) port count is less than 5.
events:
metadata.event_type = "NETWORK_CONNECTION"
AND principal.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
AND target.ip != /^(10\\.|172\\.(1[6-9]|2[0-9]|3[0-1])\\.|192\\.168\\.)/
principal.ip = $p_ip
match:
$p_ip over 5m
outcome:
$port_count = count_distinct(target.port)
$MoreThen5Ports = if(count_distinct(target.port) >= 5, "YES", "NO")
order:
$port_count desc
@AymanC this is incredibly helpful and solves the use-case we were looking to detect within search. Much appreciated!
All good, glad we were able to clarify. At the moment within search, you'd need to sort by that count and get the whole list and manually cut it off or add back in a limit statement for a row count. I realize that isn't optimal at the moment, but we have a request to be able to set a threshold similar to a condition for search that I am hoping becomes available soon.
Regarding resources, here are some that may be helpful
Awesome thank you for these - we are new to Chronicle so looking for all the resources I can get.
So I guess now for this rule to be operational I would need to be able to run it at a set time (say every day at 5 pm) but it would need to look for intervals of say 5 minutes where over a certain amount of ports were scanned within that time window.
If I'm understanding the current rule correctly - I have set a time window with which to look for an excessive number of ports being scanned but this means that the rule would have to run every 5 minutes to detect this activity - is that correct?
Ideally - the rule would say look back 24 hours and show me any time a single IP has scanned more than X number of ports in X number of minutes
When running it as a rule, you can define the rule frequency as 10m, 1h or 24h. When the rule is ready, flip the switch in the rule option to Live and if you want to generate an alert, toggle that as well.
The match section of the rule is where your bucketizing the results and what the threshold you define in the condition section will come into play, that is if you want to be notified when you get 50 in 5 minutes, your match window should be $p_ip over 5m, whatever that udm field is that you want represented in the detection as the common field(s) in the detection.
If you are looking for the 50 in 5 minutes, I'd suggest starting by setting that in your rule, set the run frequency for 10m and turn the rule to Live, but not alerting. Review the output of the detection and see what you get and then tune the thresholding and match window if you want and modify the frequency if you start stretching your time window to be larger.
That makes sense - the rule frequency is defined in the rule settings, not the rule syntax. Thanks for clarifying.
hi jstoner
could you please help me create these rules follows , it will be very helpful for me
1 Suspicious_Powershell_Process_Executing_Encoded_Command
2 Suspicious_PowerShell_Cmdlets
3 Outbound_Activity_to_C&C_IP_Addresses
4 Traffic_Activity_from_Tor
5 SQL_Injection_Attempt
6 Potential_Cobalt_Strike_Activity
7 Scheduled_Cron_Task
8 Process_Launch_VT_Enrichment
9 Suspicious_Process_Injection
thanks in advance
@NASEEF one place you may want to start would be the github community for chronicle detections:
https://github.com/chronicle/detection-rules
