Catching the Uncatchable: True Composite Detections for Advanced Persistent Threats

Author: David Nehoda, Technical Solutions Consultant

What We're Solving

Advanced Persistent Threats don't announce themselves with a single loud alert. They execute multi-stage attack chains with an initial phishing click, a malicious file drop, a process execution, and an outbound C2 beacon where each stage generates a low-severity alert that, in isolation, looks like noise. Your SOC receives 5,000 of these disconnected fragments per week. Analysts spend hours manually grouping them across Jira tickets and shift handoffs, trying to reconstruct the attack narrative after the fact. Most of the time, they can't because the context is lost between queue items, and the attacker has already completed their objective.

Alert fatigue isn't just an operational inefficiency. It's the #1 reason SOC analysts quit within 18 months. When every alert looks the same and none of them tell a story, the work becomes psychologically unsustainable.
 

What We're Delivering

This guide builds a True Composite Detection architecture in Google SecOps that:

• Collapses 5,000 low-severity alerts into 5 high-fidelity incidents per week; a 99% noise reduction
• Automatically stitches previously fired alerts into chronologically ordered kill chains; no manual Jira grouping
• Binds attack stages by shared identity and infrastructure; same user, same host, strict temporal sequence
• Fires only when the complete attack narrative is confirmed; file drop → process execution → C2 beacon, in order
• Delivers a single, CRITICAL-severity case to the analyst with full context from every stage pre-attached

How It Works

    BASE RULES (LOW severity)              

                                                      

  Rule A: File Drop    → exports $user, $hostname    

  Rule B: Process Exec → exports $user, $hostname    

  Rule C: C2 Beacon    → exports $user, $hostname    
                                                      

  (Each fires independently. LOW severity.            

   Individually = noise. Together = kill chain.)      

                        
                        ▼

       COMPOSITE RULE (CRITICAL severity)          
                                   

  Queries outcomes[] from Rules A, B, C              

  Binds on: same $user + same $hostname              

  Enforces: A → B → C (chronological order)      

  Window: 24h starting after Rule A fires            
                                         

  Fires ONLY when all 3 base alerts align            


                        ▼


                  SINGLE CASE    

                 CRITICAL alert 

                Full kill chain                

                 context ready  

The Numbers

Who This Is For

This guide is for Detection Engineers building base and composite rules, SOC Managers who need to understand the noise reduction architecture, and CISOs evaluating how their SIEM investment translates into actionable threat detection rather than raw alert volume.

Impact & ROI Detail

The Architecture: Base Rules → Composite Detections

Before writing a single line of composite logic, you must understand the fundamental architectural difference between multi-event correlation rules and true Composite Detections.
 

Multi-Event Correlation (What Most SOCs Do)

A standard multi-event YARA-L rule queries raw UDM logs directly. It correlates raw telemetry events,like firewall logs, EDR process launches, and authentication records, within a single rule using shared variables and match windows. This is powerful, but it has a ceiling: as the attack chain grows beyond 3–4 stages, the rule becomes impossibly complex, the match window must expand to accommodate timing variance, and the evaluation engine slows under the computational load.
 

True Composite Detections (The Next Level)

A Composite Detection does not search raw UDM logs. Instead, it queries the outcomes arrays generated by other YARA-L rules that have already fired. Think of it as a detection that hunts detections; a meta-rule that stitches together previously confirmed alerts into a higher-order narrative.

This architecture has three critical implications:

1. Base Rules must explicitly export data into outcome. If your Base Rule doesn't populate the outcome section with the variables the Composite rule needs (username, hostname, file hash, etc.), the Composite rule has nothing to correlate, and thus the chain breaks silently.
2. Composite rules evaluate against detection metadata, not raw events. The $d1.detection.detection.rule_name syntax targets a specific Base Rule by name. The $d1.detection.detection.outcomes["variable_name"] syntax extracts the specific outcome values that Base Rule exported.
3. Chronological enforcement uses raw epoch timestamps from the underlying events. The $d3.detection.collection_elements.references.event.metadata.event_timestamp.seconds path reaches through the detection metadata to access the original UDM event's timestamp, enabling strict temporal ordering.

Below: a complete kill chain implementation; file drop → process execution → C2 beacon; built as three independent Base Rules feeding one master Composite rule.

The Base Rules: Building the Foundation

Each Base Rule is designed to be deliberately low-severity on its own. A file creation in a temp directory is not inherently malicious. An unsigned binary executing is suspicious but common. An outbound network connection to a known-bad IP is concerning but could be a false positive. The magic happens when all three occur on the same host, for the same user, in sequence.
 

Base Rule A: Suspicious File Drop

This rule fires when a file is created in a user-writable temporary directory—the most common staging location for initial payload drops. On its own, this is noise. Combined with process execution and network activity, it becomes the first link in a cyber kill chain.
 

rule Base_Suspicious_File_Creation {

  meta:

    author = "Detection Engineering"

    description = "Flags file creation in user-writable temp directories. Deliberately low-severity as a standalone detection—designed to feed the Composite Kill Chain rule."

    severity = "LOW"

    // Tag this rule so the SOAR platform knows it's a composite feeder, not a standalone alert

    tags = "composite_base_rule"



  events:

    // Target file creation events from Sysmon (Event 11), CrowdStrike, or any EDR

    $file.metadata.event_type = "FILE_CREATION"



    // Restrict to high-risk user-writable directories where payloads typically land

    // AppData\Local\Temp is the most common staging directory for initial access payloads

    // because standard user accounts have write access without triggering UAC prompts

    re.regex($file.target.file.full_path, `(?i).*\\AppData\\Local\\Temp\\.*`)



    // Bind the hostname for grouping — this becomes the join key in the Composite rule

    $file.principal.hostname = $hostname



  match:

    // Group events by hostname over a 5-minute window

    // This window should be short — we're looking for a burst of file drops, not gradual accumulation

    $hostname over 5m

  outcome:

    // CRITICAL: These exact keys are queried by the Composite rule via

    // $d1.detection.detection.outcomes["user"] and $d1.detection.detection.outcomes["hostname_out"]

    // If these are missing or misspelled, the Composite correlation chain breaks silently

    $user = array_distinct($file.principal.user.userid)

    $hostname_out = array_distinct($file.principal.hostname)

    $file_paths = array_distinct($file.target.file.full_path)



  condition:

    $file

}

Design decisions:

• Severity LOW: This rule should never generate a standalone SOC alert. It exists purely to feed the Composite layer. Configure it as alerting = false if your environment supports silent detection mode, or tag it so SOAR filters it from the main queue.
• $file_paths in outcome: We extract the actual file paths so the Composite rule can surface them in the final alert context, giving the analyst the exact payload location without requiring a manual UDM search.
• 5-minute match window: Deliberately short. A file drop that's part of an active attack chain happens within seconds to minutes of the execution phase—not hours later.
   

Base Rule B: Malicious Process Execution

This rule fires when a binary with a non-empty MD5 hash executes on a host. The hash requirement ensures we're tracking identified binaries, not system-generated transient processes.
 

rule Base_Malicious_Process_Launch {

  meta:

    author = "Detection Engineering"

    description = "Flags execution of tracked binaries (non-empty hash). Designed as a composite feeder rule for kill chain correlation."

    severity = "LOW"

    tags = "composite_base_rule"



  events:

    // Target process launch events — Sysmon Event 1, CrowdStrike ProcessRollup2, etc.

    $exec.metadata.event_type = "PROCESS_LAUNCH"



    // Require a non-empty hash — ensures we're tracking an identified binary

    // If the EDR sensor doesn't log hashes (misconfigured CrowdStrike, etc.), this rule won't fire

    $exec.target.process.file.md5 != ""



    // Bind hostname for cross-rule correlation

    $exec.principal.hostname = $hostname



  match:

    $hostname over 5m



  outcome:

    // Export the same variable names as Rule A — the Composite rule binds on these keys

    $user = array_distinct($exec.principal.user.userid)

    $hostname_out = array_distinct($exec.principal.hostname)

    $process_cmdline = array_distinct($exec.target.process.command_line)

    $process_hash = array_distinct($exec.target.process.file.md5)



  condition:

    $exec

}

Design decisions:

• MD5 hash filter: The md5 != "" check serves double duty: it ensures the binary is trackable (for downstream VirusTotal detonation), and it filters out system noise from transient processes that don't get hashed.
• $process_cmdline in outcome: The exact command line is often the single most valuable forensic artifact. Surfacing it in the Composite alert saves the analyst from digging through raw UDM events.

Base Rule C: Outbound C2 Beacon

This rule fires when a host initiates an outbound connection to a known Command & Control IP address maintained in a Data Table. As a standalone alert, it could be a false positive from an expired TI indicator. In the context of a file drop + process execution sequence, it confirms active compromise.
 

rule Base_Outbound_C2_Beacon {

  meta:

    author = "Detection Engineering"

    description = "Flags outbound network connections to known C2 infrastructure from Data Table. Composite feeder rule."

    severity = "LOW"

    tags = "composite_base_rule"



  events:

    // Target firewall, proxy, or EDR network connection events

    $net.metadata.event_type = "NETWORK_CONNECTION"



    // Match against the dynamic C2 Data Table — updated via API or manual upload

    // Using a Data Table instead of hardcoded IPs means the detection stays current

    // without rule modifications

    $net.target.ip in %known_c2_ips



    // Bind hostname for correlation

    $net.principal.hostname = $hostname



  match:

    $hostname over 5m



  outcome:

    // Standard composite export variables

    $user = array_distinct($net.principal.user.userid)

    $hostname_out = array_distinct($net.principal.hostname)

    $c2_destination = array_distinct($net.target.ip)

    $c2_port = array_distinct($net.target.port)



  condition:

    $net

}

Design decisions:

• Data Table (%known_c2_ips): Using a dynamic Data Table instead of hardcoded IPs means the threat intelligence team can update C2 indicators via API without touching the detection rule. When a new C2 IP is added, SecOps can sweep historical data against the updated table automatically.
• $c2_destination and $c2_port in outcome: The exact C2 IP and port are critical for network-level containment actions (firewall blocks, DNS sinkholing) that the SOAR playbook downstream will execute.

The Master Composite Detection

Now that the three Base Rules are firing and populating their outcome arrays, the Composite rule stitches them together. It fires only when all three Base Rules trigger on the same host, for the same user, in strict chronological order: file drop → process execution → network beacon.
 

rule Composite_Kill_Chain_Sequence {

  meta:

    author = "Detection Engineering"

    description = "Correlates a file drop, process execution, and outbound C2 beacon on the same host and user in strict chronological order. Fires only when all three base detections align within 24 hours."

    severity = "CRITICAL"

    // This is a composite rule — it queries detection outcomes, not raw UDM events

    tags = "composite_detection"



  events:

    // ──────────────────────────────────────────────

    // ALERT 1: THE FILE DROP

    // ──────────────────────────────────────────────

    // Target the specific base rule by its exact rule name string

    $d1.detection.detection.rule_name = "Base_Suspicious_File_Creation"

    // Extract the $user outcome variable exported by Alert 1

    // The key "user" must exactly match the outcome variable name in the base rule

    $userid = $d1.detection.detection.outcomes["user"]

    // Extract the $hostname_out variable exported by Alert 1

    $hostname = $d1.detection.detection.outcomes["hostname_out"]



    // ──────────────────────────────────────────────

    // ALERT 2: THE PROCESS LAUNCH

    // ──────────────────────────────────────────────

    // Target the process execution base rule

    $d2.detection.detection.rule_name = "Base_Malicious_Process_Launch"

    // Bind to the SAME $userid — this is the cross-rule join key

    // If Alert 2 fired for a different user, the bind fails and the composite doesn't fire

    $userid = $d2.detection.detection.outcomes["user"]

    // Bind to the SAME $hostname — ensures all three alerts are on the same machine

    $hostname = $d2.detection.detection.outcomes["hostname_out"]



    // ──────────────────────────────────────────────

    // ALERT 3: THE C2 BEACON

    // ──────────────────────────────────────────────

    // Target the network beacon base rule

    $d3.detection.detection.rule_name = "Base_Outbound_C2_Beacon"

    // Bind to the SAME $userid and $hostname as Alerts 1 and 2

    $userid = $d3.detection.detection.outcomes["user"]

    $hostname = $d3.detection.detection.outcomes["hostname_out"]



    // ──────────────────────────────────────────────

    // STRICT CHRONOLOGICAL ENFORCEMENT

    // ──────────────────────────────────────────────

    // Reach through the detection metadata to access the original UDM event timestamps

    // This path navigates: detection → collection_elements → references → event → metadata

    // The .seconds field contains the raw Unix epoch timestamp

    //

    // Enforce: Alert 3 (network beacon) must occur AFTER Alert 2 (process execution)

    // This prevents false correlations where the network event predates the process launch

    $d3.detection.collection_elements.references.event.metadata.event_timestamp.seconds >

      $d2.detection.collection_elements.references.event.metadata.event_timestamp.seconds



    // Enforce: Alert 2 (process execution) must occur AFTER Alert 1 (file drop)

    // The complete sequence is now: file drop → process launch → C2 beacon

    $d2.detection.collection_elements.references.event.metadata.event_timestamp.seconds >

      $d1.detection.collection_elements.references.event.metadata.event_timestamp.seconds



  match:

    // 24-hour correlation window

    // The clock starts ONLY when Alert 1 ($d1) fires — the "after $d1" syntax is critical

    // Without "after $d1", the engine evaluates a rolling 24h window continuously,

    // which is computationally expensive and can cause late-triggering behavior

    $userid over 24h after $d1



  condition:

    // All three base alerts must fire within the 24h window for the composite to trigger

    $d1 and $d2 and $d3

}

How the Correlation Engine Works (Step by Step)

1. Alert 1 fires: Base_Suspicious_File_Creation detects a file drop in \AppData\Local\Temp\ on WORKSTATION-42 by user jdoe. The 24-hour composite correlation clock starts.
2. Alert 2 fires: Base_Malicious_Process_Launch detects an unsigned binary executing on WORKSTATION-42 by user jdoe. The Composite engine checks: same Same $userid?  Same 

Same $hostname?

Timestamp after Alert 1? 

Two conditions met.

3. Alert 3 fires: Base_Outbound_C2_Beacon detects an outbound connection from WORKSTATION-42 by jdoe to 198.51.100.42:443. 

Same $userid?  

Same $hostname? 

Timestamp after Alert 2? 

All three conditions met.

4. Composite fires: A single CRITICAL alert surfaces in the SOAR queue containing the full kill chain narrative: file path, process command line, C2 destination—all extracted from the Base Rule outcomes. The analyst receives one actionable case instead of three disconnected low-severity tickets.
    

What Happens If the Sequence Breaks
 

Troubleshooting Composite Detections
 

Problem 1: Composite Rule Doesn't Fire (Silent Failure)

Most common cause: The Base Rules are not populating their outcome sections correctly.
 

Diagnostic steps:

1. Open the SecOps UI and navigate to the Base Rule's detection history.
2. Click on a specific detection instance and inspect the raw JSON payload.
3. Look for the outcomes object. Verify that the keys (user, hostname_out) exist and contain non-null values.
4. If outcomes is empty or missing, the Base Rule's outcome section has a syntax error, a field mapping failure, or the underlying UDM event doesn't contain the expected data.

Common mistakes:

• Typo in outcome key: The Composite rule queries outcomes["user"] but the Base Rule exports $username instead of $user. The key names must match exactly.
• Empty UDM field: The Base Rule maps $user = array_distinct($file.principal.user.userid) but the EDR vendor doesn't populate principal.user.userid for file creation events. The outcome exports an empty array, and the Composite binding fails.
• Rule not enabled: The Base Rule exists but is set to disabled or test mode. Disabled rules don't generate detections, so the Composite rule has nothing to query.

Problem 2: Composite Fires Too Late (Delayed Alert)

Cause: The match window is too large, or the Base Rules themselves are firing late due to upstream ingestion delays.
Diagnostic steps:

1. Check the Base Rule detection timestamps: compare metadata.event_timestamp vs metadata.ingested_timestamp. A large gap indicates the delay is upstream (vendor or forwarder), not in the Composite engine.
2. If Base Rules fire promptly but the Composite fires late, the over 24h window may be forcing the engine to hold excessive state. Shrink to over 4h or over 1h if the attack chain completes quickly.

Problem 3: Too Many False Composite Alerts

Cause: The Base Rules are too broad, generating thousands of detections that create spurious composite correlations.

Fix: Tighten the Base Rules with additional UDM filters:

• Add file extension filters to the file drop rule (e.g., .exe, .dll, .ps1)
• Add parent process filters to the process launch rule (e.g., only flag powershell.exe spawned by winword.exe)
• Use a curated, high-confidence C2 Reference List instead of a noisy bulk TI feed

Advanced Pattern: Adding a Fourth Stage (Lateral Movement)

The three-stage kill chain (file → process → C2) catches the initial compromise. To detect full APT campaigns, extend the Composite with a fourth Base Rule for lateral movement:

rule Base_Lateral_Movement_RDP {

  meta:

    author = "Detection Engineering"

    description = "Detects outbound RDP connections from a host to an internal server — potential lateral movement."

    severity = "LOW"

    tags = "composite_base_rule"



  events:

    $rdp.metadata.event_type = "NETWORK_CONNECTION"

    $rdp.target.port = 3389

    // Only internal-to-internal connections (not external RDP access)

    net.ip_in_range_cidr($rdp.target.ip, "10.0.0.0/8")

    $rdp.principal.hostname = $hostname



  match:

    $hostname over 5m



  outcome:

    $user = array_distinct($rdp.principal.user.userid)

    $hostname_out = array_distinct($rdp.principal.hostname)

    $lateral_target = array_distinct($rdp.target.ip)



  condition:

    $rdp

}

Then extend the Composite rule by adding a $d4 variable bound to Base_Lateral_Movement_RDP, with chronological enforcement ensuring RDP happens after the C2 beacon. The four-stage composite now catches: initial access → execution → command & control → lateral movement—the complete APT kill chain in a single CRITICAL alert.

Conclusion & Next Steps

Single-event alerts are noise. Composite Detections transform the SIEM from a firehose of disconnected signals into a precision instrument that surfaces only confirmed, multi-stage attack narratives.

Base Rules must explicitly export variables into the outcome section. Without that contract, the Composite rule has nothing to bind. Treat outcome variables like an API schema—document them, version them, and test them rigorously.

By collapsing thousands of LOW-severity alerts into a handful of CRITICAL composite incidents, the SOC achieves three things simultaneously: 99% noise reduction, sub-second correlation, and analyst retention through meaningful work.

Next steps: With high-fidelity composite alerts flowing, the natural progression is autonomous remediation. Proceed to the "Universal Phishing Containment Architecture" to bind these kill-chain detections directly to SOAR playbooks that execute containment actions without human intervention.