Introduction

This guide provides a high-level technical overview of GTI's core YARA-based hunting features. It is designed to help analysts understand the application of each tool in a typical threat intelligence workflow.

For complete technical specifications, parameters, and API details, refer to the full, version-controlled documentation linked in each section.

1. LiveHunt: Real-Time Ingestion Stream Matching

Function: LiveHunt applies a user-defined set of YARA rules against the real-time stream of files ingested and analyzed by Google Threat Intelligence.

Mechanism:

• Rules are applied to every file submitted to GTI, including re-analyzed files.

• A match triggers an immediate notification, enabling real-time discovery.

• For PE files, LiveHunt performs automatic unpacking and scans both the packed and unpacked layers to increase detection efficacy.

Primary Use Cases:

• Discovering novel malware not yet covered by public signatures.

• Classifying malware families based on custom rulesets.

• Collecting specific file types (e.g., specific packers, new file formats).

• Flagging suspicious files with heuristic rules for further manual review.

Operational Parameters:

• LiveHunt does not scan files larger than 100MB.

Refer to the [Official LiveHunt Documentation] for complete specifications and rule management guides.

2. Retrohunt: Historical Corpus Scanning

Function: Retrohunt executes YARA rules against GTI's historical file corpus. This is primarily used for testing new rule efficacy and identifying historical threat campaigns.

Mechanism:

• Full Corpus Hunt: Scans the historical file database (12-month lookback for Hunting Pro users, 3-month for standard). A typical hunt scans over 500 million files (~680TB) and completes in approximately 2-3 hours.

• Goodware Testing: Validates a rule for potential false positives by running it against a 1-million-file set of known-benign files. This test typically completes in under 60 seconds.

Operational Parameters:

• Matches: Jobs are limited to 10,000 matches.

• Rules: A maximum of 300 YARA rules may be used per job.

• Rule Size: The total size of the ruleset cannot exceed 1MB.

• Concurrency: Limited to 10 concurrent Retrohunt jobs.

• File Size: Files larger than 100MB are not scanned.

Refer to the [Official Retrohunt Documentation] for all job parameters and syntax.

3. DIFF: Automated YARA Pattern Generation

Function: DIFF is an intelligent YARA rule generator that automates the identification of unique binary patterns specific to a given set of samples.

Mechanism:

1. Sample Submission: The analyst provides a set of file hashes (e.g., 10+ samples from the same malware family).

2. Comparative Analysis: DIFF compares the samples to identify common binary patterns.

3. Noise Filtering: The tool automatically filters out common patterns (e.g., standard compiler stubs, "This program cannot be run in DOS mode" strings) by checking their prevalence across the entire GTI dataset.

4. Pattern Generation: DIFF outputs optimized, low-noise YARA patterns that are highly specific to the provided samples, ideal for building high-confidence rules with low false-positive rates.

Refer to the [Official DIFF Documentation] for detailed usage guides.

4. IoC Stream: Operationalizing LiveHunt Detections

Function: The IoC Stream is a mechanism to operationalize LiveHunt matches by organizing them into real-time threat feeds.

Mechanism & Use Case: This feature aggregates all YARA matches from LiveHunt into dedicated feeds. These feeds can be analyzed directly within the GTI interface or exported for ingestion into other security tools. This allows analysts to feed high-fidelity, YARA-generated IoCs directly into a SIEM, SOAR, or EDR platform to automate defensive actions based on proactive hunting finds.

Refer to the [Official IoC Stream Documentation] for integration and export details.

___________________________________________________________

Ismahan El-Frih, Digital Customer Success & Excellence Team