An architecture walkthrough of a purple-team agent that turns detection logic into synthetic telemetry, validates it locally, sends it An architecture walkthrough of a purple-team agent that turns detection logic into synthetic telemetry, validates it locally, sends it to Google SecOps, and reports how far that evidence travelled.to Google SecOps, and reports how far that evidence travelled.
Purple-team validation works, but repeating it is expensive.
Testing a detection can mean preparing a host, deploying attack tooling, arranging an EDR exclusion, scheduling a window, running the technique, waiting for telemetry, and cleaning everything up afterwards.We wanted to use agentic AI to make that process easier to repeat, while keeping telemetry structure, verification, platform access, and evidence in deterministic code.
The system starts from the detection logic instead of executing the attack:
plain-language request → retrieve Sigma rules → generate telemetry → verify locally → mark synthetic data → approval → ingest → search logs → search detection → downstream alert/case behaviour
The model interprets the request, selects from supported detection logic, coordinates the tools, and explains the result.
In this post, we walk through the architecture and show two verified outputs from our own Google SecOps tenant: one where the complete detection path produced a case, and another where only part of the targeted path responded.
1. Reading the rule backwards
A traditional purple-team exercise starts with the behaviour: run the technique and observe what telemetry appears.
A Sigma detection rule already describes an observable it expects to see. For supported rules, the synthesis layer works backwards from those conditions and turns them into concrete event values.
Choosing which rule to read backwards is a retrieval problem of its own. A request is usually a behaviour, not a rule name, so the Phase 1 Windows Sigma corpus is parsed into a structured schema rather than treated as text, and indexed three ways at once. Structured filters narrow by log source, category, and MITRE technique. A keyword index matches the exact tool and API names that carry most of the signal mimikatz, sekurlsa, lsass where embeddings are weakest. And a local Chroma vector database handles intent that shares no vocabulary with the rule text, such as “dumping credentials from memory.” The three result sets are combined with Reciprocal Rank Fusion, because keyword scores and vector distances are not on the same scale and only their rankings are comparable. We deliberately avoided the simpler approach of chunking rules into a vector store and retrieving prose: a Sigma rule’s value is in its detection block, and flattening those fields, modifiers, and conditions into an embedding would discard the exact part the generator needs. Chroma runs locally and offline with no external service, so retrieval adds no dependency and no rule text leaves the machine.
For example, a credential-access rule may require:
- a process accessing lsass.exe
- a specific access mask
- a call trace containing a dump-related library
We do not run Mimikatz to create that evidence. However, we generate a labelled Sysmon-shaped event containing the observable the rule expects.
But satisfying a rule and producing plausible telemetry are two different requirements.
A condition such as:
TargetImage|endswith: '\lsass.exe'
can technically be satisfied by:
\lsass.exe
That satisfies the condition, but it is not a normal process path a Windows host would emit. So the generator expands the value into a plausible path while preserving the condition.
The same principle applies across the synthesis layer. Multiple modifiers on one field are composed deterministically. Rule exclusions are actively avoided. Conditions the generator cannot faithfully invert such as some regex, correlation, threshold, or field-reference logic remain unresolved or are reported as UNSUPPORTED.
2. From matching values to usable telemetry
Values that satisfy a detection rule are not yet usable Windows telemetry. The next layer turns those values into a coherent event. A deterministic mapping connects the Sigma rule to a Windows event type we know how to generate. If there is no supported mapping, the rule is declined rather than approximated.
Provider, channel, event ID, field names, numeric formats, hashes, and XML structure are generated by code. Events within the same run also share coherent context such as hostname, user, SID, timestamps, and process relationships.
Every generated event also receives a run-specific synthetic marker:
PT-LAB-<run-id>
Before anything is imported, the batch is checked for that marker. If even one event is missing it, the batch is reported unsafe and does not proceed.
The marker serves two purposes: it makes the telemetry attributable to a specific run, and it gives the verification workflow something concrete to search for afterwards.
That second purpose matters.
An ingestion success response is only a transport result. It does not prove the event became searchable.
So the import step also records the hostname queries and time window needed to retrieve that run. The verification workflow uses those searches to determine whether the synthetic telemetry actually reached the searchable layer before drawing any conclusion about detection behavior.
3. The agent coordinates; deterministic tools provide the evidence
The orchestration layer is built with Google Agent Development Kit.
The model handles the parts that benefit from interpretation:
- understanding the user's request
- retrieving relevant Sigma rules
- selecting from supported observables
- coordinating the workflow
- explaining the result
The underlying facts come from deterministic tools.
They answer questions such as:
- Did the generated event satisfy its source rule?
- Did the event become searchable?
- Which detections were returned?
- Did the expected alert or case evidence appear?
The agent reads those outputs and assembles the final report.The model coordinates. The tools establish the facts.
The same separation applies when the workflow reaches the live platform. Reaching that platform is itself an MCP connection. Google SecOps exposes a remote MCP server, and the ADK agent connects to it as a streamable-HTTP tool source, authenticated with Google Application Default Credentials so a fresh token is attached to every request and no long-lived key is stored. ADK treats those remote SecOps tools exactly as it treats the local generation and verification tools, which is what lets one model coordinate retrieval, search, and ingestion across a real tenant through a single consistent interface. The ingest call itself is run from code rather than emitted by the model, so a full batch of event XML never has to travel back through the language model to reach the platform.
Before Stage B begins, the agent asks for explicit human approval to ingest the synthetic telemetry. Today, that approval is enforced as part of the agent workflow rather than as a separate API-level authorization gate. Strengthening that boundary is part of the next phase of the work.
Before any request is sent to Google SecOps, the client checks the HTTP method and path against a fixed allowlist implemented in code.
Through this interface, telemetry ingestion is the only permitted write operation.
The model is allowed to coordinate a test and explain what happened. It is not given an open-ended interface for changing the system being tested.
4. “Nothing fired” is not one result
When the SIEM reports nothing after a run, the obvious conclusion is that the detection failed.
That can be wrong.
The same empty result can come from different points in the workflow: the generated event may have failed local verification, the platform may still be processing it, the event may never have become searchable, the expected detection may not have matched, or the downstream alert/case path may not have completed.
The important question is not simply whether something failed, but where the evidence stopped.
A locally invalid event says nothing about the deployed detection. An event that never became searchable points to ingestion rather than detection content. A searchable event with no expected detection match is a discrepancy worth investigating, not automatically proof that the technique is uncovered.
Timing can create the same ambiguity.
In our lab, searchability, detection evaluation, and case creation do not complete at the same time. An empty result checked too early can look identical to a genuine negative.
We initially tried telling the model to wait. Instead, the timing check is now deterministic: the tool will not return a verdict before the relevant stage is due.
If the workflow still cannot establish what happened, it says so.
“I could not verify this” is a valid result.
5. Two verified outputs from our tenant
The final report is designed to show how far the exercised observable travelled rather than reduce everything to a single pass/fail label.
Scenario 1 - Complete SIEM-side path
In one verified LSASS credential-access run, all three synthetic events were searchable at the first verification check, two minutes after import. A deployed detection matched, and a high-priority case was present by the downstream verification stage.
There is one important attribution detail.
The downstream SOAR alert did not carry the synthetic hostname. That means case attribution cannot rely on finding the run marker directly in the alert. Instead, the workflow links the case back through the associated detection.
Scenario 2 - Partial path with bounded search scope
A second verified run exercised three T1547.001 persistence rules.
All three generated events became searchable. One targeted rule matched, two did not, and no case followed. The system returned PARTIAL together with the scope of the search used to reach that verdict.
In this run, 11 deployed Sigma rules were examined specifically, rules whose names overlapped the three source rules being exercised.
That scope matters because the search was not exhaustive.
A later full-inventory review found that another, broader behavioural detection had also fired outside the name-overlap candidate set used by the run.
So the correct interpretation is not:
“The tenant lacked coverage for T1547.001.”
It is:
“Within the 11 deployed rules examined by this run, one expected detection matched and two did not.”
The broader detection firing outside that candidate set exposed a limitation in our current attribution strategy, not a lack of tenant-wide coverage.
We are tightening the verdict wording so that this distinction is stated directly rather than left for the reader to infer.
6. Phase 1 scope and what comes next
Our current Phase 1 Windows Sigma corpus contains 2,855 rules.
Of those, 429 are declined because the generator does not currently have a supported telemetry mapping for them. We do not approximate those rules using a nearby event type.
That leaves 2,426 attempted rules.
Of those attempted rules:
- 2,182 MATCH, 89.9%
- 241 NO_MATCH
- 3 UNSUPPORTED
- 0 build errors
MATCH means the generated event satisfied its source-rule check. It does not mean that rule has been verified through a live Google SecOps tenant.
Live verification is a narrower set.
So far, we have confirmed generation, ingestion, and searchability through our tenant for five Sysmon event types plus PowerShell script-block logging. The generator contains additional event templates that have not yet been exercised through the live path.
These are Phase 1 measurements, not claims of complete Sigma or MITRE ATT&CK coverage.
The system is currently strongest on single-event detection logic. The next phase is focused on broader rule shapes, correlation and time-window behaviour, stronger detection attribution, timing controls, structural approval enforcement, and evaluation of the model-driven parts of the workflow.
Internal evaluation
For internal testing, we containerized the agent and deployed it to Google Cloud Run so the wider security engineering team can evaluate the workflow without running it from a local development environment.
This is an internal evaluation environment, not a production service.
Our team is now exercising the agent across different requests, rules, generated telemetry, and verification outcomes. We are looking for cases where the workflow is confusing, the model makes a poor decision, a deterministic check is insufficient, or the final report gives more confidence than the underlying evidence supports.
This also helps us evaluate things that corpus measurements alone cannot answer, including how people interact with the agent, whether the approval flow is clear, and whether the reported evidence is actually useful to a detection engineer.
Public repository
The repository is now public.
The goal is for Stage A, including Sigma rule retrieval, inversion, event generation, synthetic marking, and local verification, to be usable without a Google SecOps tenant. Teams with an authorised environment can then connect Stage B to exercise the live detection path.
https://github.com/GHS-SOC/PurpleTeam-Agent
Built by Kumar Sashank Ghanta,, Farzaneh Abazari,, Ayat Kamona ,and Khanh Vu..
If you try it, we would genuinely value any feedback that helps make the system more accurate, safer, clearer, or more useful.
That could include incorrect Sigma interpretations, unrealistic generated telemetry, unsupported rule shapes, attribution or timing edge cases, workflow problems, usability issues, architectural concerns, or simply a better approach to something we have built.
If something looks wrong, incomplete, confusing, or unnecessarily complicated, we want to know.
This is Phase 1, and external review is an important part of how we expect the next phase to improve.
All measurements above reflect the current Phase 1 implementation and our own Google SecOps tenant. They should not be read as platform-wide guarantees.
