Set up data processing using API methods to redact okta user context logs

Hi all, I am trying to test googles “Data processing using API methods” (No bindplane)

I was wondering has anyone tried to use this feature yet and have any tips.

I am testing with OKTA_USER_CONTEXT and trying to redact PII but just for testing I am trying to redact the “Title” field. I tried it but not sure if I can redact a whole field using blockedKeyPatterns or does it only work on with blockedValues.

Documentation used: 

Hi all,

I am testing Google SecOps Data Processing Pipelines using the API methods directly, without deploying Bindplane agents/collectors. My use case is OKTA_USER_CONTEXT, where I am trying to redact PII from the Okta user profile payload before the data is stored/parsed in Google SecOps.

For initial testing, I associated a data processing pipeline to a specific OKTA_USER_CONTEXT and confirmed the pipeline is associated using fetchAssociatedPipeline. The pipeline currently has a redactProcessor. 

I first tried testing field-based redaction using blockedKeyPatterns against the Okta profile field:

{
  "processors": [
    {
      "redactProcessor": {
        "allowAllKeys": true,
        "blockedKeyPatterns": [
          "^title$"
        ]
      }
    }
  ]
}

The sample Okta payload has the field nested under profile.title, for example:

"profile": {
  "email": "user@example.com",
  "login": "user@example.com",
  "startDate": "2022-01-01",
  "title": "Manager",
  "userType": "Employee"
},
"status": "ACTIVE",
"statusChanged": "2025-01-13T06:36:59.000Z"

However, I still see the profile.title value in new events, so I am trying to understand whether blockedKeyPatterns supports nested JSON keys such as profile.title, or whether it only matches top-level/flattened key names depending on how the payload is represented internally.

I am also testing blockedValues, which matches the Google examples for regex/value-based redaction:

{
  "processors": [
    {
      "redactProcessor": {
        "blockedValues": [
          "\\b[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}\\b",
          "Manager",
          "2022-01-01"
        ],
        "allowAllKeys": true
      }
    }
  ]
}

My questions are:

1. Does blockedKeyPatterns support nested JSON fields like profile.title, or only direct key names?
2. If using blockedValues, is there any way to scope value redaction to specific keys, or does it apply globally across all allowed keys?
3. For Okta user profile payloads, is the recommended approach to redact PII using blockedValues regex patterns rather than key-based redaction?

Any examples using OKTA_USER_CONTEXT with the API-based Data Processing Pipeline would be appreciated.

https://docs.cloud.google.com/chronicle/docs/ingestion/data-processing-pipeline#using_secops_data_pipeline_apis