CTime / MTime Change Monitoring/Tracking

Question

Hello,I'm parsing Qumulo JSON audit logs in Google SecOps and I'm able to successfully extract top-level and nested fields such as:{"operation": "fs_write_metadata","details": {"file_id": "9283XXXXXXXXXX329","path": "/smb/underwritingscan/Windows (C) - Shortcut.lnk","before": {"ctime": "2026-07-16T16:06:05.106082716Z"},"after": {"ctime": "2026-07-16T17:56:43.431152651Z"}}}-- Using statedump, I can confirm the parser is exposing: details.before.ctimedetails.after.ctime--- and temporary test fields are populated:before_ctime_test = 2026-07-16T16:06:05.106082716Zafter_ctime_test = 2026-07-16T17:56:43.431152651Z------Current UDM mappings work successfully for:principal.user.useridprincipal.iptarget.file.full_pathmetadata.product_log_idmetadata.product_event_type------- My goal is to expose the following fields in UDM for dashboarding:details.before.ctimedetails.after.ctimedetails.before.mtimedetails.after.mtime------I attempted to use:event.idm.read_only_udm.additional.fields---but received:repeated key "event.idm.read_only_udm.additional.fields" in option ---------------Questions:What is the recommended UDM field for storing before/after file timestamps?	Is additional.fields the correct approach?	How are others handling nested metadata values such as:	before.ctime		after.ctime		before.mtime		after.mtime		Any examples of dashboarding historical vs updated file metadata timestamps in Google SecOps?------------------The end goal is to build dashboard panels showing:Event TimestampBefore CTimeAfter CTimeBefore MTimeAfter MTimeUserFile PathOperation

whathehack81 · Accepted Answer

That error confirms the source reference is the issue.

[details][before][ctime] is not populated as a parser token at the point where the replace operation runs. Since your statedump already confirms these intermediate fields are populated:

before_ctime_test
after_ctime_test
before_mtime_test
after_mtime_test

use those fields directly rather than referencing the nested JSON path again.

For example:

if [before_ctime_test] != "" {
  mutate {
    replace => {
      "_qumulo_before_ctime.key" => "qumulo_before_ctime"
      "_qumulo_before_ctime.value.string_value" =>
        "%{before_ctime_test}"
    }
  }

  mutate {
    merge => {
      "event.idm.read_only_udm.additional.fields" =>
        "_qumulo_before_ctime"
    }
  }
}

Apply the same correction to the remaining values:

%{after_ctime_test}
%{before_mtime_test}
%{after_mtime_test}

The normalized timestamp block should likewise use the intermediate token:

if [after_mtime_test] != "" {
  date {
    match => [
      "after_mtime_test",
      "yyyy-MM-ddTHH:mm:ss.SSSSSSSSSZ",
      "ISO8601"
    ]
    target =>
      "event.idm.read_only_udm.target.file.last_modification_time"
    on_error => "_qumulo_after_mtime_parse_error"
  }
}

Also initialize the intermediate variables before the JSON extraction so events missing one of these fields do not fail during conditional or replacement processing:

mutate {
  replace => {
    "before_ctime_test" => ""
    "after_ctime_test" => ""
    "before_mtime_test" => ""
    "after_mtime_test" => ""
  }
}

Then allow the existing extraction logic to populate them.

Google’s parser documentation notes that replace fails when its source token does not exist, and recommends initializing intermediate variables before extraction when they may be absent from some events.

So the UDM model remains the same; only the source references in the mapping block need to be corrected.

R.Q whathehack81

whathehack81 · Answer

You are reaching the nested Qumulo values correctly. The issue is how the four values are being written into additional.fields, not the JSON extraction.

For the UDM mapping:

details.after.mtime can optionally map to target.file.last_modification_time, because that field represents the file’s current last-modified timestamp.
details.before.mtime should remain vendor-specific because UDM has only one scalar last_modification_time.
Both before.ctime and after.ctime should remain vendor-specific unless Qumulo explicitly defines ctime as file creation time. Do not automatically map ctime to target.file.create_time; UDM defines create_time as the time the file was created.

additional.fields is therefore the appropriate place to preserve the complete before/after state. Google documents it as the location for important vendor-specific event data that does not fit the formal UDM schema.

The error:

repeated key "event.idm.read_only_udm.additional.fields" in option

usually occurs when the same destination is declared multiple times inside one parser option block. additional.fields is interpreted as a set of key-value pairs, not as a normal repeated UDM field. Create a separate temporary key/value object for each value, and then merge each object separately into the same destination. This matches Google’s documented parser-extension pattern.

Using the temporary fields shown in the question, insert the following before the existing final @output merge:

if [before_ctime_test] != "" {
  mutate {
    replace => {
      "qumulo_before_ctime.key" => "qumulo_before_ctime"
      "qumulo_before_ctime.value.string_value" => "%{before_ctime_test}"
    }
  }
  mutate {
    merge => {
      "event.idm.read_only_udm.additional.fields" => "qumulo_before_ctime"
    }
  }
}

if [after_ctime_test] != "" {
  mutate {
    replace => {
      "qumulo_after_ctime.key" => "qumulo_after_ctime"
      "qumulo_after_ctime.value.string_value" => "%{after_ctime_test}"
    }
  }
  mutate {
    merge => {
      "event.idm.read_only_udm.additional.fields" => "qumulo_after_ctime"
    }
  }
}

if [before_mtime_test] != "" {
  mutate {
    replace => {
      "qumulo_before_mtime.key" => "qumulo_before_mtime"
      "qumulo_before_mtime.value.string_value" => "%{before_mtime_test}"
    }
  }
  mutate {
    merge => {
      "event.idm.read_only_udm.additional.fields" => "qumulo_before_mtime"
    }
  }
}

if [after_mtime_test] != "" {
  mutate {
    replace => {
      "qumulo_after_mtime.key" => "qumulo_after_mtime"
      "qumulo_after_mtime.value.string_value" => "%{after_mtime_test}"
    }
  }
  mutate {
    merge => {
      "event.idm.read_only_udm.additional.fields" => "qumulo_after_mtime"
    }
  }
}

The resulting UDM should expose:

additional.fields["qumulo_before_ctime"]
additional.fields["qumulo_after_ctime"]
additional.fields["qumulo_before_mtime"]
additional.fields["qumulo_after_mtime"]

You can also normalize the current after.mtime into the first-class file field while retaining the additional-field copy:

if [after_mtime_test] != "" {
  date {
    match => [
      "after_mtime_test",
      "yyyy-MM-ddTHH:mm:ss.SSSSSSSSSZ",
      "ISO8601"
    ]
    target => "event.idm.read_only_udm.target.file.last_modification_time"
    on_error => "after_mtime_parse_error"
  }
}

Google’s parser date function supports timestamp normalization into UDM timestamp fields, including ISO 8601 and nine-digit fractional-second formats.

For a table-style dashboard query, group by the event ID so that each audit event remains one row:

metadata.log_type = "<QUMULO_LOG_TYPE>"
additional.fields["qumulo_before_ctime"] != ""

$event_id = metadata.id

match:
  $event_id

outcome:
  $event_timestamp = timestamp.get_timestamp(
    min(metadata.event_timestamp.seconds),
    "%F %T",
    "UTC"
  )
  $before_ctime = array_distinct(
    additional.fields["qumulo_before_ctime"]
  )
  $after_ctime = array_distinct(
    additional.fields["qumulo_after_ctime"]
  )
  $before_mtime = array_distinct(
    additional.fields["qumulo_before_mtime"]
  )
  $after_mtime = array_distinct(
    additional.fields["qumulo_after_mtime"]
  )
  $user = array_distinct(principal.user.userid)
  $file_path = array_distinct(target.file.full_path)
  $operation = array_distinct(metadata.product_event_type)

order:
  $event_timestamp desc

limit:
  1000

Native dashboard queries require a match section, and outcome variables are the supported mechanism for exposing calculated or selected columns to dashboard widgets. timestamp.get_timestamp() can format the event epoch into a readable UTC timestamp.

Recommended final model:

metadata.event_timestamp
target.file.last_modification_time       # optional normalized after.mtime

additional.fields["qumulo_before_ctime"]
additional.fields["qumulo_after_ctime"]
additional.fields["qumulo_before_mtime"]
additional.fields["qumulo_after_mtime"]

This preserves the vendor’s before/after audit semantics without overloading scalar UDM file fields or losing the historical values.

Sign up

Login with SSO

Login to the community

Login with SSO