Google SecOps Field Notes: statedump — The Parser Debugger We all should use to make our parser creation process easier

If writing your first Google SecOps parser feels like typing in the dark, this is the light switch.

Let me say out loud what most tutorials skip: building a parser in Google SecOps is the least friendly part of the platform. Other SIEMs hand you a form, a wizard, a field-mapping screen. SecOps hands you a code block — a filter { } pipeline inherited from Logstash, full of things called grok, kv and mutate, with a regex dialect that isn't quite the one you know, and a preview that answers your mistakes with a single line: "No UDM events generated."

No line number you can use. No list of variables. No hint.

If you have spent years in Logstash, this feels like home. If you haven't — and most security analysts haven't — your first parser is a wall.

To be fair: the design is not wrong. A parser-as-code is versionable, reviewable, portable, and far more powerful than any mapping wizard — this is why serious detection engineering teams end up preferring it. The problem is not the model. The problem is the on-ramp: the platform assumes you can already see what the pipeline is doing, and gives a beginner no obvious way to look inside.

Except it does. It is called statedump, it lives in the official parser syntax reference (https://docs.cloud.google.com/chronicle/docs/reference/parser-syntax), and almost nobody starting out knows it exists.

This piece explains it the way I wish someone had explained it to me: what it is, what it is for, and how to use it — like you're four.

First, the mental model: a parser is a kitchen

Forget syntax for a minute.

A raw log arrives as one long string of text. Your parser is a kitchen line: each station does one small job and passes the work forward. One station cuts the timestamp out of the text. Another splits key=value pairs. Another renames things. The last one packs everything into the box that Google SecOps stores — the UDM event.

Between stations, everything lives on a big table: the ingredients extracted so far, each with a name and a value. Every station takes things from the table and puts things back on it.

That table is the parser's internal state. And here is the beginner's real problem, the one nobody names:

The table is invisible.

You write your stations, you run the preview, and you only ever see the final box — or an error saying there is no box. Whether the timestamp got cut correctly, whether the key-values split, whether that field you need is called user or username on the table right now — all hidden. So you guess, change something, re-run, and guess again. That loop is why parsing feels miserable.

What statedump is

statedump is a camera pointed at the table.

You write one line anywhere inside your parser:

statedump { label => "my_checkpoint" }

and when the preview runs, it prints a photo of everything on the table at that exact point — every field, every value, exactly as the pipeline sees them.

That's it. That is the whole tool. One line, one photo.

How to use it: your first statedump

Say a firewall sends this log (invented, but realistic):

2026-07-12T10:15:23Z fw01 action=allow src=10.0.0.5 user=jdoe bytes=4096

Your parser has two working stations — a grok that cuts off the timestamp and hostname, and a kv that splits the key=value pairs. Drop a statedump right after them:

grok {
  match => { "message" => ["%{TIMESTAMP_ISO8601:when} %{HOSTNAME:host} %{GREEDYDATA:payload}"] }
  overwrite => ["when", "host", "payload"]
  on_error => "not_grok"
}
kv {
  source => "payload"
  field_split => " "
  value_split => "="
  on_error => "not_kv"
}
statedump { label => "after_extraction" }

Run the preview (in the console: Settings → Parsers, the Self-Service Parsing editor — no special permissions needed). This appears, captured from a real run and trimmed:

Internal State (label=after_extraction):
{
  "action": "allow",
  "bytes": "4096",
  "host": "fw01",
  "message": "2026-07-12T10:15:23Z fw01 action=allow src=10.0.0.5 user=jdoe bytes=4096",
  "not_grok": false,
  "not_kv": false,
  "payload": "action=allow src=10.0.0.5 user=jdoe bytes=4096",
  "src": "10.0.0.5",
  "user": "jdoe",
  "when": "2026-07-12T10:15:23Z"
}

Read it like a beginner, because that is the point:

• message — the raw log, untouched. Always there.
• when, host, payload — the grok worked: it cut three pieces and put them on the table with those names.
• action, src, user, bytes — the kv worked: every key=value became a field.
• not_grok: false, not_kv: false — those on_error safety flags you attached: false means "that station did NOT fail."

Every question you were about to guess at — did my grok match? what did the kv actually produce? is the field called `user` or `username`? — is answered in one screen. No guessing. You are, for the first time, seeing what the parser sees.

What it is for: the three beginner questions

1. "Is my field even there?" You try to map %{username} and the UDM comes out empty. Statedump the table: the field is called user. Two minutes instead of forty.

2. "Did my extraction work?" The grok pattern is the most fragile line a beginner writes (one literal space matters). The dump after it tells you instantly: pieces on the table = it matched; only message on the table = it didn't.

3. "Where did my parser die?" This is the big one. Place three statedumps — after your initialization block, after extraction, right before the final @output merge — and use one rule:

The last statedump that prints marks where the pipeline died. The problem lives between that label and the next one.

Silence is not lack of information. Silence is the location of the bug.

The rule in action: the log that kept dying

One real case, reproduced and captured, because it is the exact wall most beginners hit first. A parser guards against malformed lines:

if [not_kv] != "" {
  drop { tag => "TAG_MALFORMED_MESSAGE" }
}

Reads fine: "if the kv failed, discard the log." The preview: no UDM events — on a valid log. Nothing is red. For a beginner, this is the moment the platform feels broken.

Now the protocol. The statedump after extraction prints; the one before @output never does. So the killer lives between them — and there is only one candidate: the drop. Scroll back up to the photo, and it was already telling you why:

"not_kv": false,

The kv succeeded, so the flag is the boolean false — not an empty string. And false != "" is true. The guard meant to discard broken logs discards every healthy one. Fix: test flags as booleans — if [not_kv] or if ![not_kv] — never against "".

Without the photo, that bug is an afternoon of despair. With it, it is two preview runs. (Statedump keeps paying off long after the beginner phase — it is also how you catch advanced traps like merge inserting references instead of copies — but that is another article.)

Important Disclaimer — the three rules of the camera

• Remove every statedump before you submit the parser. It is a debugging tool for the preview, not production code. No official parser ships with one; neither should yours.
• Use one to three sample logs while debugging, not five hundred. The camera shoots one photo per statedump per log line — with a big sample set the preview becomes the haystack.
• It shows you the state, not the reason. The photo says when is empty; you still have to work out that your grok is missing a literal space. It removes the guessing, not the thinking.

Use it when / don't use it when

Use it when:

• You are writing your first parsers and the preview's errors feel like riddles — make the three-checkpoint placement (init / extraction / pre-output) a habit from day one.
• A field comes out empty or lands in the wrong UDM spot and you don't know which station lost it.
• You are about to conclude "the platform is broken" or "the log is bad." Look at the table first.

Don't use it when:

• The parser doesn't compile. Statedump only exists at runtime; syntax errors are a different fight.
• You are measuring parsing success across thousands of lines — that is what the validation report is for. Statedump is a microscope, not a survey.

The point

Parsing in Google SecOps is genuinely hard to start: it is code where other products offer forms, and it assumes a Logstash fluency most security people were never given a chance to build. That on-ramp problem is real, and pretending otherwise helps nobody.

But most of the misery is not the syntax. It is working blind — guessing at an invisible table, run after run. statedump makes the table visible, it has been in the official documentation all along, and one line of it turns the guessing loop into a reading exercise.

This is not about becoming a parser expert overnight. It is about refusing to debug blind when the platform already gave you eyes.

— gromero-sec | Detection Engineering | medium.com|github.com/gromerosec/