How to handle non-matching Logstash grok filters

When possible, I’d go with a conditional wrapper just like the one you’re using. Feel free to post that as an answer!

If your application produces only a few different line formats, you can use multiple match patterns with the grok filter. By default, the filter will process up to the first successful match:

grok {
    patterns_dir => "./patterns"
    match => {
        "message" => [ 
              "%{BASE_PATTERN} %{EXTRA_PATTERN}",
              "%{BASE_PATTERN}",
              "%{SOME_OTHER_PATTERN}"
        ]
    }
}

If your logic is less straightforward (maybe you need to check the same condition more than once), the grep filter can be useful to add a tag. Something like this:

grep {
    drop => false #grep normally drops non-matching events
    match => ["message", "/took\s\d+/"]
    add_tag => "has_traceback"
}


...

if "has_traceback" in filter,logstash,logstash-grok {
    ...
}

Leave a Comment

error code: 521