At it’s base, grok is based on regular expressions, so you can surround a pattern with ()? to make it optional — for example (%{NUMBER:requestId})?, If there isn’t a grok pattern that suits your needs, you can always create a named extraction like this: (?<version>[\d\.]+) which would extract into version, a string that has any … Read more
All GREEDYDATA is is .*, but . doesn’t match newline, so you can replace %{GREEDYDATA:message} with (?<message>(.|\r|\n)*)and get it to be truly greedy.
Just create a template. run curl -XPUT localhost:9200/_template/template_1 -d ‘{ “template”: “*”, “settings”: { “index.refresh_interval”: “5s” }, “mappings”: { “_default_”: { “_all”: { “enabled”: true }, “dynamic_templates”: [ { “string_fields”: { “match”: “*”, “match_mapping_type”: “string”, “mapping”: { “index”: “not_analyzed”, “omit_norms”: true, “type”: “string” } } } ], “properties”: { “@version”: { “type”: “string”, “index”: “not_analyzed” … Read more
When possible, I’d go with a conditional wrapper just like the one you’re using. Feel free to post that as an answer! If your application produces only a few different line formats, you can use multiple match patterns with the grok filter. By default, the filter will process up to the first successful match: grok … Read more