...

This query matches our 4XX alarm. All requests that had a 4XX error, except 401s (which are surprisingly common) and 404s (frequent bot-scan failuresscans), and orders them by most frequent code & URI combination

...

This simple query parses the userId from our MetricsFilter. The nodrop means that if the line doesn’t have a userId, we still preserve the line in our results, but the parsed userId shows up as blank.

(_source=bridge-exporter-prod or _source=bridgeworker-prod or _source=BridgeServer2-prod) not ("connection reset by peer" or " HTTP/1.1\" 404 " or "com.newrelic") | where _loglevel = "ERROR"

Find all ERROR logs across all 3 applications, ignoring new relic errors, 404s for URIs containing the string “error”, and TCP connection resets.

(_source=bridge-exporter-prod or _source=bridgeworker-prod or _source=BridgeServer2-prod) not (MetricsFilter or " HTTP/1.1\" ") | where isBlank(_loglevel)

Find all unparsed log messages across all applications.

Graphs

_source=BridgeServer2-Prod MetricsFilter | parse "\"elapsedMillis\":*}" as latency | num(latency) | timeslice 1h | pct(latency,50,95,99) by _timeslice | order by _timeslice asc

...