Heka Message Monitor

Analyzes each unique message in a data stream based on the specified hierarchy. This schema is used for automated monitoring of all message attributes for any change in behavior.

1. Sample Configuration

The defaults are commented out.

filename        = "heka_message_monitor.lua"
ticker_interval = 360
preserve_data   = true
message_matcher = "Uuid < '\003'" -- slightly greater than a 1% sample
timer_event_inject_limit = 1000

-- hierarchy           = {"Logger"}
-- max_set_size        = 255
-- samples             = 25
-- sample_interval     = 3600
-- histogram_buckets   = 25
-- exclude             = nil -- table of field names to exclude from monitoring/analyis
                             -- e.g. `{"Pid", "Fields[Date]"}` hierarchy will always be appended
-- preservation_version = 1  -- increase when altering any of the above configuration values

alert = {
--  disabled  = false,
  prefix    = true,
--  throttle  = 90,
  modules   = {
    email = {recipients = {"example@example.com"}},
  },
  thresholds = {
    -- pcc              = 0.3,  -- minimum correlation coefficient (less than or equal alerts)
    -- submissions      = 1000, -- minimum number of submissions before alerting in at least the current and two previous interval (pcc)
    -- duplicate_change = nil, -- +/- percentage of the duplicate range i.e. 50% of a 10-12% range is +/- 1% so anything outside of 9-13% will alert allowing the range to creep and self adjust; warning this limits it to only detecting large jumps. The minimum number of submissions before alerting is 50K due to the HLL estimation error.  The alert is based on the last full hour, not the current hour, as the estimate takes a while to stabilize for some fields.
  }
}

2. Analysis Behavior (based on subtype)

• unknown - no analysis
• unique - hyperloglog percent duplicate calculation (per interval)
• range - histogram analysis of the range
• set - histogram analysis of the enumerated set
• sparse - weights of each of the most frequent items are computed

source code: heka_message_monitor.lua