Stackstate-Splunk Metric Integration

Connects Splunk to Stackstate in order to:

Overview

  • Report metrics from a splunk saved search to StackState

The StackState Agent can be configured to execute splunk saved searches and provide the results as metrics to the StackState intake api. It will dispatch the saved searches periodically, specifying last metric timestamp to start with up until now.

The StackState Agent expects the results of the saved searches to contain certain fields, as described below in the Metric Query Format. If there are other fields present in the result, they will be mapped to tags, where the column name is the key, and the content the value. The Agent will filter out Splunk default fields (except _time), like e.g. _raw, see the Splunk documentation for more information about default fields.

The agent check prevents sending duplicate metrics over multiple check runs. The received saved search records have to be uniquely identified for comparison. By default, a record’s identity is composed of Splunk’s default fields _bkt and _cd. The default behavior can be changed for each saved search by setting the unique_key_fields in the check’s configuration. Please note that the specified unique_key_fields fields become mandatory for each record. In case the records can not be uniquely identified by a combination of fields then the whole record can be used by setting unique_key_fields to [], i.e. empty list.

Metric Query Format

All these fields are required.

_timelongData collection timestamp, millis since epoch
metric%stringName of the metric
value%numericThe value of the metric

\% The name of the field is configurable in the configuration file

Configuration

  1. Edit your conf.d/splunk_metric.yaml file.
  2. Restart the agent