Manage monitors

StackState Self-hosted v5.0.x

This page describes StackState version 5.0.

Go to the documentation for the latest StackState release.

Overview

Monitors process 4T data, such as metrics, events and topology, to produce a health state for elements (components and relations). The states are calculated and attached to relevant topology elements by a specific monitor function that is selected by the user.

Monitors are run by a dedicated subsystem of StackState called the monitor runner. The main task of the monitor runner is to schedule the execution of all existing monitors in such a way as to ensure that all of them produce viable results in a timely manner. The monitor runner is maintenance free - it starts whenever StackState starts and picks up any newly applied monitor definitions automatically whenever they are created, changed or removed. Any changes that have been applied to the monitors are reflected with the next execution cycle.

Add a monitor

Most Monitors in StackState are created as part of a StackPack installed by the user. They are added automatically upon installation and start producing health state results immediately afterwards, no further user action is required. Monitors automatically handle newly created topology elements and do not need to be specifically reconfigured after any topology changes occur or otherwise added to the newly created elements.

• Details of the monitor functions provided by StackPacks can be found in the StackPack documentation.

• You can create a custom monitor from scratch using the StackState CLI.

It might be beneficial to modify an existing monitor definition to change its parameters, run interval or to disable it. All of these actions are done by utilizing the StackState CLI and are described in greater detail in the following sections.

Make and apply changes to a monitor

Monitor configuration can be changed by modifying the monitor definition.

1. Find the ID or the identifier of the monitor to be modified. For example:

   • In the StackState UI: Inspect the monitor definition using the context menu (...) of the monitor result panel.

   • In the StackState CLI: List the monitors using sts monitor list or stac monitor list.

2. Export the monitor definition into a file named path/to/export.stj:

   • new sts CLI: sts settings describe --ids <id-of-a-monitor> -f path/to/export.stj

   • stac CLI: stac monitor describe <id-or-identifier-of-a-monitor> > path/to/export.stj

3. Modify the exported file to change the monitor parameters or intervalSeconds.

4. Apply the changes to the monitor:

   • new sts CLI: sts monitor apply -f path/to/export.stj

   • stac CLI: stac monitor apply < path/to/export.stj

Once applied, the updated monitor definition will be in effect. Changes will be reflected with the next execution cycle.

Set the run interval for a monitor

The monitor runner schedules monitor execution using an interval parameter that is configured on a per-monitor basis - the intervalSeconds. The runner will attempt to schedule a monitor execution every intervalSeconds, counting from the end of the previous execution cycle, in parallel to the other existing monitors (subject to resource limits). For example, setting intervalSeconds of a monitor definition to the value 600 will cause the monitor runner to attempt to schedule the execution of this monitor every ten minutes, assuming that the execution time itself is negligible.

To set a new run interval for a monitor, adjust the intervalSeconds parameter in the monitor STJ definition as described in the instructions to make and apply changes to the monitor.

For example, to run the monitor every 5 minutes, set the intervalSeconds to 300:

Monitor STJ definition

{
  "_version": "1.0.39",
  "timestamp": "2022-05-23T13:16:27.369269Z[GMT]",
  "nodes": [
    {
      "_type": "Monitor",
      "name": "CPU Usage",
      "description": "A simple CPU-usage monitor. If the metric is above a given threshold, the state is set to CRITICAL.",
      "identifier": "urn:system:default:monitor:cpu-usage",
      "remediationHint": "Turn it off and on again.",
      "function": {{ get "urn:system:default:monitor-function:metric-above-threshold" }},
      "arguments": [
        ...
      ],
      "intervalSeconds": 300
    }
  ]
}

Check the status of a monitor

The status of a monitor can be obtained via the StackState CLI:

CLI: sts (new)

# By ID
sts monitor status --id <id-of-a-monitor>
# By Identifier
sts monitor status --identifier <identifier-of-a-monitor>

⚠️ PLEASE NOTE - from StackState v5.0, the old sts CLI has been renamed to stac and there is a new sts CLI. The command(s) provided here are for use with the new sts CLI.

➡️ Check which version of the sts CLI you are running

CLI: stac

stac monitor status <id-or-identifier-of-a-monitor>

⚠️ PLEASE NOTE - from StackState v5.0, the old sts CLI is called stac.

In a future release of StackState, the new sts CLI will fully replace the stac CLI. It is advised to install the new sts CLI and upgrade any installed instance of the old sts CLI to stac. For details see:

• Which version of the sts CLI am I running?

• Install the new sts CLI and upgrade the old sts CLI to stac

• Comparison between the CLIs

The output of this command indicates the specific errors that occurred along with the counts of how many times they happened and the health stream statistics associated with this monitor. Any execution issues are also logged in the global StackState log file.

Preview a monitor

You can use the CLI run a monitor and preview its output without persisting its results.

CLI: sts (new)

# By ID
sts monitor run --id <id-of-a-monitor>
# By Identifier
sts monitor run --identifier <identifier-of-a-monitor>

⚠️ PLEASE NOTE - from StackState v5.0, the old sts CLI has been renamed to stac and there is a new sts CLI. The command(s) provided here are for use with the new sts CLI.

➡️ Check which version of the sts CLI you are running

CLI: stac

stac monitor preview <id-or-identifier-of-a-monitor>

⚠️ PLEASE NOTE - from StackState v5.0, the old sts CLI is called stac.

In a future release of StackState, the new sts CLI will fully replace the stac CLI. It is advised to install the new sts CLI and upgrade any installed instance of the old sts CLI to stac. For details see:

• Which version of the sts CLI am I running?

• Install the new sts CLI and upgrade the old sts CLI to stac

• Comparison between the CLIs

Disable a single monitor

When a monitor is disabled (removed), all health states associated with the monitor will also be removed and they will no longer be visible in the StackState UI.

Monitors can be disabled by removing them. Once a monitor to be disabled is identified, either by inspecting the definition of a monitor available under the context menu of a monitor result panel, or otherwise by obtaining the Monitors identifier, a dedicated CLI command can be used to remove it:

CLI: sts (new)

# By ID
sts monitor delete --id <id-of-the-monitor>
# By Identifier
sts monitor delete --identifier <identifier-of-the-monitor>

⚠️ PLEASE NOTE - from StackState v5.0, the old sts CLI has been renamed to stac and there is a new sts CLI. The command(s) provided here are for use with the new sts CLI.

➡️ Check which version of the sts CLI you are running

CLI: stac

stac monitor delete --identifier <identifier-of-the-monitor>

⚠️ PLEASE NOTE - from StackState v5.0, the old sts CLI is called stac.

In a future release of StackState, the new sts CLI will fully replace the stac CLI. It is advised to install the new sts CLI and upgrade any installed instance of the old sts CLI to stac. For details see:

• Which version of the sts CLI am I running?

• Install the new sts CLI and upgrade the old sts CLI to stac

• Comparison between the CLIs

Disable the monitor runner

The monitor runner subsystem can be disabled in the StackState configuration by appending the following line at the end of the file etc/application_stackstate.conf:

stackstate.featureSwitches.monitorRunner = false

See also

• StackState sts CLI

• Integrations