StackState log files
StackState Self-hosted v4.6.x
Last updated
StackState Self-hosted v4.6.x
Last updated
This page describes StackState version 4.6.
Go to the .
In a Kubernetes setup, StackState functions are distributed and logs for each function are stored per pod and container. You can using kubectl, although for long term storage it is recommended to set up .
The most recent logs can be retrieved from Kubernetes using the . For example:
You can use the synchronization name to locate specific log information in a log snapshot. For example:
StackState logs are stored per pod and container. The table below shows the pod to access for logs relating to specific StackState functions. Note that actual pod names will include a number or random string suffix (e.g. stackstate-receiver-5b9d79db86-h2hkz) and may also include the release name specified when StackState was deployed as a prefix.
For long term storage of StackState log data, it is advised that you set up log aggregation on your Kubernetes cluster. This can be done using a third party system for storage such as Elasticsearch, Splunk or Logz.io and a log shipper such as Logstash or Fluentd.
For more details of how this can be done, check:
In a Linux setup, StackState keeps all log files in the var/log subdirectory of the StackState installation directory. By default this is /opt/stackstate/var/log. In case of a two-node installation, logs are kept in the var/log directory on each node. Note that the logs are node-specific - the StackState node keeps StackState related logs and the StackGraph node keeps logs related to StackGraph.
var/log detailsInside var/log, there is a subdirectory for each type of logs kept by StackState. By default, each subdirectory has a size cap of 2 GB; when exceeded, the oldest files are deleted. Example of var/log contents:
From StackState v1.15.0, the version of Elasticsearch used by StackState changed. Elasticsearch logs are now saved in ./elasticsearch7. The old ./elasticsearch subdirectory can be removed to free some disk space.
StackState keeps logs in files that have a maximum size of 50 MB. When a log file exceeds 50 MB size cap, it is divided into ordered parts, as in the below example:
StackState builds log files using the following default log pattern: "%date [%thread] %-5level %logger{60} - %msg%n"
You can access logs on a specific pod using the . For example:
Note that logs stored on pods will be regularly removed. For long term access to logs, it is advised that you set up for your Kubernetes cluster.
Shipping logs with
A complete overview of setting up
API (including topology, charts and settings)
stackstate-api
Checks
stackstate-checks
Data indexing into Elasticsearch
stackstate-mm2es (metrics) stackstate-e2es (events) stackstate-trace2es (traces) stackstate-sts2es (events generated by StackState)
Data ingestion
stackstate-receiver
Event handlers
stackstate-view-health
State propagation
stackstate-state
Synchronization
stackstate-sync
View health state
stackstate-view-health