LogoLogo
StackState.comDownloadSupportExplore playground
StackState v6.0
StackState v6.0
  • StackState docs!
  • Docs for all StackState products
  • 🚀Get started
    • Quick start guide
    • StackState walk-through
    • SUSE Rancher Prime
      • Air-gapped
      • Agent Air-gapped
  • 🦮Guided troubleshooting
    • What is guided troubleshooting?
    • YAML Configuration
    • Changes
    • Logs
  • 🚨Monitors and alerts
    • Monitors
    • Out of the box monitors for Kubernetes
    • Notifications
      • Configure notifications
      • Notification channels
        • Slack
        • Teams
        • Webhook
        • Opsgenie
      • Troubleshooting
    • Customize
      • Add a monitor using the CLI
      • Override monitor arguments
      • Write a remediation guide
  • 📈Metrics
    • Explore Metrics
    • Custom charts
      • Adding custom charts to components
      • Writing PromQL queries for representative charts
      • Troubleshooting custom charts
    • Advanced Metrics
      • Grafana Datasource
      • Prometheus remote_write
      • OpenMetrics
  • 📑Logs
    • Explore Logs
    • Log Shipping
  • 🔭Traces
    • Explore Traces
  • 📖Health
    • Health synchronization
    • Send health data over HTTP
      • Send health data
      • Repeat Snapshots JSON
      • Repeat States JSON
      • Transactional Increments JSON
    • Debug health synchronization
  • 🔍Views
    • Kubernetes views
    • Custom views
    • Component views
    • Explore views
    • View structure
      • Filters
      • Overview perspective
      • Highlights perspective
      • Topology perspective
      • Events perspective
      • Metrics perspective
      • Traces perspective
    • Timeline and time travel
  • 🕵️Agent
    • Network configuration
      • Proxy Configuration
    • Using a custom registry
    • Custom Secret Management
    • Request tracing
      • Certificates for sidecar injection
  • 🔭Open Telemetry
    • Getting started
    • Open telemetry collector
    • Languages
      • Generic Exporter configuration
      • Java
      • Node.js
      • .NET
      • Verify the results
    • Troubleshooting
  • CLI
    • StackState CLI
  • 🚀Self-hosted setup
    • Install StackState
      • Requirements
      • Kubernetes / OpenShift
        • Kubernetes install
        • OpenShift install
        • Required Permissions
        • Non-high availability setup
        • Small profile setup
        • Override default configuration
        • Configure storage
        • Exposing StackState outside of the cluster
      • Initial run guide
      • Troubleshooting
        • Logs
    • Configure StackState
      • Slack notifications
      • Stackpacks
    • Release Notes
      • v1.11.0 - 18/07/2024
      • v1.11.3 - 15/08/2024
      • v1.11.4 - 29/08/2024
      • v1.12.0 - 24/10/2024
      • v1.12.1 - 08/11/2024
    • Upgrade StackState
      • Steps to upgrade
      • Version-specific upgrade instructions
    • Uninstall StackState
    • Air-gapped
      • StackState air-gapped
      • StackState Kubernetes Agent air-gapped
    • Data management
      • Backup and Restore
        • Kubernetes backup
        • Configuration backup
      • Data retention
      • Clear stored data
    • Security
      • Authentication
        • Authentication options
        • File-based
        • LDAP
        • Open ID Connect (OIDC)
        • KeyCloak
        • Service tokens
      • RBAC
        • Role-based Access Control
        • Permissions
        • Roles
        • Scopes
      • Self-signed certificates
  • 🔐Security
    • Service Tokens
    • Ingestion API Keys
  • ☁️SaaS
    • User Management
  • Reference
    • StackState Query Language (STQL)
    • Chart units
Powered by GitBook
LogoLogo

Legal notices

  • Privacy
  • Cookies
  • Responsible disclosure
  • SOC 2/SOC 3
On this page
  • Prerequisites
  • Configuring your Prometheus instance
  • Header authentication
  • Basic authentication
  • Finishing up
  1. Metrics
  2. Advanced Metrics

Prometheus remote_write

StackState v6.0

PreviousGrafana DatasourceNextOpenMetrics

Last updated 11 months ago

When you have your own on-premise or self-hosted Prometheus running where metrics for your environment are aggregated, you can mirror these metrics in StackState. This will allow you to use StackState's powerful topology and correlation features to troubleshoot your Kubernetes environment without having to switch between tools.

To make this possible, StackState exposes the Prometheus remote_write protocol as an endpoint. Using this endpoint you can configure your Prometheus instance to send metrics to StackState. The metrics are then automatically ingested, and it will be possible to bind the metrics to the components observed by StackState. The following diagram shows how this works:

Prerequisites

To mirror your Prometheus metrics in StackState, you need to lookup the API Key that's used to send in metrics into StackState. The API Key can be found in the description of the installed Kubernetes StackPack in StackState. The following steps show how to find the API Key:

  1. Open the StackState UI and navigate to the StackPacks page

  2. Find the Kubernetes StackPack and click on it.

  3. In the description of the StackPack, you will find the API Key that's used to send in metrics into StackState.

Configuring your Prometheus instance

To configure your Prometheus instance to send metrics to StackState, you need to add a new remote write endpoint to your Prometheus configuration. This can be done by updating the remote_write section in your prometheus.yml file. There are 2 variants that you can use for the authentication to the endpoint, either through a header or through basic authentication.

Header authentication

You can authenticate using the API Key by adding the sts-api-key header to the prometheus remote_write requests. The following example shows how to configure this:

remote_write:
- url: https://<tenant>.app.stackstate.io/receiver/prometheus/api/v1/write
  headers:
    sts-api-key: <API Key>

Basic authentication

You can authenticate using basic authentication by adding the basic_auth section to the prometheus remote_write requests. Because the requests to this API are made from a headless service, the username field is set to the static value apikey. The following example shows how to configure this:

remote_write:
- url: https://<tenant>.app.stackstate.io/receiver/prometheus/api/v1/write
  basic_auth:
    username: apikey
    password: <API Key>

Finishing up

After the configuration changes have been applied to the Prometheus configuration file, Prometheus needs to be restarted. After the restart, Prometheus will start sending metrics to StackState. The metrics will be automatically correlated with the rest of your environment and will be visible in the StackState UI.

📈
Prometheus Remote Write
StackPacks
API Key