Stackstate-HAProxy Integration

Overview

Capture HAProxy activity in Stackstate to:

  • Visualize HAProxy load-balancing performance.
  • Know when a server goes down.
  • Correlate the performance of HAProxy with the rest of your applications.

Installation

First make sure that stats are enabled on your HAProxy configuration:

The most common method to access HAProxy metrics is to enable the stats page, which you can then view with any web browser. This page is not enabled out of the box, and requires modification of HAProxy’s configuration to get it up and running.

Configuration

To enable the HAProxy stats page, add the following to the bottom of the file /etc/haproxy/haproxy.cfg (adding your own username and password to the final line):

listen stats :9000  # Listen on localhost:9000
mode http
stats enable  # Enable stats page
stats hide-version  # Hide HAProxy version
stats realm Haproxy\ Statistics  # Title text for popup window
stats uri /haproxy_stats  # Stats URI
stats auth Username:Password  # Authentication credentials

This sets up a listener on port 9000 in HTTP mode with statistics enabled.

Next you’ll need to restart HAProxy, which can interrupt client sessions and cause downtime. If you want to be very careful about how you restart HAProxy, check out Yelp’s research on the least disruptive means by which you can reload HAProxy’s configuration.

If you’re comfortable with session interruption, you can restart HAProxy with

sudo service haproxy restart.

After restarting HAProxy with your modified configuration, you can access a stats page like the one below after authenticating via the URL: http://:9000/haproxy_stats

HAProxyStats

If you see a page like the page above the stats are enabled.

Configuration

  1. Configure the Agent to connect to HAProxy. Edit conf.d/haproxy.yaml:

    init_config:
    
    instances:
        -   username: username
            password: password
            url: https://localhost/admin?stats
    
  2. Restart the Agent

Validation

  1. Execute the info command and verify that the integration check has passed. The output of the command should contain a section similar to the following:

    Checks
    ======
    
    [...]
    
    haproxy
    -------
        - instance #0 [OK]
        - Collected 8 metrics & 0 events
    

For more details about configuring this integration refer to the following file(s) on GitHub:

Metrics

haproxy.backend_hosts
(gauge)
Number of backend hosts.
shown as host
haproxy.backend.bytes.in_rate
(gauge)
Rate of bytes in on backend hosts.
shown as byte/second
haproxy.backend.bytes.out_rate
(gauge)
Rate of bytes out on backend hosts.
shown as byte/second
haproxy.backend.connect.time
(gauge)
Average connect time over the last 1024 requests.
shown as millisecond
haproxy.backend.denied.req_rate
(gauge)
Number of requests denied due to security concerns.
shown as request/second
haproxy.backend.denied.resp_rate
(gauge)
Number of responses denied due to security concerns.
shown as response/second
haproxy.backend.errors.con_rate
(gauge)
Rate of requests that encountered an error trying to connect to a backend server.
shown as error/second
haproxy.backend.errors.resp_rate
(gauge)
Rate of responses aborted due to error.
shown as error/second
haproxy.backend.queue.current
(gauge)
Number of requests without an assigned backend.
shown as request
haproxy.backend.queue.time
(gauge)
Average queue time over the last 1024 requests.
shown as millisecond
haproxy.backend.response.1xx
(gauge)
Backend HTTP responses with 1xx code.
shown as response
haproxy.backend.response.2xx
(gauge)
Backend HTTP responses with 2xx code.
shown as response
haproxy.backend.response.3xx
(gauge)
Backend HTTP responses with 3xx code.
shown as response
haproxy.backend.response.4xx
(gauge)
Backend HTTP responses with 4xx code.
shown as response
haproxy.backend.response.5xx
(gauge)
Backend HTTP responses with 5xx code.
shown as response
haproxy.backend.response.other
(gauge)
Backend HTTP responses with other code (protocol error).
shown as response
haproxy.backend.response.time
(gauge)
Average response time over the last 1024 requests (0 for TCP).
shown as millisecond
haproxy.backend.session.current
(gauge)
Number of active backend sessions.
shown as connection
haproxy.backend.session.limit
(gauge)
Configured backend session limit.
shown as connection
haproxy.backend.session.pct
(gauge)
Percentage of sessions in use (backend.session.current/backend.session.limit * 100).
shown as percent
haproxy.backend.session.rate
(gauge)
Number of backend sessions created per second.
shown as connection/second
haproxy.backend.session.time
(gauge)
Average total session time over the last 1024 requests.
shown as millisecond
haproxy.backend.warnings.redis_rate
(gauge)
Number of times a connection to a server was retried.
shown as error/second
haproxy.backend.warnings.retr_rate
(gauge)
Number of times a request was redispatched to another server.
shown as error/second
haproxy.count_per_status
(gauge)
Number of hosts by status (UP/DOWN/NOLB/MAINT).
shown as host
haproxy.frontend.bytes.in_rate
(gauge)
Rate of bytes in on frontend hosts.
shown as byte/second
haproxy.frontend.bytes.out_rate
(gauge)
Rate of bytes out on frontend hosts.
shown as byte/second
haproxy.frontend.denied.req_rate
(gauge)
Number of requests denied due to security concerns.
shown as request/second
haproxy.frontend.denied.resp_rate
(gauge)
Number of responses denied due to security concerns.
shown as response/second
haproxy.frontend.errors.req_rate
(gauge)
Rate of request errors.
shown as error/second
haproxy.frontend.requests.rate
(gauge)
Number of HTTP requests per second.
shown as request/second
haproxy.frontend.response.1xx
(gauge)
Frontend HTTP responses with 1xx code.
shown as response
haproxy.frontend.response.2xx
(gauge)
Frontend HTTP responses with 2xx code.
shown as response
haproxy.frontend.response.3xx
(gauge)
Frontend HTTP responses with 3xx code.
shown as response
haproxy.frontend.response.4xx
(gauge)
Frontend HTTP responses with 4xx code.
shown as response
haproxy.frontend.response.5xx
(gauge)
Frontend HTTP responses with 5xx code.
shown as response
haproxy.frontend.response.other
(gauge)
Frontend HTTP responses with other code (protocol error).
shown as response
haproxy.frontend.session.current
(gauge)
Number of active frontend sessions.
shown as connection
haproxy.frontend.session.limit
(gauge)
Configured backend session limit.
shown as connection
haproxy.frontend.session.pct
(gauge)
Percentage of sessions in use (frontend.session.current/frontend.session.limit * 100).
shown as percent
haproxy.frontend.session.rate
(gauge)
Number of frontend sessions created per second.
shown as connection/second