💠AWS
StackState core integration
Last updated
StackState core integration
Last updated
This page describes StackState version 4.3.
The StackState 4.3 version range is End of Life (EOL) and no longer supported. We encourage customers still running the 4.3 version range to upgrade to a more recent release.
Amazon Web Services (AWS) is a major cloud provider. This StackPack enables in-depth monitoring of AWS services.
Three AWS Lambdas collect topology data from AWS and push this to StackState:
stackstate-topo-cron scans AWS resources every hour using the AWS APIs and pushes this to StackState.
stackstate-topo-cwevents listens to CloudWatch events, transforms the events and publishes them to Kinesis.
stackstate-topo-publisher publishes retrieved topology data from a Kinesis stream to StackState.
StackState translates incoming data into topology components and relations.
The StackState CloudWatch plugin pulls available telemetry data per resource at a configured interval from AWS.
StackState maps retrieved telemetry (metrics) onto the associated AWS components and relations.
To set up the StackState AWS integration, you need to have:
AWS CLI version 2.0.4 or later installed and configured.
An AWS user with the required access to retrieve Cloudwatch metrics:
cloudwatch:GetMetricData
cloudwatch:ListMetrics
A policy file to create a user with the correct rights can be downloaded from the the StackState UI screen StackPacks > Integrations > AWS.
An AWS user with the required access rights to install StackState monitoring in your account. See AWS IAM policies, below.
If your StackState instance is behind a proxy, you need to configure the proxy URL and port for the AWS authorization to work. You can configure a proxy URL environment variable or JVM system property.
Environment variable HTTP_PROXY and/or HTTPS_PROXY
Pass following properties when starting StackState instance -Dhttp.proxyHost -Dhttp.proxyPort and/or -Dhttps.proxyHost -Dhttps.proxyPort
Install the AWS StackPack from the StackState UI StackPacks > Integrations screen. You will need to provide the following parameters:
AWS instance name - the user-defined name of the AWS account shown in configurations such as views.
AWS Access Key id - the access key for the user for retrieving Cloudwatch metrics.
AWS Secret Access Key - the secret key for the user for retrieving Cloudwatch metrics.
AWS Role ARN - Optional: IAM role ARN - the ARN of the IAM role to be used
The StackState AWS Cloudformation stacks are deployed on your AWS account to enable topology monitoring. There are two options for StackState monitoring:
Full install - all changes to AWS resources will be picked up and pushed to StackState.
Minimal install - changes will be picked up only at a configured interval.
A full installation will install the following CloudFormation Stacks:
stackstate-topo-cron
stackstate-topo-kinesis
stackstate-topo-cloudtrail
stackstate-topo-cwevents
stackstate-topo-publisher
Follow the steps below to complete a full install:
Download the manual installation zip file and extract it. This is included in the AWS StackPack and can be accessed at the link provided in StackState after you install the AWS StackPack.
Make sure the AWS CLI is configured with the proper account and has the default region set to the region that should be monitored by StackState.
For further information on authentication via the AWS CLI, see using an IAM role in the AWS CLI (docs.aws.amazon.com).
From the command line, run the command:
If you wish to use a specific AWS profile or an IAM role during installation, run either of these two commands:
These environment variables have the same names used by the AWS_CLI utility and will be overridden with options:
--profile
--role-arn
--session-name
--external-id
The minimal installation is useful when less permissions are available. This installs only the stackstate-topo-cron Cloudformation stack, which means StackState's topology will only get a full topology update every hour. Updates between the hour are not sent to StackState.
Follow the steps below to complete a minimal install:
Download the manual installation zip file and extract it. This is included in the AWS StackPack and can be accessed at the link provided in StackState after you install the AWS StackPack.
Make sure the AWS CLI is configured with the proper account and has the default region set to the region that should be monitored by StackState.
For further information on authentication via the AWS CLI, see using an IAM role in the AWS CLI (docs.aws.amazon.com).
From the command line, run the command:
You can also optionally specify the following:
--topo-cron-bucket - a custom S3 bucket to be used during deployment.
--topo-cron-role - a custom AWS IAM role. Note that the role must have an attached policy like that specified in the file sts-topo-cron-policy.json included in the manual install zip file.
If you wish to use a specific AWS profile or an IAM role during installation, run either of these two commands:
These environment variables have the same names used by the AWS_CLI utility and will be overridden with options:
--profile
--role-arn
--session-name
--external-id
The following AWS policies can be downloaded during the installation of the AWS StackPack in your StackState instance:
Full install - StackStateIntegrationPolicyInstall.json
Minimal install - StackStateIntegrationPolicyTopoCronInstall.json
Minimal set of policies - StackStateIntegrationPolicyTopoCronMinimal.json S3 bucket and role are provided by user.
Uninstall a full install - StackStateIntegrationPolicyUninstall.json
Uninstall a minimal install - StackStateIntegrationPolicyTopoCronUninstall.json
The default read timeout for AWS is set to 30 seconds. You can specify custom read timeout with the AWS_CLI_READ_TIMEOUT environment variable.
The AWS integration does not retrieve any Events data.
Metrics data is pulled at a configured interval directly from AWS by the StackState CloudWatch plugin. Retrieved metrics are mapped onto the associated topology component.
Each AWS integration retrieves topology data for resources associated with the associated AWS access key.
Components
The following AWS service data is available in StackState as components:
API Gateway Resource | API Gateway Stage | API Getaway Method |
AutoScaling Group | CloudFormation Stack | DynamoDB Stream |
DynamoDB Table | EC2 Instance | ECS Cluster |
ECS Service | ECS Task | Firehose Delivery Stream |
Kinesis Stream | Lambda | Lambda Alias |
Load Balancer Classic | Load Balancer V2 | RDS Instance |
Redshift Cluster | Route53 Domain | Route53 Hosted Zone |
S3 bucket | Security Group | SNS Topic |
SQS Queue | Subnet | Target Group |
Target Group Instance | VPC | VPN Gateway |
Relations
The following relations between components are retrieved:
API Gateway Method → (Service) Integration Resource (varies)
API Gateway Resource → API Gateways Method
API Gateway Stage → API Gateway Resource
AutoScaling Group → EC2 Instance, Load Balancer Classic
CloudFormation Stack → Any Resource (many supported), CloudFormation Stack Parent
DynamoDB Table → DynamoDB Stream
EC2 Instance → Security Group, Subnet, VPC
ECS Cluster → EC2 Instance, ECS Task (when no group service)
ECS Service → ECS Cluster, ECS Task, Route53 Hosted Zone, Target Group
ECS Task → ECS Cluster
Firehose Delivery Stream → Kinesis Source, S3 Bucket Destination(s)
Lambda → Event Source Mapping, Security Group, VPC
Lambda Alias → VPC
Load Balancer Classic → EC2 Instance, VPC
Load Balancer V2 → Security Group, Target Group, VPC
RDS Cluster → RDS Instance
RDS Instance → Security Group, VPC
Redshift Cluster → VPC
S3 Bucket → Lambda (notification configuration of the bucket)
Security Group → VPC
SNS Topic → Subscription
Subnet → VPC
Target Group → AutoScaling Group, EC2 Instance, VPC
VPN Gateway → VPC
The AWS integration does not retrieve any Traces data.
The StackState AWS integration installs the following AWS lambdas:
| Lambda | Description |
|---|---|
| Scans the initial topology based on an interval schedule and pushes to StackState. |
| Listens to CloudWatch events, transforms the events and publishes them to Kinesis. Full install only. |
| Pushes topology from a Kinesis stream to StackState. Full install only. |
The AWS lightweight agent uses Amazon resources (Lambda and Kinesis) for which Amazon will charge a minimal fee. Amazon also charges a fee for the use of CloudWatch metrics. Metrics are only retrieved when viewed or when a check is configured on a CloudWatch metric.
When the AWS integration is enabled, three views will be created in StackState for each instance of the StackPack.
AWS - [instance_name] - All - includes all resources retrieved from AWS by the StackPack instance.
AWS - [instance_name] - Infrastructure - includes only Networking, Storage and Machines resources retrieved from AWS by the StackPack instance.
AWS - [instance_name] - Serverless - includes only S3 buckets, lambdas and application load balancers retrieved from AWS by the StackPack instance.
Components retrieved from AWS will have an additional action available in the component context menu and component details pane on the right side of the screen. This provides a deep link through to the relevant AWS console at the correct point.
For example, in the StackState Topology Perspective:
Components of type aws-subnet have the action Go to Subnet console, which links directly to this component in the AWS Subnet console.
Components of type ec2-instance have the action Go to EC2 console, which links directly to this component in the EC2 console.
The AWS StackPack converts tags in AWS to labels in StackState. In addition, the following special tags are supported:
| Tag | Description |
|---|---|
| Adds the specified value as an identifier to the StackState component |
| Places the StackState component in the environment specified |
Troubleshooting steps can be found in the StackState support Knowledge base guide to troubleshoot the StackState AWS StackPack.
To uninstall the StackState AWS StackPack, click the Uninstall button from the StackState UI StackPacks > Integrations > AWS screen. This will remove all AWS specific configuration in StackState.
Once the AWS StackPack has been uninstalled, you will need to manually uninstall the StackState AWS Cloudformation stacks from the AWS account being monitored. To execute the manual uninstall follow these steps:
Download the manual installation zip file and extract it. This is included in the AWS StackPack and can be accessed at the link provided in StackState after you install the AWS StackPack.
Make sure the AWS CLI is configured with the proper account and has the default region set to the region that should be monitored by StackState.
For further information on authentication via the AWS CLI, see using an IAM role in the AWS CLI (docs.aws.amazon.com).
From the command line, run the below command to de-provision all resources related to the StackPack instance:
If you wish to use a specific AWS profile or an IAM role during uninstallation, run either of these two commands:
These environment variables have the same names used by the AWS_CLI utility and will be overridden with options:
--profile
--role-arn
--session-name
--external-id
AWS StackPack v5.2.2 (2021-04-09)
Bugfix: Fixed upgrading AWS StackPack when you upgrade StackState from 4.2.x to 4.3.x
AWS StackPack v5.2.1 (2021-04-02)
Bugfix: Updated the manual_trigger.sh to verify if the lambda exists, Trigger and monitor the lambda outcome and if it failed then the error will be displayed with a possible solution to allow the user to fix the problem and 'press any key' to retry
Improvement: Update documentation.
Improvement: Enable auto grouping on generated views.
Improvement: Common bumped from 2.2.3 to 2.5.1
Improvement: StackState min version bumped to 4.3.0
AWS StackPack v5.1.3 (2021-03-01)
Features: Added support for security groups
Features: Give RDS databases an identifier so that they can be referred to by other StackPacks
Bugfix: Fixed the EC2 instances being stranded. Relationship restored to either a Subnet or VPC fallback.
Bugfix: Added a delete event for the removal of a ELB Target Group
Bugfix: Fixed the ELB, ELB Target Group and ELB Target Group Instance not mapping on cloud watch events
Bugfix: Restored the EC2 identifier to the i- mapping instead of urn:aws/i-
Bugfix: Updated the ELB Instance identifier to map to urn:aws/target-group-instance/i- instead of i-
AWS StackPack v5.1.2 (2021-02-01)
Bugfix: Merged the elb_v2_target_group_instance and ec2-instance. The elb_v2_target_group_instance will no longer display as a generic aws resource but rather show up as the merged EC2 instance.
AWS StackPack v5.1.1 (2021-01-22)
Features: New component type elb_v2_gateway and its metrics added.
Features: Metrics added for type elb_v2_network.
Improvement: Metrics fixed for different target group based on load balancer type
Improvement: Restricted the full, minimal and uninstall policies resources from all (*) to only certain resources. This will restrict the IAM user to only access resources created by StackState or Specified by the user
Improvement: Lambda version is send in the payload
Improvement: Better error logs
Improvement: Cloudformation memory cache improvements
AWS StackPack v5.1.0 (2021-01-04)
Features: Added support for EC2 Nitro based metrics EBSWriteBytes and EBSReadBytes.
Improvement: Check if TargetGrouArn exist in the loadBalancer for relation.
AWS StackPack v5.0.2 (2020-11)
Bugfix: Fixed and improved the parsing of custom StackState identifier tags making it more flexible and ignoring case sensitivity.
Bugfix: Fixed the merging between ECS service components with Traefik trace service components.
Bugfix: Fixed profile selection doesn't work when you run ./install --profile.
AWS StackPack v5.0.1 (2020-08-18)
Feature: Introduced the Release notes pop up for customer.
AWS StackPack v5.0.0 (2020-08-13)
Bugfix: Fixed the upgradation of other StackPacks due to AWS old layers using common.
