You can support us by downloading this article as PDF from the Link below. Download the guide as PDF

One of the many advantages that Kubernetes has to offer is auto-scaling. This enables the automatic provisioning of resources based on traffic/demand. As the traffic demand increases more resources are provisioned and vice versa. Kubernetes offers three types of scaling:

  • Vertical Pod autoscaler: Scaling that increases/decreases the CPU or memory allocated to Pods.
  • Horizontal Pod autoscaler: Increase/Decreases the number of pods per service in response to traffic/demand.
  • Cluster autoscaler: Scales your cluster nodes based on the number of pending pods.

Running Kubernetes on AWS (EKS) offers the option to scale your cluster nodes based on metrics from your cluster resource utilization. This article guides the reader on configuring auto-scaling based on cluster metrics. We will use the Cluster Node Memory utilization metric to scale our nodes.

N/B: This is only possible on an EKS cluster with a managed node group (Provisioned Worker Nodes) and not an EKS Fargate cluster. With Fargate there is no need to provision instances for your cluster because it provides compute capacity for your containers. It also automatically scales for you, resources based on traffic demands.

Requirements/Prerequisites

  • An AWS Account.
  • Created a User on the account with Permissions to provision resources on the account and Access to CloudWatch.
  • Provisioned an AWS EKS Cluster with a managed Node Group.
  • Enabled Container Insights for your EKS Cluster by deploying CloudWatch Agent on your cluster.

Part 1: Create the AWS EKS SNS Topic & Subscriptions

We will create two SNS Topics, The Scale Up and Scale Down Topics and add email subscriptions to the topics. The below CloudFormation Templates can be used for this.

The Scale Down Topic and Subscriptions Template.

AWSTemplateFormatVersion: "2010-09-09"
Description: "Template to create SNS Topic and Subscriptions"

Parameters:
    Name:
        Type: String
        Description: The Topic Name
        Default: EKS_CLUSTER_MEMORY_LESS_THAN_50

Resources:
    Email1Notification:
        Type: AWS::SNS::Subscription
        Properties:
           Endpoint: [email protected]
           Protocol: email
           TopicArn: !Ref ScaleDownTopic

    Email2Notification:
        Type: AWS::SNS::Subscription
        Properties:
           Endpoint: [email protected]
           Protocol: email
           TopicArn: !Ref ScaleDownTopic

    ScaleDownTopic:   
        Type: AWS::SNS::Topic
        Properties: 
           Tags: 
              - Key: Name
                Value: !Ref Name
              - Key: createdBy
                Value: Maureen Barasa
              - Key: Project
                Value: test-blog
           TopicName: !Ref Name

Outputs:    
    ScaleDown:
        Description: The scale down topic
        Value: !Ref ScaleDownTopic
        Export:
          Name: "EKSScaleDown"
          Value: !Ref ScaleDownTopic

N/B: The user should customize the template. The names of Resources, Email endpoints and Tags should match the user’s specific requirements.

The Scale Up Topic and Subscriptions Template

AWSTemplateFormatVersion: "2010-09-09"
Description: "Template to create SNS Topic and Subscriptions"
Parameters:
    Name:
        Type: String
        Description: The Topic Name
        Default: EKS_CLUSTER_MEMORY_MORE_THAN_70

Resources:
    Email1Notification:
        Type: AWS::SNS::Subscription
        Properties:
           Endpoint: <your-email-address>
           Protocol: email
           TopicArn: !Ref ScaleUpTopic

    Email2Notification:
        Type: AWS::SNS::Subscription
        Properties:
           Endpoint: <your-email-2-addresS>
           Protocol: email
           TopicArn: !Ref ScaleUpTopic

    ScaleUpTopic:   
        Type: AWS::SNS::Topic
        Properties: 
           Tags: 
              - Key: Name
                Value: !Ref Name
              - Key: createdBy
                Value: <your-name>
              - Key: Project
                Value: <project-name>
           TopicName: !Ref Name

Outputs:
    ScaleUp:
        Description: The scale up topic
        Value: !Ref ScaleUpTopic
        Export:
          Name: "EKSScaleUp"
          Value: !Ref ScaleUpTopic

N/B: The user should customize the template. The names of Resources, Email endpoints and Tags should match the user’s specific requirements.

Under Outputs we have allowed our values to be exported so we can use them in the template that will create the CloudWatch Alarms.

Once the SNS Topics and Subscriptions are created, go to the given email endpoints and confirm the subscription. We are now ready to create the Scaling Policies and CloudWatch Alarms.

Part 2: Create Scaling Policies and Attach to your AWS EKS NodeGroup ASG

Next, we will create two policies and attach them to the EKS nodegroup autoscaling group. Use the below Cloud formation Template.

AWSTemplateFormatVersion: "2010-09-09"
Description: "create a scaling policy and apply it to EKS ASG Group"

Resources:
  ScaleDownPolicy: 
    Type: AWS::AutoScaling::ScalingPolicy
    Properties: 
      AdjustmentType: "ChangeInCapacity"
      PolicyType: "SimpleScaling"
      Cooldown: "300"
      AutoScalingGroupName: "eks-ASG-GroupName"
      ScalingAdjustment: -2

  ScaleUpPolicy: 
    Type: AWS::AutoScaling::ScalingPolicy
    Properties: 
      AdjustmentType: "ChangeInCapacity"
      PolicyType: "SimpleScaling"
      Cooldown: "300"
      AutoScalingGroupName: "eks-ASG-GroupName"
      ScalingAdjustment: 2

Outputs:
    ScaleUp:
        Description: The scale up policy
        Value: !Ref ScaleUpPolicy
        Export:
          Name: "EKSScaleUpPolicy"
          Value: !Ref ScaleUpPolicy

    ScaleDown:
        Description: The scale down policy
        Value: !Ref ScaleDownPolicy
        Export:
          Name: "EKSScaleDownPolicy"
          Value: !Ref ScaleDownPolicy

Part 3: Create the AWS EKS CloudWatch Alarms

Finally, we will create the CloudWatch Alarms. Here, we will use values created from the above templates. i.e. The SNS Topics and the Scaling Policies.

On the CloudWatch console, select alarms. Then click create alarms.

Create Alarms
Create Alarm

Under Select Metric, choose “Container Insights” Namespace then select by Cluster name. Under your EKS Cluster name, choose node_memory_utilization metric and click select metric. Next, specify Metric Conditions. For Period one can choose a period from a minute to 1 day. It is the time taken to analyze the metric or expression for creating individual data points for an alarm.

Specify Metric Conditions
Specify Metric Conditions

Next, we define the conditions for the alarm. For the first case, we will create an Alarm, where node memory utilization is greater than 70. Click next.

Define Alarm Conditions
Define Alarm Conditions

Then we configure actions to take when the alarm is triggered. For, our case our first action is to send a notification to the respective SNS topic created in part 1 above. For example, when the memory utilization average is greater than 70, it should send a notification to EKS_CLUSTER_MEMORY_MORE_THAN_70 SNS topic.

Configure actions 1
Configure Alarm Actions

The 2nd action to take will be to scale up based on the scaling policies we defined in part 2 of this article. Ensure to select the EKS cluster node, scaling group. For the high memory utilization alarm, choose the scale-up policy created and click next.

Select Scaling Action
Select Scaling Action

Finally, you create a Name and Description for the Alarms as below and click next. After carefully evaluating your configurations, you can now click, create alarm.

Create Alarm Name and Description
Create Alarm Name and Description

Equally important is to create the alarm for low memory utilization, follow the same steps as above. This time ensure that for the alarm, under conditions, we should select the relevant SNS topic and scaling policies. For example, use the EKS_CLUSTER_MEMORY_LESS_THAN_50 SNS topic and the scale-down policy for scaling actions.

You have now successfully configured EKS cluster auto-scaling using CloudWatch Alarms and SNS topics.

Cloudwatch Alarms
The Created CloudWatch Alarms

Other Kubernetes Guides:

Happy Building!!!

As an appreciation for the content we put out,
we would be thrilled if you support us!


As we continue to grow, we would wish to reach and impact more people who visit and take advantage of the guides we have on our blog. This is a big task for us and we are so far extremely grateful for the kind people who have shown amazing support for our work over the time we have been online.

Thank You for your support as we work to give you the best of guides and articles. Click below to buy us a coffee.

LEAVE A REPLY

Please enter your comment!
Please enter your name here