Introduction

Kubernetes is about scaling but it doesn’t mean that we have auto scaling out-of-the-box – we must activate some additional components and configure them. In this article I want to show you working example of simple service scaling on Kubernetes using Horizontal Pod Autoscaler. After reaching maximum cluster capacity we will automatically add more workers to our cluster using Cluster Autoscaler. Everything will run on Amazon EKS – managed Kubernetes on AWS.

As always full source code can be found on my github: https://github.com/jakubbujny/article-scale-containers-and-eks

Simple deployment creating load

To test auto scaling we need some simple service which will create big load for us so scaling can be triggered. For that task we are going to use following Python code:

import flask
import uuid
import hashlib
app = flask.Flask(__name__)
@app.route("/")
def hello():
    for i in range(0,800000):
      hashlib.sha224(uuid.uuid4().hex.upper()[0:6].encode()).hexdigest()
    return "Done"

app.run(host="0.0.0.0", threaded=True)

That code is really simple – just create web endpoint using Flask framework – GET request on “/” will cause long loop which calculate a lot of SHA hashes from random UUID what should take about 5–10 seconds and consume a lot of CPU during that time.

To avoid building own Docker image as we want to avoid creating docker registry (to simplify example) we can use simple trick by taking docker image jazzdd/alpine-flask:python3 which is available on dockerhub and contains Python/Flask installed. So we can create our python file in “command” section and run it, see full yaml below:

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: default
  name: microservice
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: microservice
    spec:
      containers:
      - name: microservice
        image: jazzdd/alpine-flask:python3
        command: ["sh"]
        args: ["-c", "printf \"import flask\\nimport uuid\\nimport hashlib\\napp = flask.Flask(__name__)\\n@app.route(\\\"/\\\")\\ndef hello():\\n    for i in range(0,800000):\\n     hashlib.sha224(uuid.uuid4().hex.upper()[0:6].encode()).hexdigest()\\n    return \\\"Done\\\"\\napp.run(host=\\\"0.0.0.0\\\", threaded=True)\" > script.py && python3 script.py"]
        ports:
        - name: http-port
          containerPort: 5000
        resources:
          requests:
            cpu: 200m

Important thing there is resources request block which says that on 1 CPU core machine (which we are going to use in this article) we can create 5 microservice PODs (200m x 5 = 1000m = 1CPU) and reaching that number means end of capacity of particular node. Reaching cluster capacity will be trigger for Cluster Autoscaler.

Horizontal Pod Autoscaler

Horizontal scaling in Kubernetes world means adding more pods in particular deployment. To achieve that Horizontal Pod Autoscaler can be used but we need to note one important thing: In the newest Kubernetes version metrics-server need to be installed to use HPA – Heapster is deprecated and shouldn’t be used anymore. 

To test that on minikube you need just to type:

minikube addons enable metrics-server

To deploy metrics-server on EKS you need to clone following repository: https://github.com/kubernetes-incubator/metrics-server and then issue command:

kubectl apply -f metrics-server/deploy/1.8+/

To activate HPA for our microservice we need to apply following yaml file:

---
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  namespace: default
  name: microservice
spec:
  scaleTargetRef:
    apiVersion: apps/v1beta1
    kind: Deployment
    name: microservice
  minReplicas: 1
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        targetAverageUtilization: 50

targetAverageUtilization: 50 means that Kubernetes will try to maintain half of usage of CPU requested by our microservice (50% * 200m = 100m) on particular POD. E.g. when we have single POD which is having 200m of CPU, Kubernetes will create new POD so 200m can be divided on 2 PODs (100m and 100m).

AWS EKS and Cluster Autoscaler

Disclaimer – why use Cluster Autoscaler instead of ASG scaling trigger based on CPU?

From Cluster Autoscaler FAQ:

“Cluster Autoscaler makes sure that all pods in the cluster have a place to run, no matter if there is any CPU load or not. Moreover, it tries to ensure that there are no unneeded nodes in the cluster.

CPU-usage-based (or any metric-based) cluster/node group autoscalers don’t care about pods when scaling up and down. As a result, they may add a node that will not have any pods, or remove a node that has some system-critical pods on it, like kube-dns. Usage of these autoscalers with Kubernetes is discouraged.”

For EKS deployment we are going to use modified EKS version from my previous article.

Cluster Autoscaler is component which will be installed on EKS cluster. It will look in Kubernetes API and make request to AWS API to scale worker nodes’s ASG. It means that node on which Cluster Autoscaler will reside need proper IAM policy which will allow container from that node to make operations on ASG.

 resource "aws_iam_role_policy" "for-autoscaler" {
  name = "for-autoscaler"
  policy = <<POLICY
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "autoscaling:DescribeAutoScalingGroups",
                "autoscaling:DescribeAutoScalingInstances",
                "autoscaling:SetDesiredCapacity",
                "autoscaling:DescribeTags",
                "autoscaling:TerminateInstanceInAutoScalingGroup"
            ],
            "Resource": "*"
        }
    ]
}
POLICY
  role = "${aws_iam_role.eks-node.name}"
}

That policy should be probably limited in Resource section but we will leave * to simplify example.

We put some additional tags to ASG to use them in Cluster Autoscaler

  tag {
    key = "k8s.io/cluster-autoscaler/enabled"
    value = "whatever"
    propagate_at_launch = false
  }

  tag {
    key                 = "kubernetes.io/cluster/eks"
    value               = "owned"
    propagate_at_launch = true
  }

We must also setup security groups to allow 443 port communication from cluster control plane to nodes as mentioned in this issue: https://github.com/kubernetes-incubator/metrics-server/issues/45

For Cluster Autoscaler we will modify a little example deployment from here: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/examples/cluster-autoscaler-autodiscover.yaml

We need to modify tags which Cluster Autoscaler will use to discover ASG which will be scaled:

- --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,kubernetes.io/cluster/eks

Add env with region in which we are operating:

 env:
   - name: "AWS_REGION"
     value: eu-west-1

Change certificate to use what is required for EKS:

 volumeMounts:
   - name: ssl-certs
     mountPath: /etc/ssl/certs/ca-bundle.crt
     readOnly: true

Cluster Autoscaler is ready to use and will scale up or down worker nodes using ASG scaling between 1 and 10 instances.

Testing

Last step is to create load balancer attached to microservice and test auto scaling by making some requests to create load.

apiVersion: v1
kind: Service
metadata:
  name: microservice
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
  labels:
    app: microservice
spec:
  ports:
  - port: 80
    targetPort: http-port
  selector:
    app: microservice
  type: LoadBalancer

You can just simply try to open load balancer endpoint on root in web browser and hit f5 a few times to generate load or use script like:

while true; do sleep 2; timeout 1 curl http://<elb_id>.elb.eu-west-1.amazonaws.com/; done

 

After that you should see in Kubernetes that HPA scaled your containers up and reached maximum node capacity. After while Cluster Autoscaler should scale AWS ASG and add new worker node so HPA can complete PODs scaling.