Horizontal Node Autoscaler

MetaKube Kubernetes clusters support horizontal node autoscaling out of the box. This document describes the functionality and features related to autoscaling. If you want to install and test the horizontal node autoscaler in your own cluster, we recommend also reading the node autoscaler tutorial.

Activating the Node Autoscaler

The horizontal node autoscaler is automatically installed into the kube-system namespace of every MetaKube cluster. It can be activated by setting the minimum and maximum number of replicas for a node deployment. One way to achieve this is the MetaKube dashboard:

The node autoscaler can also be activated via the MetaKube API. It is further possible to add the labels for the node autoscaler manually to the machinedeployment resource using kubectl:

  ...
  annotations:
    cluster.k8s.io/cluster-api-autoscaler-node-group-min-size: "1"
    cluster.k8s.io/cluster-api-autoscaler-node-group-max-size: "15"
  ...

Note: On clusters using Kubernetes 1.17 and below the labels for the autoscaler are named cluster-autoscaler/max-size and cluster-autoscaler/min-size respectively.

For more information please have a look at the documentation for the cluster-autoscaler.

Scaling up

The horizontal node autoscaler will try to add nodes to a node deployment when the minimum and maximum number of replicas are specified (see annotations above) and the current number of replicas is lower than the maximum number of replicas and pods can not be scheduled due to limited resources.

When all conditions are met, the autoscaler calculates the required nodes to schedule all pending pods and will create the respective number of nodes in a single scaling step.

Note: It can happen that pods are temporarily in a pending state, even when the number of nodes is technically sufficient to schedule all pods. In such situations it is possible that the autoscaler adds one extra node temporarily.

Scaling down

The autoscaler will reduce the number of nodes in a machine deployment when the minimum and maximum number of replicas are specified and the current number of replicas is higher than the specified minimum number of replicas and the last scale-up has happened longer than 10 minutes ago and all pods can be scheduled on fewer nodes.

When all the above conditions are met, the autoscaler will calculate the number of spare nodes. Those nodes will be drained and removed from the node deployment. The autoscaler will not drain and remove nodes running certain pods from the kube-system namespace or pods using local storage. Therefore, it can happen, that the autoscaler will not scale down to the minimum number of replicas, even in cases where pods could theoretically be scheduled on fewer nodes.