Create a Load Balancer

The Kubernetes Manifest

Kubernetes defines a Service resource with type LoadBalancer, that you can use to create a managed load balancer in SysEleven Stack. To create a Kubernetes managed load balancer, use a manifest like the following:

kind: Service
apiVersion: v1
metadata:
  name: $LOADBALANCER_NAME
spec:
  selector:
    $APPLICATION_LABEL: $APPLICATION_NAME
  ports:
    - protocol: TCP
      port: 80
      name: http
    - protocol: TCP
      port: 443
      name: https
  type: LoadBalancer

Be sure to match the label and value of the selector to your application and preferably choose a speaking name for the load balancer. The created load balancer might look like this:

$ kubectl get svc
NAME           TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)                      AGE
loadbalancer   LoadBalancer   10.10.10.202   195.192.128.46   80:31228/TCP,443:30279/TCP   11s

This load balancer now exposes port 80 to the outside world and maps it to port 31228 on all cluster nodes, and it exposes port 443 to the outside world and maps it to ports 30279 on all cluster nodes. This means that a loadbalancer service is also a NodePort service (i.e. a service that exposes pods on specific "NodePorts" on all nodes).

Keeping load balancer floating IPs on OpenStack

When a service of type LoadBalancer is created without further configuration it gets an ephemeral IP address from the OpenStack network pool of the cluster (usually ext-net). When the load balancer is deleted the floating IP is also released. So, when a new service is created it could get the same IP but in most cases it just gets another random IP. When working with Kubernetes it can be desirable to have a more fine grained control of the used IP addresses.

This has been tested with Kubernetes 1.17 and 1.18. Please keep that in mind if you run an older version (and consider upgrading). The openstack external cloud controller is also required for this to work. You can check this requirement by running the following command in your cluster:

kubectl get csidrivers.storage.k8s.io
NAME                       ATTACHREQUIRED   PODINFOONMOUNT   MODES        AGE
cinder.csi.openstack.org   true             false            Persistent   96d

In the output you should see the cinder csi driver.

Two options in the LoadBalancer resource come into play: The annotation loadbalancer.openstack.org/keep-floatingip and .spec.loadBalancerIP.

kind: Service
apiVersion: v1
metadata:
  name: ...
  annotations:
    loadbalancer.openstack.org/keep-floatingip: "true"
spec:
  loadBalancerIP: "xxx.xxx.xxx.xxx"
  ...
  • Both can be set during creation as well as for an existing service.
  • If a service with the annotation is deleted the cloud-controller will not release the floating IP in OpenStack.
  • loadBalancerIP comes with a few remarks:
    • For an existing service it should be the already assigned IP.
    • The IP needs to be already allocated (but unassigned) by your tenant, either manually or by a previous services which had the annotation set.
    • The IP has to be in the same region as the load balancer.

Among others this makes a few interesting scenarios possible:

  • Delete and recreate an existing service without losing the IP. Sometimes this is for example necessary to reinstall/upgrade a helm chart.
  • Move a floating IP to another cluster in the same region (and the same project). (The nodes of the new cluster need to be created before the old service is deleted, otherwise a node gets the IP assigned.)
  • Making sure to always have the same IP because other systems/firewalls depend on it.

If you need even more control of your used IP addresses you can also rent a dedicated IP space from us or bring your own IP space, please contact us if you are interested.

Migrate an existing floating IP from one loadbalancer another

With the solution above you may create a workflow to migrate an existing floating IP from one loadbalancer to another (e.g. when switching the type of ingress controller or other migration cases). This only works if the floating IP stays in the same OpenStack project.

  • Set keep-floating IP on old loadbalancer Service to true to keep the floating IP allocated in the OpenStack Project
    • metadata.annotations.loadbalancer.openstack.org/keep-floatingip="true"
  • Delete old loadbalancer
    • Kubernetes will delete the loadbalancer in OpenStack, but will keep the IP in the project
  • Create the new loadbalancer service with keep-floatingip set to true and point to the old IP
    • metadata.annotations.loadbalancer.openstack.org/keep-floatingip="true"
    • spec.loadBalancerIP="xxx.xxx.xxx.xxx"
    • Kubernetes will create a new loadbalancer in OpenStack and associate the old IP the new loadbalancer

Use a custom floating ip network

The example below configures the service to use a custom floating ip network. You will need to add the floating ip network id you have to the service.
By default the standard metakube floating ip network will be used and no further configuration is required.

Additional yaml to add to the service for a custom floating ip network

kind: Service
apiVersion: v1
metadata:
  name: ...
  annotations:
    loadbalancer.openstack.org/floating-network-id: "*****"
spec:
  ...