Networking

Node Addresses

Every MetaKube Node has one address of type InternalIP: the private IP in the OpenStack network of the cluster.

They have another address of type ExternalIP in case the MachineDeployment is configured to use floating IPs.

CNI Plugin

The CNI plugin is installed as a DaemonSet, so a Pod runs on each node.
The nodes must facilitate the Pod traffic (VXLAN overlay or direct routing) and allow for the respective connections.

Floating IPs

MetaKube can manage floating IPs for each server of a MachineDeployment.

This has certain implications you should consider:

  1. Node acts as NAT gateway directly

    The source IP of nodes and its Pods is distinct from other Nodes.
    This may be desirable e.g. to avoid certain IP based rate limits of certain APIs.

    Secondly, it avoids port collision for egress at the shared NAT gateway (router).

  2. Node ports are open to the public

    This may be intentional and the reason to use floating IPs in the first place.
    But also consider other ports you may not want to expose.

  3. Additional cost for floating IPs

Dedicated floating IP Pool

If you need all egress from your cluster to be from a well-known CIDR, you may consider a dedicated floating IP Pool.

Use free (unmanaged) floating IPs

This behavior is deprecated.
We are looking to replace this functionality with a more explicit mechanism.

When there's free floating IPs in the project, MetaKube will first attempt to use these to associate with machine ports.

Network peers and required connections

The nodes must be able to communicate with the following peers.

Egress

MetaKube does not restrict any egress by default

  • Apiserver
  • Konnectivity
  • DNS resolvers
  • OpenStack API
  • Various container image registries & package repositories

Ingress

Traffic from these following peers are enabled by default through security group rules

  • Other nodes for Pod networking, see CNI
  • (Optional) Node ports (30000-32767) from OpenStack Octavia Load Balancer ports (node subnet)
  • (Optional) Node ports (30000-32767) from external networks if used