RabbitMQ

Overview

RabbitMQ is a lightweight message broker which can easily be deployed on our Metakube Core.

The source and a default configuration can be found like of any other building block on code.sysEleven.de. Infos on release notes and new features please follow Release notes redis-ha

Adding the Building Block

You are good to go with the recommended general cluster configuration to meet the RabbitMQ recommended configuration. Keep in mind that it also needs to fit to your use case and your requirements.

To get your rabbitmq building block in service quickly, proceed with our recommended way and proceed with the mkactl tool.

mkactl block list-all

existing building blocks:

 NAME                    LATEST      INSTALLED
 pg-operator             0.1.0
 rabbitmq                0.3.0
 velero                  1.6.5
 local-path-provisioner  1.24.2
 redis-ha                1.31.0
 kube-prometheus-stack   26.0.1
 ingress-nginx           5.30.3
 loki-promtail           6.3.3
 openvpn                 3.23.3
 external-dns            5.16.2
 elasticsearch           2.32.3
 cert-manager            4.31.1
 memcached               2.6.3
 tideways-daemon         1.24.1
 pxc-operator            3.34.18

#choose Rabbitmq

mkactl block add rabbitmq

Last thing you proceed with is to publish the building block to your control repository.

mkactl repo publish

Required configuration

If you have provisioned the RabbitMQ building block you have already a running building block.

To access the management interface you need to fulfill some prerequisites :

  • setting up environment variables in GitLab for necessary configuration
  • a port forwarding must be activated:

The following configuration block must be added to the ./rabbitmq/.gitlab-ci.yml file.

variables:
  BASEDIR_RABBITMQ: "${CI_PROJECT_DIR}/rabbitmq"

You need to set the password for the RabbitMQ Managment UI in gitlab.

The following variables must be created in the ControlRepository,

  • RABBITMQ_ADMIN_PRODUCTION_PASSWD
  • RABBITMQ_ADMIN_STAGING_PASSWD
  • RABBITMQ_ADMIN_DEVELOPMENT_PASSWD

It is recommended to mask the variables in the settings. See: Mask a CI/CD Variable

Depending on the environments which you want to use, appropriate passwords should be provided. You can of course configure the passwords for all 3 environments.

GitLab Variables

Next step: check out your control repo.


git clone https://code.syseleven.de/<YourGitlabUser>/<youControlRepo>.git

Depending on your environment you need to add the following variable to the appropriate files.

rabbitmq/values-rabbitmq-staging.yaml.gotmpl

auth:
  password: {{ requiredEnv "RABBITMQ_ADMIN_STAGING_PASSWD" }}

rabbitmq/values-rabbitmq-production.yaml.gotmpl

auth:
  password: {{ requiredEnv "RABBITMQ_ADMIN_PRODUCTION_PASSWD" }}

rabbitmq/values-rabbitmq-development.yaml.gotmpl

auth:
  password: {{ requiredEnv "RABBITMQ_ADMIN_DEVELOPMENT_PASSWD"  }}
kubectl port-forward --namespace syseleven-rabbitmq svc/rabbitmq 15672:15672

Now you can access the management UI by using 127.0.0.1:15672.

RabbitMQ Landingpage

In the Overview section of the Management Interface you can export and import the definitions. You can export the current configuration as a Json file or import existing configuration from other RabbitMQ clusters, see: Schema Definition Export and Import.

An overview of possible configuration options are available on RabbitMQ BuildingBlock.

RabbitMQ CLI

RabbitMQ provides a number of CLI tools. All these tools are documented in detail under: Command Line Tools
You can use the CLI tools in any rappitMQ pod.

Example

kubectl exec -n syseleven-rabbitmq rabbitmq-0 -- rabbitmqctl --help

list of provided CLI tools

  • rabbitmqctl for service management and general operator tasks
  • rabbitmq-diagnostics for diagnostics and health checking
  • rabbitmq-plugins for plugin management
  • rabbitmq-queues for maintenance tasks on queues, in particular quorum queues
  • rabbitmq-upgrade for maintenance tasks related to upgrades

Scaling Setup

Stick to the official documentation of RabbitMQ to learn more about scaling and further configuration options.

Known issues

Besides the following known issues please always keep an eye on the official announcements from the upstream project documentation.

Admin password change

If you want to change the admin password for the cluster you need to remove the rabbitmq secret before applying the changed version.

$ kubectl delete --namespace syseleven-rabbitmq secret rabbitmq

Broken cluster state

IMPORTANT: Some of these procedures can lead to data loss. Always make a backup beforehand. !

The RabbitMQ cluster is able to support multiple node failures but, in a situation in which all the nodes are brought down at the same time, the cluster might not be able to self-recover.

This happens if the pod management policy of the statefulset is not parallel and the last pod to be running wasn't the first pod of the statefulset. If that happens, update the pod management policy to recover a healthy state:


$ kubectl delete statefulset STATEFULSET_NAME --cascade=false
$ helm upgrade RELEASE_NAME bitnami/rabbitmq \
--set podManagementPolicy=Parallel \
--set replicaCount=NUMBER_OF_REPLICAS \
--set auth.password=PASSWORD \
--set auth.erlangCookie=ERLANG_COOKIE

For a faster resynchronization of the nodes, you can temporarily disable the readiness probe by setting readinessProbe.enabled=false. Bear in mind that the pods will be exposed before they are actually ready to process requests.

If the steps above don't bring the cluster to a healthy state, it could be possible that none of the RabbitMQ nodes
think they were the last node to be up during the shutdown. In those cases, you can force the boot of the nodes by specifying the clustering.forceBoot=true parameter (which will execute rabbitmqctl force_boot in each pod):


$ helm upgrade RELEASE_NAME bitnami/rabbitmq \
--set podManagementPolicy=Parallel \
--set clustering.forceBoot=true \
--set replicaCount=NUMBER_OF_REPLICAS \
--set auth.password=PASSWORD \
--set auth.erlangCookie=ERLANG_COOKIE

Please stick to the official RabbitMQ Documentation to get more details on that topic

Release-Notes

Please find more infos on release notes and new features Release notes Rabbitmq