-
Notifications
You must be signed in to change notification settings - Fork 8.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[bitnami/rabbitmq] RabbitMQ: explain how to safely avoid a deployment deadlock #25931
base: main
Are you sure you want to change the base?
Conversation
Explain the problem and a widely used solution instead of recommending force booting nodes. These practices are generally used by existing RabbitMQ K8S Cluster Operators but this PR intentionally does not recommend any of them. Signed-off-by: Michael Klishin <michael@clojurewerkz.org> Signed-off-by: Michael Klishin <klishinm@vmware.com> Signed-off-by: Michael Klishin <mikhail.klishin@broadcom.com> Commit a suggested edit to bitnami/rabbitmq/README.md Co-authored-by: Carlos Rodríguez Hernández <carlosrh@vmware.com> Signed-off-by: Michael Klishin <michaelklishin@icloud.com>
@rafariossaa @carrodher @javsalgar is there anything else I can do to help push this forward? Not having this aspect documented is a big deal for RabbitMQ users. |
Bump. This is still as relevant and important to document as before. |
Thanks @michaelklishin and sorry for the delay. Our team will review and provide feedback. |
|
||
This happens if the pod management policy of the statefulset is not `Parallel` and the last pod to be running wasn't the first pod of the statefulset. If that happens, update the pod management policy to recover a healthy state: | ||
The following combination of deployment settings avoids the problem: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi,
Is there any kind of process that user could perform to once he is in this situation ?.
I mean, I am not sure if the user has already deployed without the parameters you indicate here he could avoid the issue by just upgrading the deployment with the parameter or if he may need to execute other steps.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rafariossaa change to the recommended deployment parameters and change the readiness probe. The absolute minimum required would be using rabbitmq-diagnostics ping
for readiness probe but both recommendations can be applied at the same time in a single deployment update.
The goal of this PR is to recommended the safe setup from the start.
Explain the problem and a widely used solution
instead of recommending force booting nodes.
These practices are generally used by existing
RabbitMQ K8S Cluster Operators but this PR
intentionally does not recommend any of them.
References #25698, #16081, #25916.