What's a Totem "Retransmit List" all about in Corosync?

Posted on Thu 15 March 2012 in hints-and-kinks • Tagged with Corosync • 2 min read

Occasionally, you may see errors similar to this in your system logs:

corosync [TOTEM ] Retransmit List: e4 e5 e7 e8 ea eb ed ee

Here’s what causes them, and what you can do to fix the issue.

Corosync, more specifically its Totem protocol implementation, defines a maximum number of …


Continue reading

Finding out which OSDs currently store a specific RADOS object

Posted on Fri 09 March 2012 in hints-and-kinks • Tagged with Ceph • 2 min read

Ever wanted to know just which of your OSDs a RADOS object is currently stored in? Here’s how.

Suppose you’ve got an RBD device, named test. Then you can use the rbd info command to display which name prefix is used by the RADOS objects that make up …


Continue reading

Solve a DRBD split-brain in 4 steps

Posted on Tue 06 March 2012 in hints-and-kinks • Tagged with DRBD • 3 min read

Whenever a DRBD setup runs into a situation where the replication network is disconnected and fencing policy is set to dont-care (default), there is the potential risk of a split-brain. Even with resource level fencing or STONITH setup, there are corner cases that will end up in a split-brain.

When …


Continue reading

Checking Corosync cluster membership

Posted on Sun 04 March 2012 in hints-and-kinks • Tagged with Corosync • 2 min read

It’s simple and easy to get Pacemaker’s view of the status of members in a cluster – just invoke crm_mon. But what if you want to check on the cluster membership when Pacemaker is not running, or you want to make sure whether Corosync’s view of the cluster …


Continue reading

Fencing in Libvirt/KVM virtualized cluster nodes

Posted on Wed 29 February 2012 in hints-and-kinks • Tagged with Pacemaker • 2 min read

Often, people deploy the Pacemaker stack in virtual environments for purposes of testing and evaluation. In such environments, it’s easy to test Pacemaker’s fencing capabilities by tying in with the hypervisor.

This quick howto illustrates how to configure fencing for two virtual cluster nodes hosted on a libvirt …


Continue reading