Membership Protocol Explained

Ringpop's membership protocol allows nodes in a cluster to efficiently communicate with each other and maintain a consistent view of the overall cluster membership. It is a form of a gossip protocol, which is a decentralized way for nodes to broadcast information throughout a network.

Specifically, Ringpop uses a variation of the SWIM protocol (Scalable Weakly-consistent Infection-style process group Membership). This is a well-known gossip protocol for managing cluster membership.

How SWIM Works

The key ideas behind SWIM are:

• Each node maintains a local membership list of other nodes in the cluster. This contains:
  • Node address
  • Status (alive, faulty, suspect, etc)
  • Incarnation number (logical clock for that node)
• Nodes randomly ping other nodes to detect failures.
• When a node detects a change (node join, failure, etc), it gossips this change to a random subset of other nodes.
• Nodes receiving a gossip message merge it with their local membership list.
• A checksum is calculated from the membership list to detect discrepancies.
• If a gossiped membership list has a different checksum than the local list, it is forwarded to a random subset of nodes.

This allows membership changes to spread epidemically through the cluster without any centralized coordination.

Benefits of Gossip Protocols

Gossip-style membership protocols have many benefits compared to traditional heartbeat protocols:

• Lower overhead - only periodic random pings instead of all-to-all heartbeats
• Faster failure detection - changes propagate exponentially fast
• Eventually consistent views - frequent anti-entropy pings reconcile differences
• Fault tolerant - no single point of failure, decentralized

This makes them a very scalable and resilient way to manage cluster membership.

Membership List Details

Ringpop's membership list contains some additional details:

• Node status - alive, faulty, suspect
• Incarnation number - logical clock for that node
• Timestamp - wall clock time of last status change

The membership list checksum is calculated by hashing all of these fields.

When a node receives a membership list gossip message, it checks if the checksum matches its local list. If not, it merges the changes and forwards the updated list to a subset of other nodes.

This allows ringpop to efficiently detect and propagate cluster changes while maintaining loosely consistent views across all nodes.