30.12. Common Address Redundancy Protocol (CARP)

Contributed by Tom Rhodes.
Updated by Allan Jude.

The Common Address Redundancy Protocol (CARP) allows multiple hosts to share the same IP address and provide high availability. One or more hosts can fail, and the others will take over for the failed system transparently. In addition to the shared IP address, hosts also have a unique IP address for management and configuration, as in the example provided here.

30.12.1. Using CARP for High Availability

CARP is often used to provide high availability for one or more services. This example configures failover support with three hosts, all with unique IP addresses, but providing the same web content. These machines are load balanced with a Round Robin DNS configuration. The master and backup machines are configured identically except for their hostnames and management IP addresses. These servers must have the same configuration and run the same services. When the failover occurs, requests to the service on the shared IP address can only be answered correctly if the backup server has access to the same content. The backup machine has two additional CARP interfaces, one for each of the master content server's IP addresses. When a failure occurs, the backup server will pick up the failed master machine's IP address. Users will not see a service failure at all.

This example has two different masters named hosta.example.org and hostb.example.org, with a shared backup named hostc.example.org.

Each virtual IP address has a unique identification number known as a Virtual Host Identification (VHID). All of the machines that share an IP address have the same VHID. The VHID for each virtual IP address must be unique across the broadcast domain of the network interface.

30.12.2. Using CARP on FreeBSD 10 and Later

Enable support for CARP by loading the carp.ko kernel module in /boot/loader.conf:

carp_load="YES"

The CARP module can also be built into the FreeBSD kernel as described in Chapter 9, Configuring the FreeBSD Kernel:

device	carp

The hostname, management IP address, CARP configuration, and the IP address to be shared are all set by adding entries to /etc/rc.conf. This example is for hosta.example.org:

hostname="hosta.example.org"
ifconfig_em0="inet 192.168.1.3 netmask 255.255.255.0"
ifconfig_em0_alias0="vhid 1 pass testpass alias 192.168.1.50/32"

On hostb.example.org:

hostname="hostb.example.org"
ifconfig_em0="inet 192.168.1.4 netmask 255.255.255.0"
ifconfig_em0_alias0="vhid 2 pass testpass alias 192.168.1.51/32"

Note:

The passwords specified with ifconfig(8) pass must be identical. CARP will only listen to and accept advertisements from machines with the correct password.

The third machine, hostc.example.org, is prepared to handle failover from either of the previous hosts. This machine is configured with two CARP VHIDs, one to handle the virtual IP address of each of the master hosts. advskew, the CARP advertising skew, is set to ensure that the backup host advertises later than the master. advskew controls the order of precedence when there are multiple backup servers. Set the configuration in /etc/rc.conf:

hostname="hostc.example.org"
ifconfig_em0="inet 192.168.1.5 netmask 255.255.255.0"
ifconfig_em0_alias0="vhid 1 advskew 100 pass testpass alias 192.168.1.50/32"
ifconfig_em0_alias1="vhid 2 advskew 100 pass testpass alias 192.168.1.51/32"

Having two CARP VHIDs configured means that hostc.example.org will notice if either of the master servers becomes unavailable. If a master fails to advertise before the backup server, the backup server will pick up the shared IP address until the master becomes available again.

Note:

Preemption is disabled by default. If preemption has been enabled, hostc.example.org might not release the virtual IP address back to the original master server. The administrator can force the backup server to return the IP address to the master with the command:

# ifconfig em0 vhid 1 state backup

At this point, either networking must be restarted or the machine rebooted, then CARP is enabled.

CARP functionality can be controlled via several sysctl(8) variables documented in the carp(4) manual pages. Other actions can be triggered from CARP events by using devd(8).

30.12.3. Using CARP on FreeBSD 9 and Earlier

Enable support for CARP by loading the if_carp.ko kernel module in /boot/loader.conf:

if_carp_load="YES"

CARP can also be built into the FreeBSD kernel as described in Chapter 9, Configuring the FreeBSD Kernel:

device	carp

The CARP devices themselves may be created using ifconfig(8):

# ifconfig carp0 create

Set the hostname, configure the management IP address, then configure CARP and the IP address to be shared by adding the required lines to /etc/rc.conf. Here are example lines for hosta.example.org:

hostname="hosta.example.org"
ifconfig_fxp0="inet 192.168.1.3 netmask 255.255.255.0"
cloned_interfaces="carp0"
ifconfig_carp0="vhid 1 pass testpass 192.168.1.50/24"

On hostb.example.org:

hostname="hostb.example.org"
ifconfig_fxp0="inet 192.168.1.4 netmask 255.255.255.0"
cloned_interfaces="carp0"
ifconfig_carp0="vhid 2 pass testpass 192.168.1.51/24"

Note:

The passwords specified with ifconfig(8) pass must be identical. CARP will only listen to and accept advertisements from machines with the correct password. The VHID must also be unique for each virtual IP address.

The third machine, hostc.example.org, is prepared to handle failover from either of the previous hosts. This machine is configured with two CARP devices, one to handle each of the virtual IP address of each of the master hosts. Setting the advskew controls the CARP advertising skew. The skew ensuring that the backup hosts advertises later than the master, and controls the order of precedence when there are multiple backup servers. Set the configuration in /etc/rc.conf:

hostname="hostc.example.org"
ifconfig_fxp0="inet 192.168.1.5 netmask 255.255.255.0"
cloned_interfaces="carp0 carp1"
ifconfig_carp0="vhid 1 advskew 100 pass testpass 192.168.1.50/24"
ifconfig_carp1="vhid 2 advskew 100 pass testpass 192.168.1.51/24"

Having two CARP devices configured means that hostc.example.org will notice if either of the master servers becomes unavailable. If a master fails to advertise before the backup server, the backup server will pick up the shared IP address until the master becomes available again.

Note:

Preemption is disabled in the GENERIC FreeBSD kernel. If Preemption has been enabled with a custom kernel, hostc.example.org may not release the IP address back to the original content server. The administrator can force the backup server to return the IP address to the master with the command:

# ifconfig carp0 down && ifconfig carp0 up

This should be done on the carp interface which corresponds to the correct host.

At this point, either networking must be restarted or the machine rebooted, then CARP is enabled.

CARP functionality can be controlled via several sysctl(8) variables documented in the carp(4) manual pages. Other actions can be triggered from CARP events by using devd(8).

All FreeBSD documents are available for download at http://ftp.FreeBSD.org/pub/FreeBSD/doc/

Questions that are not answered by the documentation may be sent to <freebsd-questions@FreeBSD.org>.
Send questions about this document to <freebsd-doc@FreeBSD.org>.