Comprehensive Guide: Configuring VRRP (Keepalived) with Apache on Debian

This guide will walk you through setting up Virtual Router Redundancy Protocol (VRRP) using keepalived on two Debian servers to achieve high availability for your Apache web service.

Our Scenario:

Device 1 (Master):
- Hostname: debian-master
- Physical IP: 192.168.1.10
- Interface: eth0 (or your primary network interface)
- Apache installed and running
Device 2 (Backup):
- Hostname: debian-backup
- Physical IP: 192.168.1.20
- Interface: eth0
- Apache installed and running
Virtual IP (VIP): 192.168.1.1 (Clients will connect to this IP)

Prerequisites:

Two Debian devices: Both running a recent, identical version of Debian (e.g., Debian 11 "Bullseye" or 12 "Bookworm").
Network Connectivity: Both devices must be on the same Layer 2 network segment (VLAN, broadcast domain) and able to communicate with each other using their physical IPs.
Apache Installed: Apache (apache2 package) should be installed and configured on both servers. Ensure your web content is synchronized between them (e.g., via rsync, csync2, or a shared storage solution).
Root/Sudo Access: You'll need administrative privileges on both machines.

Step 1: Install `keepalived` on Both Devices

On both debian-master and debian-backup, open a terminal and run:

sudo apt update
sudo apt install keepalived -y

Step 2: Configure `keepalived` on `debian-master` (Master)

Backup the original configuration file (optional but recommended):

sudo cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak

Edit the configuration file:

sudo nano /etc/keepalived/keepalived.conf

Add the following content (delete existing content or comment it out). Important: Replace eth0 with your actual network interface name if different. Adjust mysecretpassword to a strong, unique password.

# Global Configuration (Optional)
global_defs {
   router_id LVS_DEBIAN_MASTER
   # Uncomment and configure if you want email alerts
   # notification_email {
   #   admin@example.com
   # }
   # notification_email_from keepalived@example.com
   # smtp_server 127.0.0.1
   # smtp_connect_timeout 30
}

# Health check script for Apache
vrrp_script chk_apache {
    script "systemctl is-active apache2" # Checks if the apache2 service is active
    interval 2                           # Check every 2 seconds
    weight 2                             # Increase priority by 2 if the script succeeds
    fall 2                               # Number of failures before state change (e.g., if Apache stops 2 times)
    rise 2                               # Number of successes before state change (e.g., if Apache starts 2 times)
}

# VRRP Instance for the Virtual IP
vrrp_instance VI_1 {
    state MASTER                   # This device is the master initially
    interface eth0                 # Your primary network interface (e.g., ens18, enp0s3)
    virtual_router_id 51           # Unique ID for this VRRP instance (1-255). Must be same on both servers.
    priority 100                   # Higher priority means it will be master (1-254, 255 is for CARP).
    advert_int 1                   # Advertisement interval in seconds (how often heartbeats are sent).

    authentication {
        auth_type PASS
        auth_pass mysecretpassword # Simple password for VRRP authentication. Must be same on both servers.
    }

    virtual_ipaddress {
        192.168.1.1/24             # The Virtual IP address. Use CIDR notation.
    }

    track_script {
        chk_apache                 # Track the custom script for Apache's health
    }

    # Custom scripts to run on state changes (optional but recommended for alerts/actions)
    notify_master "/etc/keepalived/notify.sh MASTER"
    notify_backup "/etc/keepalived/notify.sh BACKUP"
    notify_fault "/etc/keepalived/notify.sh FAULT"
    notify_stop "/etc/keepalived/notify.sh STOP"
    # smtp_alert                   # Enable email alerts if global_defs are configured
}

Key Directives Explained:

global_defs: Optional block for global settings like router_id and email alerts.
vrrp_script chk_apache: Defines a health check. systemctl is-active apache2 checks if the Apache service is running. If it fails, keepalived will reduce the server's priority, potentially triggering a failover. fall and rise provide robustness against flapping.
vrrp_instance VI_1: Defines the VRRP group.
state MASTER: This server will attempt to become the master.
interface eth0: The network interface keepalived will monitor and add the VIP to.
virtual_router_id 51: A unique ID for this specific VRRP group. All servers in the group must use the same ID.
priority 100: The priority for this server. Higher priority wins.
advert_int 1: How frequently VRRP packets are sent.
authentication: Crucial for security. Use PASS for simple password authentication. The password must match across all VRRP group members.
virtual_ipaddress: The actual IP address that clients will use to access Apache.
track_script chk_apache: Links the vrrp_script to this VRRP instance, so its health check influences the server's priority.
notify_master, notify_backup, notify_fault, notify_stop: These allow keepalived to execute custom shell scripts (notify.sh) whenever the server's state in the VRRP instance changes.

Create the notify.sh script (optional but highly recommended for logging/alerts):

sudo nano /etc/keepalived/notify.sh

Add the following content:

#!/bin/bash
TYPE=$1    # e.g., "MASTER", "BACKUP", "FAULT", "STOP"
# NAME=$2  # Not used in this simple script but available (e.g., "VI_1")
STATE=$2   # Current state

TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

case $STATE in
    "MASTER")
        echo "$TIMESTAMP: VRRP State Change: Master for VRRP instance on $(hostname) ($TYPE). VIP: $(ip addr show eth0 | grep -oP 'inet (\d+\.\d+\.\d+\.\d+)/24' | awk '{print $2}' | cut -d'/' -f1)" | systemd-cat -t keepalived-notify
        # Add any other master-specific actions here, e.g., starting other services
        ;;
    "BACKUP")
        echo "$TIMESTAMP: VRRP State Change: Backup for VRRP instance on $(hostname) ($TYPE)." | systemd-cat -t keepalived-notify
        ;;
    "FAULT")
        echo "$TIMESTAMP: VRRP State Change: Fault state for VRRP instance on $(hostname) ($TYPE). Investigate issues!" | systemd-cat -t keepalived-notify
        # Consider sending email alert here if not using smtp_alert
        ;;
    "STOP")
        echo "$TIMESTAMP: VRRP State Change: Keepalived stopping for VRRP instance on $(hostname) ($TYPE)." | systemd-cat -t keepalived-notify
        ;;
    *)
        echo "$TIMESTAMP: VRRP State Change: Unknown state $STATE for VRRP instance on $(hostname) ($TYPE)." | systemd-cat -t keepalived-notify
        ;;
esac

exit 0

Make the script executable:

sudo chmod +x /etc/keepalived/notify.sh

Step 3: Configure `keepalived` on `debian-backup` (Backup)

Backup the original configuration file:

sudo cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak

Edit the configuration file:

sudo nano /etc/keepalived/keepalived.conf

Add the following content. Notice the state is BACKUP and priority is lower than the master.

# Global Configuration (Optional)
global_defs {
   router_id LVS_DEBIAN_BACKUP
   # notification_email {
   #   admin@example.com
   # }
   # notification_email_from keepalived@example.com
   # smtp_server 127.0.0.1
   # smtp_connect_timeout 30
}

# Health check script for Apache
vrrp_script chk_apache {
    script "systemctl is-active apache2" # Checks if the apache2 service is active
    interval 2
    weight 2
    fall 2
    rise 2
}

# VRRP Instance for the Virtual IP
vrrp_instance VI_1 {
    state BACKUP                   # This device is the backup initially
    interface eth0                 # Your primary network interface
    virtual_router_id 51           # Must be the same as the master
    priority 90                    # Lower priority than the master (e.g., 90 vs 100)
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass mysecretpassword # Must be the same as the master
    }

    virtual_ipaddress {
        192.168.1.1/24             # The Virtual IP address
    }

    track_script {
        chk_apache
    }

    notify_master "/etc/keepalived/notify.sh MASTER"
    notify_backup "/etc/keepalived/notify.sh BACKUP"
    notify_fault "/etc/keepalived/notify.sh FAULT"
    notify_stop "/etc/keepalived/notify.sh STOP"
    # smtp_alert
}

Create the same notify.sh script on the backup server:

sudo nano /etc/keepalived/notify.sh

Add the same content as before and make it executable:

#!/bin/bash
TYPE=$1
STATE=$2

TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

case $STATE in
    "MASTER")
        echo "$TIMESTAMP: VRRP State Change: Master for VRRP instance on $(hostname) ($TYPE). VIP: $(ip addr show eth0 | grep -oP 'inet (\d+\.\d+\.\d+\.\d+)/24' | awk '{print $2}' | cut -d'/' -f1)" | systemd-cat -t keepalived-notify
        ;;
    "BACKUP")
        echo "$TIMESTAMP: VRRP State Change: Backup for VRRP instance on $(hostname) ($TYPE)." | systemd-cat -t keepalived-notify
        ;;
    "FAULT")
        echo "$TIMESTAMP: VRRP State Change: Fault state for VRRP instance on $(hostname) ($TYPE). Investigate issues!" | systemd-cat -t keepalived-notify
        ;;
    "STOP")
        echo "$TIMESTAMP: VRRP State Change: Keepalived stopping for VRRP instance on $(hostname) ($TYPE)." | systemd-cat -t keepalived-notify
        ;;
    *)
        echo "$TIMESTAMP: VRRP State Change: Unknown state $STATE for VRRP instance on $(hostname) ($TYPE)." | systemd-cat -t keepalived-notify
        ;;
esac

exit 0

sudo chmod +x /etc/keepalived/notify.sh

Step 4: Enable and Start `keepalived` on Both Devices

On both debian-master and debian-backup, run:

sudo systemctl enable keepalived
sudo systemctl start keepalived

Step 5: Verify VRRP Configuration

Check keepalived service status on both machines:
```
sudo systemctl status keepalived
```
- Expected output: Active: active (running). Look for any error messages in the output.
Check for the Virtual IP on the Master:
On debian-master, run:
```
ip addr show eth0 # Replace eth0 with your interface name
```
- Expected output: You should see inet 192.168.1.1/24 configured on eth0 (or sometimes as eth0:vrrp).
On debian-backup, you should not see the VIP initially when the master is healthy.
Check the keepalived logs for state transitions:
On both machines, check the journal logs for keepalived messages:
```
sudo journalctl -u keepalived -f
```
- Expected output: You should see messages like Transition to MASTER state on debian-master and Transition to BACKUP state on debian-backup. You should also see messages related to chk_apache script execution.
Test Failover (Apache Service Failure):
- On debian-master (the current master), stop the Apache service:
```
sudo systemctl stop apache2
```
- Observe logs: Watch sudo journalctl -u keepalived -f on both servers. You should see debian-master reduce its priority due to chk_apache failing, and debian-backup should transition to MASTER state.
- Verify VIP on Backup: On debian-backup, check ip addr show eth0. The VIP 192.168.1.1 should now be present.
- Test connectivity: From a client machine, try to access http://192.168.1.1. It should now be served by debian-backup.
- Start Apache on original master:
```
sudo systemctl start apache2
```
- Observe Preemption: The original master (with its higher priority and healthy Apache) should reclaim the VIP. Check ip addr show eth0 on both to confirm the VIP moves back.
Test Failover (Server Failure):
- On debian-master, gracefully shut down the server or simply stop keepalived:
```
sudo systemctl stop keepalived
# Or sudo shutdown -h now
```
- Verify VIP on Backup: The VIP should quickly move to debian-backup.
- Bring master back online: Start keepalived or reboot the server. The master should reclaim the VIP once it's fully operational.

Basic Troubleshooting Commands

When troubleshooting VRRP with Apache, these commands are invaluable:

Check keepalived service status:
```
sudo systemctl status keepalived
```
- Purpose: See if the service is running and if there are any immediate errors.
- What to look for: Active: active (running). Any error messages like "Failed to load configuration" indicate syntax issues.
View keepalived detailed logs:
```
sudo journalctl -u keepalived -f
```
- Purpose: Real-time monitoring of keepalived actions, state changes, and script execution results.
- What to look for:
  - Transition to MASTER state or Transition to BACKUP state.
  - Messages from your notify.sh script.
  - Results of chk_apache script (script chk_apache succeeded or failed).
  - Authentication errors.
  - Warnings about network interfaces.
Check network interface configuration (especially for the VIP):
```
ip addr show eth0 # Replace eth0 with your network interface
```
- Purpose: Confirm if the Virtual IP is correctly assigned to the expected master server and not present on the backup (when the master is active).
- What to look for: inet 192.168.1.1/24 should appear on the active master.
Ping the Virtual IP from a third machine:
```
ping 192.168.1.1
```
- Purpose: Verify client accessibility to the VIP.
- What to look for: Consistent replies from the VIP. If it's intermittent during failover, that's expected briefly.
Check network connectivity between VRRP nodes (physical IPs):
```
ping 192.168.1.10 # From backup to master's physical IP
ping 192.168.1.20 # From master to backup's physical IP
```
- Purpose: Ensure basic Layer 2 network communication is working, as VRRP relies on multicast.
- What to look for: Successful pings.
Verify Apache service status:
```
sudo systemctl status apache2
```
- Purpose: Confirm if Apache is actually running independently of keepalived. This is what chk_apache checks.
- What to look for: Active: active (running).
Check for keepalived process:
```
pgrep keepalived # Returns PID if running
ps aux | grep keepalived
```
- Purpose: Quick check to see if the process is active.
Verify firewall rules (if any - ufw or iptables):
VRRP uses IP protocol number 112 (not TCP/UDP port). Ensure your firewall allows this protocol.
- For ufw:
```
sudo ufw status verbose
sudo ufw allow vrrp
sudo ufw reload
```
- For iptables:
```
sudo iptables -L -n -v
sudo iptables -A INPUT -p 112 -j ACCEPT
# If you save iptables rules, remember to persist them
```
- What to look for: Firewall blocking VRRP advertisements will prevent proper failover.
Check sysctl settings (IP Non-local Bind):
keepalived requires net.ipv4.ip_nonlocal_bind to be enabled (1). It's usually default, but worth checking.
```
sysctl net.ipv4.ip_nonlocal_bind
```
- Expected output: net.ipv4.ip_nonlocal_bind = 1. If not, enable it permanently:
```
echo "net.ipv4.ip_nonlocal_bind = 1" | sudo tee /etc/sysctl.d/99-keepalived.conf
sudo sysctl -p /etc/sysctl.d/99-keepalived.conf
```
Use tcpdump to capture VRRP traffic:
This is an advanced step, but very useful for deep network debugging.
```
sudo tcpdump -i eth0 host 224.0.0.18 and proto 112 -v
```
- Purpose: See if VRRP advertisement packets are actually being sent and received between your nodes. 224.0.0.18 is the standard multicast address for VRRP.
- What to look for: Packets originating from the master's physical IP, and the backup receiving them. If the master isn't sending, or the backup isn't receiving, it points to a network or firewall issue.

By following these steps, you should have a robust and highly available Apache setup using VRRP. Remember to thoroughly test your failover scenarios in a controlled environment before deploying to production.