DHCP failover / load balancing (and synchronization) with CentOS 6


DHCP is a wonderful piece of software. It keeps our networks running smoothly. For small networks, probably 100 machines or so, one server is enough, but to larger networks, is not a bad idea to have another one, just in case the firts one fails or the load is just to much..

DHCP has some configurations for load balancing and failover, the – failover declaration – that allows us to configure it.

The Servers:

Primary IP address: 10.1.2.1

Secondary IP address: 10.1.2.2

To keep things simple, you can create a new file, and then just insert it in the global dhcpd.conf (configuration below)

Primary DHCP server

Open a new file and put the following lines in it:

vi /etc/dhcp/dhcpd.failover

# Failover specific configurations

failover peer “dhcp” {
primary;
address 10.1.2.1;
port 647;
peer address 10.1.2.2;
peer port 647;
max-response-delay 60;
max-unacked-updates 10;
mclt 600;
split 128;
load balance max seconds 3;
}
Now, in the secondary DHCP server, just to the same:
Secondary Server
failover peer “dhcp” {
secondary;
address 10.1.2.2;
port 647;
peer address 10.1.2.1;
peer port 647;
max-response-delay 60;
max-unacked-updates 10;
load balance max seconds 3;
}
After creating the files, just add the configuration to the global config file (in both servers), somewhere before (groups | share networks | host) definitions
include “/etc/dhcp/dhcpd.failover”;
Now, to the definitions.
To take advantage of the new definitions, we need to create a pool. You can put it in any (subnet| shared-network|etc..) declaration:
 
 
network 10.1.2.0 netmask 255.255.0.0 {
option routers 10.1.2.254;
option subnet-mask 255.255.0.0;
option broadcast-address 10.1.255.255;
pool {
failover peer “dhcp”;
range 10.1.2.1 10.1.254.254;
}
}
 
Note: If you have static declarations (i have all my clients in static declarations), to avoid warnings in the log about dynamic and static mappings, reduce the range to only one client. It’s mandatory the range declaration
IMPORTANT: Is of utmost importance to have both servers with the same date and time. If they are not, DHCP will complain and the secondary server (the whole server) will not work well… You can accomplish this with ntpdate
If you are getting this messages in syslog:
Failover CONNECT to dhcp rejected: Connection rejected, time mismatch too great.

Then the time is not the same.

If you want to know more about the configuration parameters in the failover declarations, go to the ipamworldwide web site
Synchronization
DHCP doesn’t have a synchronization mechanism (as far as i know – please correct me if i’m wrong), so changes you’ll make to the primary server will not reflect in the secondary server. This could be done using scripting or yourself manually copying the changed file over to the secondary server. But sometimes, in the heat of the moment, because something important happened or someone is waiting for your attention, you forget…
There’s a small program that can accomplish the synchronization without you even remember that you must copy the files…
iwatch is a small program that monitors wherever you want (files, directories) and upon changes, it can perform several actions.
A few months ago i wrote about iwatch and how’s installed and configured (portuguese), but i’ll replicate the steps here, using the DHCP files has an example.
The CentOS minimal installation doesn’t have rsync and wget installed, so we need to install those.
yum install rsync wget
 
Note: The steps above are only required in the primary server. The changes are made here and then replicated to the secondary server.
Install the rpmforge repositories. You can get the rpm and instructions here
Install the required perl packages
yum install perl-Event perl-Mail-Sendmail perl-XML-SimpleObject perl-XML-Simple perl-Linux-Inotify2
 
After installing, we can finally install iwatch.
Download it from sourceforge
After download, untar it:
tar -zxvf iwatch-0.2.2.tgz
A new directory is created
cd iwatch
 
In there, you’ll find a few files.
Let’s copy the files to the proper places
cp iwatch /usr/local/bin/
cp iwatch.xml /etc/
cp iwatch.dtd /etc/
A few considerations before continuing:
We want to synchronize changes in the DHCP configurations, so, we’ll monitor the /etc/dhcp directory for:
  • Creation of files
  • changes in files
  • deleting of files
  • add an exception for dhcpd.failover (those are different in the servers – depending of primary or secondary server)
Now that we know what we want, let’s proceed:
Before we edit the configuration file, so we can execute iwatch as a daemon, let’s execute it in command line and edit a file so we can see what’s happening. Open two terminals: One will be used to execute iwatch and some arguments, the other will be to edit a file.
First terminal:
Execute iwatch and see some output:
/usr/local/bin/iwatch -e modify,create,close_write -c “touch /tmp/someaction” -r -v /etc/dhcp/
 

output:
Watch /etc/dhcp
Watch /etc/dhcp/Configs
Watch /etc/dhcp/dhclient.d

 
Second terminal
Let’s edit a file in the watched directory and see what’s happening in terminal 1. You can just open it, no changes, but save the file and watch the output in Terminal 1.
vi /etc/dhcp/dhcpd.conf
In terminal 2, you’ll see:
[14/Mar/2012 16:18:34] IN_CREATE /etc/dhcp/Configs/.dhcp.vlan.swp
[14/Mar/2012 16:18:34] * Command: touch /tmp/someaction
[14/Mar/2012 16:18:34] IN_CREATE /etc/dhcp/Configs/.dhcp.vlan.swx
[14/Mar/2012 16:18:34] * Command: touch /tmp/someaction
[14/Mar/2012 16:18:34] IN_CLOSE_WRITE /etc/dhcp/Configs/.dhcp.vlan.swx
[14/Mar/2012 16:18:34] * Command: touch /tmp/someaction

Now that we saw it working, let’s configure the daemon part.

Edit the file /etc/iwatch.xml. The file syntax is XML. Here’s an example of my configuration.
You can read more in iwatch sourceforge page.
<?xml version=”1.0″ ?>
<!DOCTYPE config SYSTEM “/etc/iwatch.dtd” ><config charset=”utf-8″>

<guard email=”informatica@ulscb.min-saude.pt” name=”IWatch”/>

<watchlist>

<title>DHCP Sync</title>
<contactpoint email=”sysadmin@domain.com” name=”Administrator”/>
<path type=”recursive” syslog=”on” alert=”off” events=”create,delete,close_write” exec=”/root/scripts/syncFiles”>/etc/dhcp</path>
<path type=”regexception”>b4913b</path>
<path type=”exception”>/etc/dhcp/dhcpd.failover</path>
<path type=”exception”>/etc/dhcp/dhclient.d</path>

<path type=”regexception”>.*.swp*</path>

<path type=”regexception”>.*~</path>

</watchlist>

</config>

Now, edit that file and make the changes you want

I’ve added a few exceptions, because there are files i don’t need to sync.

Also, vi creates a few temporary files (directory 4913 and backups with ~ | swp extensions) when you’re editing, and those don’t mind.

We are not also using modify, because if a file is closed with write, it was modified, right ?

The exec  parameter tells iwatch what to do when any of the events occurs. I have a script (syncFiles) that synchronizes with the secondary server and sends and email

#!/bin/bash
# Script to synchronized dhcp changes
# This script will be called by iwatch
# DO NOT EXECUTE THIS SCRIPT – IT WILL BE EXECUTED AUTOMATICALLY
# 15/12/2011
log=”/tmp/synclog.log”
echo “Syncing dhcp from primary server to secondary server” >> $log
# Using rsync so it can only copy different files – Low on bandwith/usr/bin/rsync -avz –delete -e ssh /etc/dhcp/ –exclude dhcpd.failover root@secondary:/etc/dhcp >> $log
# Restart the service with the new configurations
ssh root@secondary -C “service dhcpd restart” >> $log
service dhcpd restart >> $log
# Email
if [ -a $log ]; then
mail sysadmin@domain.com -s “Sync dhcp ” < $log
rm -f $log
fi

I use rsync to perform the copy. I exclude dhcpd.failover because the files are not the same and they depend on the server (primary or secondary)

Notes: A few security issues. iwatch is executed with root privileges – it’s started by /etc/rc.local

If you do nothing, every time the script is executed, you’ll have to give the root password of the secondary server. You can prevent this (if you want) by adding the ssh key to the authorized keys and have a password-less ssh configuration between those two servers (using only keys)

Now, just put iwatch executing when the machine start:

vi /etc/rc.local
# Exec iwatch
/usr/local/bin/iwatch -d

Execute iwatch as daemon

/usr/local/bin/iwatch -d

Now you have a dhcp failover instalation and synchronization

Hope it helps anyone

References

http://www.lithodyne.net/docs/dhcp/dhcp-4.html

http://www.madboa.com/geek/dhcp-failover/

http://www.ipamworldwide.com/dhcp-failover-a-load-balancing/declarations.html

About these ads

6 thoughts on “DHCP failover / load balancing (and synchronization) with CentOS 6

  1. Bom dia Bruno,

    fui incumbido de achar uma solução para uso de 2 links de Internet na empresa que trabalho. Primeiramente comecei a realizar pesquisas sobre roteadores (Cisco RV042 – TP-Link TL-R480T – D-Link DI-LB604 ), depois achei algumas informações dizendo que não é necessário hardware, por software seria possível realizar o uso dos dois links com redundância. O que você menciona acima é exatamente isso? (eu sou completamente novo em rede e nunca sequer utilizei linux, me me foi passado esse trabalho, então deve ser feito.) Teria algumas dicas para mim, de como devo proceder?

    • Ola Thiago.

      O que menciono no post é balanceamento, não extamente redundancia. Em redundancia estará um dos links em standby à espera que eventualmente o outro falhe para entrar em funcionamento. Neste caso da redundância, o efeito irá ser o mesmo no final – caso um dos servidores falhe, o outro está lá para responder aos pedidos, mas enquanto isso não acontece, ambos trabalham e respondem aos pedidos.

      Neste caso o balanceamento é feito pelo proprio software, sem recorrer a programas externos.

      No seu caso também poderá ser balanceamento – podem estar os dois links a trabalhar ao mesmo tempo, balanceando a carga por ambos os links. É possivel de ser feito por software, mas como nunca trabalhou com Linux, será mais complicado, porque já coloca IPTABLES e configurações especificas de rede. Para si, recomendaria ir por soluções por hardware. Se estivermos a falar de muito trâfego, a melhor solução é mesmo hardware. – são mais faceis de configurar –

      Cumprimentos

  2. Muito obrigado Bruno.

    Irei procurar por hardwares que atendam a nossa necessidade. O importante agora é colocar os 2 links de internet ativos, pois no momento, um esta ativo e o outro desconectado. Após a compra do hardware iremos conectar ambos e deixá-los rodando juntos e caso haja falha em um dos link o outro assume o trabalho sozinho, até o retorno do outro.

    Após irei ter que descobrir como fazer o backup remoto, e utilizar o SAMBA.
    Haja leitura.

    Agradeço o auxílio.

  3. Excellent! Just got this working on CentOS 6.4 minimal. Kept getting this error when starting iwatch: Malformed UTF-8 character (fatal) at /usr/lib64/perl5/XML/LibXML/Error.pm line 217. Turns out it was the quotation marks when I cut and pasted your iwatch.xml file. Similar problems with the sycnFile script. But once I figured that out, it works like a charm!

    Thank you!

    Rob McKennon

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s