Wednesday, April 7, 2010

Ethernet Bonding (Linux)

Bonding ethernet channels in Linux is easy. With the right configuration and hardware, it can be done in minutes.

In this example, I have a 4 port intel Gigabit card and I wanted to utilize this and my Network switches LACP/LAG capability. Lets assume we have the Network switch part already configured to have its ports utilize LACP/LAG.

On the RHEL 4.3 server that is hosting my samba file server below are the steps outlined to configure ethernet channel bonding/teaming on the RHEL 4.3 server.


Edit the /etc/modprobe.conf file and add the bonding kernel module and its mode.

# vi /etc/modprobe.conf

#regular network card (built-in)
alias eth0 e1000
alias eth1 e1000
#the bond interface 
alias bond0 bonding
#bond interface mode. mode=4 (802.3ad/link aggregation)
options bond0 mode=4 miimon=100 
#Intel 4 port Gigabit ethernet card
alias eth5 e1000
alias eth3 e1000
alias eth4 e1000
alias eth2 e1000
alias scsi_hostadapter ahci
alias usb-controller ehci-hcd
alias usb-controller1 uhci-hcd


Now make sure to edit the /etc/sysconfig/network-scripts/ifcfg-ethX (where X are the ethernet ports that will be part of the bond0 LACP interface) 

# vi /etc/sysconfig/network-scripts/ifcfg-eth2

DEVICE=eth2
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
TYPE=Ethernet


Now we don't want to edit every single file as its a boring and error prone process

# for i in {3..5} ; do cp /etc/sysconfig/network-scripts/ifcfg-eth2 /etc/sysconfig/network-scripts/ifcfg-eth$i ; done

#for i in {3..5} ; do sed -i "s/DEVICE=eth2/DEVICE=eth$i/g" /etc/sysconfig/network-scripts/ifcfg-eth$i ; done


Now create the bond0 interface configuration file.

# vi /etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0
IPADDR=192.168.1.30
NETWORK=192.168.1.0
NETMASK=255.255.255.0
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet


Load the bonding kernel module.

# modprobe bonding

Now bring up the bond0 interface

# ifup bond0


Verify the bond0 interface

# cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v2.6.1 (October 29, 2004)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Active Aggregator Info:
        Aggregator ID: 1
        Number of ports: 1
        Actor Key: 17
        Partner Key: 1
        Partner Mac Address: 00:00:00:00:00:00

Slave Interface: eth2
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:15:17:6a:c1:e4
Aggregator ID: 1

Slave Interface: eth3
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:15:17:6a:c1:e5
Aggregator ID: 2

Slave Interface: eth4
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:15:17:6a:c1:e6
Aggregator ID: 3

Slave Interface: eth5
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:15:17:6a:c1:e7
Aggregator ID: 4


You can now try transferring files to/from the samba servers shared folders, once the transfer process is on-going, try unplugging the network cables that are part of the bond/LACP/LAG on the Network switch or on the configured 4 port Intel ethernet card.

3 comments:

  1. Only problem with the above is that you aren't showing the extra config that has to go into each of the ifcfg-ethX files that would make each NIC part of the bond0.

    USERCTL=no
    MASTER=bond0
    SLAVE=yes

    These are the options that would go into the ifcfg-eth2, ifcfg-eth3, ifcfg-eth4, ifcfg-eth5 files so that they would join the bond as slaves. Naturally each of those files would not have IP address info either, etc.

    ReplyDelete
  2. If you wanna do this on newer Debian or Debian based systems:

    apt-get install ifenslave-2.6

    edit /etc/network/interraces to look like this:

    auto bond0
    iface bond0 inet static
    address 10.0.1.2
    netmask 255.255.255.0
    network 10.0.1.0
    broadcast 10.0.1.255
    gateway 10.0.1.1
    dns-nameservers 10.0.1.1
    dns-search
    bond_mode active-backup
    bond_miimon 100
    bond_downdelay 200
    bond_updelay 200
    slaves eth0 eth1

    ...........

    obviously if you have more than just eth0 and eth1 you can add them to the slaves line. Oh and if you want ip aliases?

    ...........

    auto bond0:1
    iface bond0:1 inet static
    address 10.0.1.3
    netmask 255.255.255.0
    network 10.0.1.0
    broadcast 10.0.1.255

    ...........

    I usually just reboot to have everything take affect but I think the rest of the commands in the blog post work the same on Debian type systems.

    ReplyDelete
  3. @Kyle

    Yes you are correct as I didn't show that part of the configuration. I have it added now, and many thanks for pointing it out.

    @Matthew

    Thanks for the Debian based system part. I will write one for Debian using your configuration that you posted.

    ReplyDelete