[plug] Ethernet Bonding

[Adrian Chadd]

On Fri, Oct 19, 2007, Arie Hol wrote:

> I have just been doing some reading at :
> 
> http://linux-net.osdl.org/index.php/Bonding

Lets take a quick look.

> I am new to all of these networking aspects, so please excuse my ignorance.
> 
> I get the impression that you need a set of ID files, 
> ie : one id file for each slave device..

Well, you're already in the realm of distribution-specific information.

> Are you using a set of ifcfg-id files ?

Yup, distribution-specific info.

> 
> Where each file name would be something like :
> 
> ifcfg-id-xx:xx:xx:xx:xx:xx
> 
> Where the xx 's are replaced by the slave device's permanent MAC address
> 
> Each file would look something like this :
> 
> BOOTPROTO='dhcp'
>  STARTMODE='on'
>  USERCTL='no'
>  UNIQUE='XNzu.WeZGOGF+4wE'
>  _nm_name='bus-pci-0001:61:01.0'
> 
> Then there is also a main configuration file with a name something like :

Redhat/Fedora/CentOS?

> ifcfg-bondX
> 
> And the contents of this file would look something like :
> 
> BOOTPROTO="static"
>  BROADCAST="10.0.2.255"
>  IPADDR="10.0.2.10"
>  NETMASK="255.255.0.0"
>  NETWORK="10.0.2.0"
>  REMOTE_IPADDR=""
>  STARTMODE="onboot"
>  BONDING_MASTER="yes"
>  BONDING_MODULE_OPTS="mode=active-backup miimon=100"
>  BONDING_SLAVE0="eth0"
>  BONDING_SLAVE1="bus-pci-0000:06:08.1"
> 
> 
> All of the above is covered under the heading - 
> 
> "Configuration with sysconfig support"
> 
> in the page at :
> 
> http://linux-net.osdl.org/index.php/Bonding
> 
> Am I on the right track here ?
> 
> I find this topic very interesting, I just hope I haven't jumped in at the 
> wrong end.

You need to read up on the theory and configuration before worrying about
how it works in sysconfig support. :p

The idea behind channel bonding is:

* You've got N links, N being generally 1 or more;
* You wish to be able to add and subtract links at will without having to
  change very much (ie, traffic keeps flowing when you add a link or when
  a link fails);
* You want to be able to load balance across the links somehow, so you can
  actually do more than "just" active/standby;

Now, there's some caveats, which people seem to hit when the configure up
"redundant links" between servers and clients. Woodley hit one of them.
If the switch doesn't know what you're up to it'll start seeing packets from
the same MAC on two different interfaces and get Slightly Confused (and to
understand why you need to learn how switches build their forwarding tables
based on where they hear traffic from!); which is why most OSes that do
this kind of "teaming" without doing it via "link aggregation" (802.3ad,
etherchannel, etc.) do what Woodley found - same MAC across all interfaces.

Then there's the algorithm for how it figures out which interfaces to send
traffic out. Read up about it on that Bonding page, then google for "cisco
etherchannel" and read up about that too. In essence it'll choose the
transmit link based on some property of the message - generally a hash
based on source and or destination information. This generally means, at least
for Cisco devices, traffic between a given source/destination always flows
over one link but various other options can mitigate that (eg hashing based
on destination IP -and- source -PORT- for TCP/UDP traffic.)

Doing round-robin is absolutely stupid. Reason - you could end up with out
of order packets and (a) destroy performance, (b) fuck with any kind of
debugging.

The way to get around it really is to configure up proper 802.3ad between
switch and client and be done with it. Everything else gets "funny" from
time to time. I see people who configure up teaming under Windows without
telling the switch the ports are bonded and it makes me cringe every freaking
time.

HTH,


Adrian

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level bandwidth-capped VPSes available in WA -