[plug] gigabit switch recommendations

[Adon Metcalfe]

heh yeah LACP/balance are more trouble than their worth

http://streakwave.com.au/store/ubiquiti-networks/edgeswitch/es-24-250w-edgeswitch-24-gigabit-ports-af-at-24v-poe-250-watt-rackmount

their pretty cheap btw, not quite sub 500 tho

On 17 November 2014 20:02, Patrick Coleman <blinken at gmail.com> wrote:

> Hey Paul,
>
> It's been a while since I configured this, but I wanted to elaborate
> on what Brad was saying before you go out and spend any money -
>
> On Mon, Nov 17, 2014 at 10:55 AM, Paul Del <p at delfante.it> wrote:
> > I kept it pretty simple because as long as it's faster than a single
> port I
> > would be happy
> > If I can get the speed of 2x gigabit ports that would be excellent
>
> You won't get this with balance-rr and a cheap switch. balance-rr != LACP:
>
> - balance-rr just blats _outgoing_ packets from the server round-robin
> over the bonded link; LACP is not used. For 2x1Gb/s links you will
> achieve 2Gb/s _outbound_; the switch will be terribly confused by this
> as it will see the same MAC on several interfaces, and cheaper models
> will probably just dump all traffic destined for the server down one
> of the bonded links only. Even cheaper switches (and some expensive
> ones) will get really confused and you'll start losing packets.
>
> On a decent managed switch you should be able to configure some sort
> of algorithm based on a hash of the source/destination and TCP ports
> so (for traffic towards the server) each _TCP session_ is switched
> down a different bonded link.
>
> You will need to configure this manually on both the server and (if
> you require inbound traffic balancing) the switch. The key advantage
> to balance-rr however is that the switch doesn't have to be involved;
> if you only require outbound traffic balancing it sometimes works well
> even with unmanaged switches.
>
> If a link fails and one side doesn't notice - the classic case being a
> fibre media converter between the two ends - traffic will continue to
> be blatted out that interface and every second outbound packet will be
> lost.
>
> This gets you 2Gb/s outbound from the server, and at most 1Gb/s
> inbound for a given TCP session (eg. file download from a NAS, but
> you'll be able to run 2x 1Gb/s downloads simultaneously).
>
> - LACP is the Link Aggregation Control Protocol, key word here being
> protocol. The idea here is that you put the Linux bonding driver into
> 802.3ad mode[1] and then it sends discovery packets down the bonded
> interfaces. The managed switch on the other end detects this, and if
> LACP is enabled an aggregate link will be negotiated between the
> server and the switch. You don't need to manually specify the
> interfaces on the switch end, and if one link fails both sides will
> agree to not transmit data down that link. This is convenient if
> you're cleaning up the server room and need to reroute cables (I kid,
> I kid, that's what active/backup application failover was invented
> for).
>
> The algorithm on both ends is generally a hash on the
> source/destination and TCP ports as above, so in both directions you
> will get 1Gb/s for any given TCP session, but if you have several TCP
> sessions they will in most cases be balanced across both links.
>
> So, the key point here is that the only thing that will get you 2Gb/s
> in every case is a 10Gb/s network adapter :) LACP, balance-rr etc were
> designed for a server with several hundred clients - with a few
> hundred un-natted TCP connections LACP will (generally) balance
> traffic nicely across a pair of 1Gb/s links in both directions (lots
> of qualifiers in that sentence ;).
>
> I wasn't kidding on the 10Gb/s adapter front either - a pair of PCIe
> adapters (one for your desktop, one for the NAS) + transceivers + a
> fibre lead may well come in under $500 if you shop around[2].
>
> Cheers,
>
> Patrick
>
>
> 1. https://www.kernel.org/doc/Documentation/networking/bonding.txt
> 2. http://www.ebay.com/bhp/10gb-network-card
> _______________________________________________
> PLUG discussion list: plug at plug.org.au
> http://lists.plug.org.au/mailman/listinfo/plug
> Committee e-mail: committee at plug.org.au
> PLUG Membership: http://www.plug.org.au/membership
>



-- 
Regards,

Adon Metcalfe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.plug.org.au/pipermail/plug/attachments/20141118/741a4069/attachment.html>