[plug] gigabit lan

[Brad Campbell]

Adrian Chadd wrote:
> The trouble is this. If you've got a gige chipset on a junk PCI 32 bit/33mhz
> bus then your max transfer rate won't ever hit 1gig. With my testing I've
> reached ~350mbit full duplex (so 350mbit in and out) using 1518 byte frames
> (ie, not jumbo frames.)

I get about 350 between the two athlons.. it was getting about 270 from the Mac which is far more 
up-to-date hardware I was grumbling about.

I tried jumbo frames for a while but I have a feeling my switch was choking on them. I never got 
around to testing further as I was in a hurry to get the system working again so I just dropped it 
back to defaults.

> But then I did all the right stuff - disabled delayed ack/sack, wound my
> send/receive buffers right the heck up, tuned the polling parameters, etc.
> The defaults aren't always great for gige LAN networking.

I just left them all default.. I still get about 350 between the two athlon boxes with cheap-o PCI 
NIC's.

> Of course, if you have a junk board that has all the PCI slots and the NIC
> on the same 32-bit 33mhz PCI bus and you're doing lots of disk IO then
> you're halving (at least) the throughput - as the same bus needs to shuffle
> disk IO and then shuffle it back out the network port.

And given the server has 15 250GB SATA drives in it all on the same PCI bus.. well, don't need to 
say more. The throughput tests were done using iperf while the disk subsystem was relatively idle. 
Perhaps I need to check it again while it's doing a RAID-6 rebuild :)

> Thats why I love PCIe. No shared busses, right? :)

Indeed :)

One of these days I'll build up a new system to replace the server. But then I need 15 SATA ports, 
and they still tend to be hard to come by without forking out for managed SATA cards of some sort.
When they sort out the Marvell driver I might splurge for a pair of Supermicro 8-port cards and 
finally upgrade the system.

30 drives across a pair of Athlon based servers is a bit of a duct-tape and chewing gum solution, 
but it's been a) very cheap, and b) incredibly reliable all things considered (like one motherboard 
is held in place with old plastic RAM carriers and double-sided tape).

-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams