Date: Sat, 1 Mar 2008 16:39:57 -0800 (PST) From: Barney Cordoba <barney_cordoba@yahoo.com> To: Erik Trulsson <ertr1013@student.uu.se> Cc: net@freebsd.org Subject: Re: FBSD 1GBit router? Message-ID: <638970.20021.qm@web63906.mail.re1.yahoo.com> In-Reply-To: <20080301225727.GA85851@owl.midgard.homeip.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--- Erik Trulsson <ertr1013@student.uu.se> wrote:
> On Sat, Mar 01, 2008 at 01:27:46PM -0800, Barney
> Cordoba wrote:
> >
> > --- Ingo Flaschberger <if@xip.at> wrote:
> >
> > > Dear Barney,
> > >
> > > > It seems absolutely ridiculous to buy such
> > > hardware
> > > > and not install a PCIx or 4x PCIe card for
> another
> > > > $100. or less. Saying a 1x is "fast enough" is
> > > like
> > > > saying a Celeron is "fast enough".
> > >
> > > The box is a small 1HE appliance and can boot
> from a
> > > CF-Card.
> > > I trust them more than a "al cheapo" pc.
> > > 1x axiomtek NA-820
> > > 1x P4 3Ghz cpu
> > > 1x 1gb ddr2
> > > ---
> > > 850eur without taxes.
> > >
> > > A good chipset, good cpu, good ram, good
> harddisk,
> > > god powersupply has
> > > same price.
> > > And don't forget that in exchanges you pay for
> each
> > > HE.
> > >
> > > And back to 1x is not fast enough:
> > > There are no 1gbit single port network cards
> that
> > > support more than 1
> > > lane, even if you plug it into a 16 lane slot.
> > > (and I'm not talking about 10gbit cards; if you
> have
> > > 10gbit upstream you
> > > have enough $$ to buy good gear)
> >
> > Ok, well I've never seen a router with 1 port. I
> > thought we were talking about building a router?
>
> He did not say anything about a single port router.
> He talked about single port network cards. You can
> use more than one of them when building a router.
His argument is that there are only 1x PCIe cards that
have 1 port. Since he needs 2 ports, and there are 2
port
PCIe cards, then his argument makes no sense. But the
point is that PCIe NICs are implemented with 1 port
per lane in the chip. So a 2 port card will use 2
lanes
>
> >
> > The lack of PCIe cards is a good reason to
> consider a
> > PCIX machine.
>
> What lack of PCI-E cards? These days there are
> quite a
> few to choose between.
Yes, but they are all 1x, while there are many 1 and 2
port PCIx cards which are twice as fast.
>
> > On the systems that we have, the 1x PCIe
> > ports are a lot slower than a PCI-X card in the
> slot.
> >
> > You need 4Gb/s of throughput to handle a gigablt
> > router. (1 GB/s full duplex times 2). 1x is 4Gb/s
> > maximum. In my view, you always need twice the
> > bandwidth on the bus to avoid contention issues.
>
> What contention issues? With PCI-E each device is
> essentially on its own
> bus and does not need to contend with other devices
> for bandwidth on that
> bus.
>
> A PCI-E 1x connection provides more bandwidth than
> one gigabit ethernet
> connection can use.
Does each PCIe slot have its own dedicated memory
controller? The concept that there is some sort of
mutually exclusive, independent path for each
controller is simply not the case in practice. You're
accessing the same memory; you're going through
shared hubs and bridges. You're doing I/O on the same
bus. North bridges typically have a 512 Byte payload
maximum, so you can't even burst a full packet. You
have transaction overhead for each transfer. There are
many factors that will chip away at your realizable
bandwidth. Its not like a hose gushing a continuos
stream of water.
Another factor is that "server" chipsets do PCIe
better than "desktop" chipsets. Server chipsets are
optimized in the chipset to handle multiple devices
more efficiently. So don't expect your "desktop"
chipset to be as efficient as a server chipset at the
same task. There's a reason that intel has desktop and
server chipsets.
I've tested dual port cards based on the 82546 and
82571 parts on the same system that has both PCIX and
PCIe slots. The parts are essentially the same, and
the driver is essentially the same, with the bus being
the only real difference. The PCIe card is a 2x card.
so its the equivalent of 2 PCIe cards with dedicated
1x lanes. The results are the that PCIx card is simply
more efficient, in that is uses less CPU with the same
load (an indication of less contention for I/O), and
the PCIX card has a higher capacity (that is, the
point in which the cpu is saturated is higher by about
10%)
In practice your little box is never going to be
routing bi-direction, full gigabit traffic, but if you
only have "just enough" bandwidth, your CPU is going
to get less and less efficient as you get higher and
higher usage. You'll need more cpu to do the same task
with less bus.
I'm not claiming that PCIX is faster than pcie at the
same "speed", but that the limitation of 1x per NIC
causes PCIX to be a better choice for a 2 port system
in most cases.
Barney
____________________________________________________________________________________
Looking for last minute shopping deals?
Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?638970.20021.qm>