Date: Fri, 26 Jul 2013 15:14:17 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: Daniel Feenberg <feenberg@nber.org>, "Alexander V. Chernikov" <melifaro@FreeBSD.org> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: Recommendations for 10gbps NIC Message-ID: <1374876857.42890.YahooMailNeo@web121603.mail.ne1.yahoo.com> In-Reply-To: <alpine.LRH.2.03.1307261646050.25254@nber.org> References: <51F0386D.2000709@acm.poly.edu> <51F16A07.9030505@FreeBSD.org> <1374852658.90079.YahooMailNeo@web121601.mail.ne1.yahoo.com> <51F2A313.9070105@FreeBSD.org> <alpine.LRH.2.03.1307261646050.25254@nber.org>
next in thread | previous in thread | raw e-mail | index | archive | help
=0A=0A=0A=0A________________________________=0A From: Daniel Feenberg <feen= berg@nber.org>=0ATo: Alexander V. Chernikov <melifaro@FreeBSD.org> =0ACc: B= arney Cordoba <barney_cordoba@yahoo.com>; "freebsd-net@freebsd.org" <freebs= d-net@freebsd.org> =0ASent: Friday, July 26, 2013 4:59 PM=0ASubject: Re: Re= commendations for 10gbps NIC=0A =0A=0A=0AOn Fri, 26 Jul 2013, Alexander V. = Chernikov wrote:=0A=0A> On 26.07.2013 19:30, Barney Cordoba wrote:=0A>> =0A= >> =0A>> ------------------------------------------------------------------= ------=0A>> *From:* Alexander V. Chernikov <melifaro@FreeBSD.org>=0A>> *To:= * Boris Kochergin <spawk@acm.poly.edu>=0A>> *Cc:* freebsd-net@freebsd.org= =0A>> *Sent:* Thursday, July 25, 2013 2:10 PM=0A>> *Subject:* Re: Recommend= ations for 10gbps NIC=0A>> =0A>> On 25.07.2013 00:26, Boris Kochergin wrote= :=0A>> > Hi.=0A>> Hello.=0A>> >=0A>> > I am looking for recommendations for= a 10gbps NIC from someone who has=0A>> > successfully used it on FreeBSD. = It will be used on FreeBSD 9.1-R/amd64=0A>> > to capture packets. Some desi= red features are:=0A>> >=0A=0AWe have experience with HP NC523SFP and Chels= io N320E. The key difference =0Aamong 10GBE cards for us is how they treat = foreign DACs. The HP would PXE =0Aboot with several brands and generic DACs= , but the Chelsio required a =0AChelsio brand DAC to PXE boot.=A0 There was= firmware on the NIC to check the =0Abrand of cable. Both worked fine once = booted. The Chelsio cables were hard =0Ato find, which became a problem. Al= so, when used with diskless Unix =0Aclients the Chelsio cards seemed to han= g from time to time. Otherwise =0Apacket loss was one in a million for both= cards, even with 7 meter cables.=0A=0AWe liked the fact that the Chelsio c= ards were single-port and cheaper. I =0Adon't really understand why nearly = all 10GBE cards are dual-port. Surely =0Athere is a market for NICs between= 1 gigabit and 20 gigabit.=0A=0AThe NIC heatsinks are too hot to touch duri= ng use unless specially cooled.=0A=0ADaniel Feenberg=0ANBER=0A=0A=0A=0A----= -----------------=0AThe same reason that they don't make single core cpus a= nymore. It costs about the=0Asame to make a 1 port chip as a 2 port chip.= =0A=0AI find it interesting =A0how so many talk about "the cards", when mos= t often the=0Adifferences are with "the drivers". Luigi made the most usefu= l comment; if you ever=0Awant to use netmap, =A0you need to buy a card comp= atible with netmap. Although=0Ayou don't =A0need netmap just to capture 10G= b/s. Forwarding, Maybe.=A0=0A=0AI also find it interesting that nobody seem= s to have a handle on the performance=0Adifferences. Obviously they're all = different. Maybe substantially different.=0A=0AThe x540 with RJ45 has the o= bvious advantage of being compatible with regular gigabit cards,=A0=0Aand s= ingle port adapters are about $325 in the US.=A0=0A=0AWhen cheap(er) 10g RJ= 45 switches become available, it will start to be used more and more.=0AVer= y soon.=0A=0ABC=0A From owner-freebsd-net@FreeBSD.ORG Sat Jul 27 08:02:40 2013 Return-Path: <owner-freebsd-net@FreeBSD.ORG> Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B36AA2FA for <freebsd-net@freebsd.org>; Sat, 27 Jul 2013 08:02:40 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from mail.ipfw.ru (unknown [IPv6:2a01:4f8:120:6141::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 742A92BAC for <freebsd-net@freebsd.org>; Sat, 27 Jul 2013 08:02:40 +0000 (UTC) Received: from v6.mpls.in ([2a02:978:2::5] helo=ws.su29.net) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from <melifaro@FreeBSD.org>) id 1V2zVc-000CyT-8n; Sat, 27 Jul 2013 12:05:48 +0400 Message-ID: <51F37E97.3090203@FreeBSD.org> Date: Sat, 27 Jul 2013 12:02:31 +0400 From: "Alexander V. Chernikov" <melifaro@FreeBSD.org> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120121 Thunderbird/9.0 MIME-Version: 1.0 To: Barney Cordoba <barney_cordoba@yahoo.com> Subject: Re: Recommendations for 10gbps NIC References: <51F0386D.2000709@acm.poly.edu> <51F16A07.9030505@FreeBSD.org> <1374852658.90079.YahooMailNeo@web121601.mail.ne1.yahoo.com> <51F2A313.9070105@FreeBSD.org> <alpine.LRH.2.03.1307261646050.25254@nber.org> <1374876857.42890.YahooMailNeo@web121603.mail.ne1.yahoo.com> In-Reply-To: <1374876857.42890.YahooMailNeo@web121603.mail.ne1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, Daniel Feenberg <feenberg@nber.org> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>; List-Post: <mailto:freebsd-net@freebsd.org> List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sat, 27 Jul 2013 08:02:40 -0000 On 27.07.2013 02:14, Barney Cordoba wrote: > > > ------------------------------------------------------------------------ > *From:* Daniel Feenberg <feenberg@nber.org> > *To:* Alexander V. Chernikov <melifaro@FreeBSD.org> > *Cc:* Barney Cordoba <barney_cordoba@yahoo.com>; > "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> > *Sent:* Friday, July 26, 2013 4:59 PM > *Subject:* Re: Recommendations for 10gbps NIC > > > On Fri, 26 Jul 2013, Alexander V. Chernikov wrote: > > > On 26.07.2013 19:30, Barney Cordoba wrote: > >> > >> > >> ------------------------------------------------------------------------ > >> *From:* Alexander V. Chernikov <melifaro@FreeBSD.org > <mailto:melifaro@FreeBSD.org>> > >> *To:* Boris Kochergin <spawk@acm.poly.edu <mailto:spawk@acm.poly.edu>> > >> *Cc:* freebsd-net@freebsd.org <mailto:freebsd-net@freebsd.org> > >> *Sent:* Thursday, July 25, 2013 2:10 PM > >> *Subject:* Re: Recommendations for 10gbps NIC > >> > >> On 25.07.2013 00:26, Boris Kochergin wrote: > >> > Hi. > >> Hello. > >> > > >> > I am looking for recommendations for a 10gbps NIC from someone who has > >> > successfully used it on FreeBSD. It will be used on FreeBSD > 9.1-R/amd64 > >> > to capture packets. Some desired features are: > >> > > > We have experience with HP NC523SFP and Chelsio N320E. The key difference > among 10GBE cards for us is how they treat foreign DACs. The HP would PXE > boot with several brands and generic DACs, but the Chelsio required a > Chelsio brand DAC to PXE boot. There was firmware on the NIC to check the > brand of cable. Both worked fine once booted. The Chelsio cables were hard > to find, which became a problem. Also, when used with diskless Unix > clients the Chelsio cards seemed to hang from time to time. Otherwise > packet loss was one in a million for both cards, even with 7 meter cables. > > We liked the fact that the Chelsio cards were single-port and cheaper. I > don't really understand why nearly all 10GBE cards are dual-port. Surely > there is a market for NICs between 1 gigabit and 20 gigabit. > > The NIC heatsinks are too hot to touch during use unless specially cooled. > > Daniel Feenberg > NBER > > > --------------------- > The same reason that they don't make single core cpus anymore. It costs > about the > same to make a 1 port chip as a 2 port chip. > > I find it interesting how so many talk about "the cards", when most > often the > differences are with "the drivers". Luigi made the most useful comment; > if you ever > want to use netmap, you need to buy a card compatible with netmap. Although > you don't need netmap just to capture 10Gb/s. Forwarding, Maybe. > > I also find it interesting that nobody seems to have a handle on the > performance > differences. Obviously they're all different. Maybe substantially different. It depends on what kind of performance you are talking about. All NICs are capable of doing linerate RX/TX for both small/big packets. The only notable exception I;m aware of are Intel 82598-based NICs which advertise PCI-E X8 gen2 with _2.5GT_ link speed, giving you maximum ~14Gbit/s bw for 2 ports instead of 20. > > The x540 with RJ45 has the obvious advantage of being compatible with > regular gigabit cards, > and single port adapters are about $325 in the US. > > When cheap(er) 10g RJ45 switches become available, it will start to be > used more and more. > Very soon. > > BC >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1374876857.42890.YahooMailNeo>