Date: Sat, 31 Aug 2013 05:41:53 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: Luigi Rizzo <rizzo@iet.unipi.it>, Alan Somers <asomers@freebsd.org> Cc: Jack F Vogel <jfv@freebsd.org>, "T.C. Gubatayao" <tgubatayao@barracuda.com>, "Justin T. Gibbs" <gibbs@freebsd.org>, Andre Oppermann <andre@freebsd.org>, "net@freebsd.org" <net@freebsd.org> Subject: Re: Flow ID, LACP, and igb Message-ID: <1377952913.44129.YahooMailNeo@web121605.mail.ne1.yahoo.com> In-Reply-To: <CA%2BhQ2%2BhgTaK1ZCOLGVFjSPY8nyNPHK4waSecyRQxR1gQcyjztg@mail.gmail.com> References: <D01A0CB2-B1E3-4F4B-97FA-4C821C0E3FD2@FreeBSD.org> <521BBD21.4070304@freebsd.org> <CAOtMX2jvKGY==t9i-a_8RtMAPH2p1VDj950nMHHouryoz3nbsA@mail.gmail.com> <521EE8DA.3060107@freebsd.org> <BCC2C62D4FE171479E2F1C2593FE508B0BE24383@BN-SCL-MBX03.Cudanet.local> <CAOtMX2h5SGh5eYV50y%2BQB_s367V9iattGU862wwXcONDV%2BTG8g@mail.gmail.com> <CA%2BhQ2%2BhgTaK1ZCOLGVFjSPY8nyNPHK4waSecyRQxR1gQcyjztg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
May I express my glee and astonishment that =A0you're debating the use of c= omplicated hash functions=0Afor something that's likely to have from 2-8 sl= ots?=0A=0AAlso, the *most* important thing is distribution with realistic d= ata. The goal should be to use the=0Amost trivial function that gives the m= ost balanced distribution with real numbers. Faster is=0Anot better if the = result is an unbalanced distribution.=0A=0AMany of your ports will be 80 an= d 53, and if you're going through a router your ethernets=0Amay not be very= unique, so why even bother to include them? Does getting a good distributi= on=0Arequire that you hash every element individually, or can you get the s= ame distribution with=0Aa faster, simpler way of creating the seed?=0A=0ATh= ere's also the other consideration of packet size. Packets on port 53 are l= ikely to be smaller=0Athan packets on port 80. What you want is equal distr= ibution PER PORT on the ports that will=0Acarry that vast majority of your = traffic.=0A=0AWhen designing efficient systems, you must not assume that po= rts and IPs are random, because they're=0Anot. 99% of your load will be on = a small number of destination ports and a limited range of source ports.=0A= =0AFor a web server application, geting a perfect distribution on the http = ports is most crucial.=0A=0AThe hash function in if_lagg.c looks like more = of a classroom exercise than a practical implementation.=A0=0AIf you're goi= ng to consider 100M iterations; consider that much of the time is wasted pa= rsing the=0Apacket (again). Why not add a simple sysctl that enables a hash= that is created in the ip parser,=0Awhen all of the pieces are available w= ithout having to re-parse the mbuf?=0A=0AOr better yet, use the same number= of queues on igb as you have LAGG ports, and use the queue id (or RSS)=0Aa= s the hash, so that=A0your traffic is sync'd between the ethernet adapter q= ueues and the LAGG ports. The card=0Ahas already done the work for you.=0A= =0ABC=0A=0A=0A=0A=0A=0A________________________________=0A From: Luigi Rizz= o <rizzo@iet.unipi.it>=0ATo: Alan Somers <asomers@freebsd.org> =0ACc: Jack = F Vogel <jfv@freebsd.org>; "net@freebsd.org" <net@freebsd.org>; Justin T. G= ibbs <gibbs@freebsd.org>; Andre Oppermann <andre@freebsd.org>; T.C. Gubatay= ao <tgubatayao@barracuda.com> =0ASent: Friday, August 30, 2013 8:04 PM=0ASu= bject: Re: Flow ID, LACP, and igb=0A =0A=0AAlan,=0A=0A=0AOn Thu, Aug 29, 20= 13 at 6:45 PM, Alan Somers <asomers@freebsd.org> wrote:=0A>=0A>=0A> ...=0A>= I pulled all four hash functions out into userland and microbenchmarked=0A= > them.=A0 The upshot is that hash32 and fnv_hash are the fastest, jenkins_= hash=0A> is slower, and siphash24 is the slowest.=A0 Also, Clang resulted i= n much=0A> faster code than gcc.=0A>=0A>=0Ai missed this part of your messa= ge, but if i read your code well,=0Ayou are running 100M iterations and the= numbers below are in seconds,=0Aso if you multiply the numbers by 10 you h= ave the cost per hash in=0Ananoseconds.=0A=0AWhat CPU did you use for your = tests ?=0A=0AAlso some of the numbers (FNV and hash32) are suspiciously low= .=0A=0AI believe that the compiler (both of them) have figure out that ever= ything=0Ais constant in these functions, and fnv_32_buf() and hash32_buf() = are=0Ainline,=0Ahence they can be optimized to just return a constant.=0ATh= is does not happen for siphash and jenkins because they are defined=0Aexter= nally.=0A=0ACan you please re-run the tests in a way that defeats the optim= ization ?=0A(e.g. pass a non constant argument to the the hashes so you act= ually need=0Ato run the code).=0A=0Acheers=0Aluigi=0A=0A=0Ahttp://people.fr= eebsd.org/~asomers/lagg_hash/=0A>=0A> [root@sm4u-4 /usr/home/alans/ctest/la= gg_hash]# ./lagg_hash-gcc-4.8=0A> FNV: 0.76=0A> hash32: 1.18=0A> SipHash24:= 44.39=0A> Jenkins: 6.20=0A> [root@sm4u-4 /usr/home/alans/ctest/lagg_hash]#= ./lagg_hash-gcc-4.2.1=0A> FNV: 0.74=0A> hash32: 1.35=0A> SipHash24: 55.25= =0A> Jenkins: 7.37=0A> [root@sm4u-4 /usr/home/alans/ctest/lagg_hash]# ./lag= g_hash.clang-3.3=0A> FNV: 0.30=0A> hash32: 0.30=0A> SipHash24: 55.97=0A> Je= nkins: 6.45=0A> [root@sm4u-4 /usr/home/alans/ctest/lagg_hash]# ./lagg_hash.= clang-3.2=0A> FNV: 0.30=0A> hash32: 0.30=0A> SipHash24: 44.52=0A> Jenkins: = 6.48=0A>=0A>=0A>=0A> > T.C.=0A> >=0A> > [1]=0A> >=0A> http://svnweb.freebsd= .org/base/head/sys/libkern/jenkins_hash.c?view=3Dmarkup=0A> _______________= ________________________________=0A> freebsd-net@freebsd.org mailing list= =0A> http://lists.freebsd.org/mailman/listinfo/freebsd-net=0A>; To unsubscri= be, send any mail to "freebsd-net-unsubscribe@freebsd.org"=0A>=0A__________= _____________________________________=0Afreebsd-net@freebsd.org mailing lis= t=0Ahttp://lists.freebsd.org/mailman/listinfo/freebsd-net=0ATo unsubscribe,= send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sat Aug 31 14:50:37 2013 Return-Path: <owner-freebsd-net@FreeBSD.ORG> Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6E9D6602 for <net@freebsd.org>; Sat, 31 Aug 2013 14:50:37 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm13-vm5.bullet.mail.ne1.yahoo.com (nm13-vm5.bullet.mail.ne1.yahoo.com [98.138.91.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2F39F25BB for <net@freebsd.org>; Sat, 31 Aug 2013 14:50:36 +0000 (UTC) Received: from [98.138.226.179] by nm13.bullet.mail.ne1.yahoo.com with NNFMP; 31 Aug 2013 14:47:48 -0000 Received: from [98.138.88.239] by tm14.bullet.mail.ne1.yahoo.com with NNFMP; 31 Aug 2013 14:47:48 -0000 Received: from [127.0.0.1] by omp1039.mail.ne1.yahoo.com with NNFMP; 31 Aug 2013 14:47:48 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 11055.63437.bm@omp1039.mail.ne1.yahoo.com Received: (qmail 1521 invoked by uid 60001); 31 Aug 2013 14:47:47 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1377960467; bh=91z46d+R+3onCaet/dXTIaEVGu76b3VAiwEAhziMStI=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=0vc/X0MuzCRm0Pe3apvKdA2M40v+jeAPakoMKmmcezHv4FpA418BfWX8f+0wN38qOI4EqLPw6/oV8V4zpwR/xgR3okQxQmu0/px1FIKnh1I/2LFr0QOf9xMW7oVgY798D0dIeej4QDaNblx3fUGULgR08Scm7NbHlplxEz7/aTw= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=eO+dtiWX9Ukfsx7CmNIEay2MFKeHjU2zVSxXsYMoM+dRwhtXLC3+tNi2x4AsW1gNjhx2wOnj4B2clfyjNVCI4mQcTdra1bGkWlQNeDvxahig6Ewwyq7/exkb4McIoRBnaRsPr13xRiMsB4sA4walhmCv5fX5gBLI94AZgsxMarQ=; X-YMail-OSG: upHBLR4VM1kga1rhHiwyEXJI5MEDL17tFUKdbT7XFWh6rKD dTA_nZG9a2OdVZW7dGnNzW64q0AiCITb7GjphParmIq7qWUnw8irC60HvmYV Pja3Q1CbuYY1C_hNWkYeO.D1dHsFgxyWyYG7Znx5WHP7Sbqk9TsPnulk3CzX 8CznhUBzvNYg7iq4Dr.gLK3DX9NOBv8mmBS48EMApkNdpV6aJx91DwG1zn_N WMOIN7gnheymnx357Cv1suudwm8In0UzSJM07f9URCEGXwTrE4YTi.5cY4m6 .QVarhLcpZ556bfIQpHW8jq_HKi6ybnKdjP3_14MyeQpquQI.oVasgGsfleK toRbHI9rHfwNwq9IcZoWpYUv2A3KVU07gWLQSDZgxQA5zF95LcyZGGIvNTTi bCuXu9lHalBSwptou_BZl5jtTtG_VErF.ESYkmIi5gkEZqCuEejxLbQNZKd6 R77O5LtTtitak5EkfNc7rBaKPJl2Lqx0XQUCrT8LD1RV5JX5JrSaCNbJ7x_h S0x8h9.Jt6iWCaiV5VX86AXe4dr.U6Hx1GCg0bQaNVBko29oNALkTyyPU_E0 - Received: from [98.203.118.124] by web121603.mail.ne1.yahoo.com via HTTP; Sat, 31 Aug 2013 07:47:47 PDT X-Rocket-MIMEInfo: 002.001, QW5kIGFub3RoZXIgdGhpbmc7IHRoZSB1c2Ugb2YgbW9kdWxvIGlzIHZlcnkgZXhwZW5zaXZlIHdoZW4gdGhlIG51bWJlciBvZiBwb3J0cwp1c2VkIGluIExBR0cgaXMgKnVzdWFsbHkqIGEgcG93ZXIgb2YgMi4gZm9vJihTTE9UUy0xKSBpcyBhIGxvdCBmYXN0ZXIgdGhhbiAoZm9vJVNMT1RTKS7CoAoKaWYgKFNMT1RTID09IDIgfHwgU0xPVFMgPT0gNCB8fCBTTE9UUyA9PSA4KQrCoCDCoCBoYXNoID0gaGFzaCYoU0xPVFMtMSk7CmVsc2UKwqAgwqAgaGFzaCA9IGhhc2ggJSBTTE9UUzsKCmlzIG1vcmUgdGhhbiABMAEBAQE- X-Mailer: YahooMailWebService/0.8.156.576 References: <D01A0CB2-B1E3-4F4B-97FA-4C821C0E3FD2@FreeBSD.org> <521BBD21.4070304@freebsd.org> <CAOtMX2jvKGY==t9i-a_8RtMAPH2p1VDj950nMHHouryoz3nbsA@mail.gmail.com> <521EE8DA.3060107@freebsd.org> <BCC2C62D4FE171479E2F1C2593FE508B0BE24383@BN-SCL-MBX03.Cudanet.local> <CAOtMX2h5SGh5eYV50y+QB_s367V9iattGU862wwXcONDV+TG8g@mail.gmail.com> <CA+hQ2+hgTaK1ZCOLGVFjSPY8nyNPHK4waSecyRQxR1gQcyjztg@mail.gmail.com> Message-ID: <1377960467.93173.YahooMailNeo@web121603.mail.ne1.yahoo.com> Date: Sat, 31 Aug 2013 07:47:47 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> Subject: Re: Flow ID, LACP, and igb To: Luigi Rizzo <rizzo@iet.unipi.it>, Alan Somers <asomers@freebsd.org> In-Reply-To: <CA+hQ2+hgTaK1ZCOLGVFjSPY8nyNPHK4waSecyRQxR1gQcyjztg@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Jack F Vogel <jfv@freebsd.org>, "T.C. Gubatayao" <tgubatayao@barracuda.com>, "Justin T. Gibbs" <gibbs@freebsd.org>, Andre Oppermann <andre@freebsd.org>, "net@freebsd.org" <net@freebsd.org> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Barney Cordoba <barney_cordoba@yahoo.com> List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>; List-Post: <mailto:freebsd-net@freebsd.org> List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sat, 31 Aug 2013 14:50:37 -0000 And another thing; the use of modulo is very expensive when the number of p= orts=0Aused in LAGG is *usually* a power of 2. foo&(SLOTS-1) is a lot faste= r than (foo%SLOTS).=A0=0A=0Aif (SLOTS =3D=3D 2 || SLOTS =3D=3D 4 || SLOTS = =3D=3D 8)=0A=A0 =A0 hash =3D hash&(SLOTS-1);=0Aelse=0A=A0 =A0 hash =3D hash= % SLOTS;=0A=0Ais more than twice as fast as=A0=0A=0Ahash % SLOTS;=0A=0ABC= =0A=0A=0A________________________________=0A From: Luigi Rizzo <rizzo@iet.u= nipi.it>=0ATo: Alan Somers <asomers@freebsd.org> =0ACc: Jack F Vogel <jfv@f= reebsd.org>; "net@freebsd.org" <net@freebsd.org>; Justin T. Gibbs <gibbs@fr= eebsd.org>; Andre Oppermann <andre@freebsd.org>; T.C. Gubatayao <tgubatayao= @barracuda.com> =0ASent: Friday, August 30, 2013 8:04 PM=0ASubject: Re: Flo= w ID, LACP, and igb=0A =0A=0AAlan,=0A=0A=0AOn Thu, Aug 29, 2013 at 6:45 PM,= Alan Somers <asomers@freebsd.org> wrote:=0A>=0A>=0A> ...=0A> I pulled all = four hash functions out into userland and microbenchmarked=0A> them.=A0 The= upshot is that hash32 and fnv_hash are the fastest, jenkins_hash=0A> is sl= ower, and siphash24 is the slowest.=A0 Also, Clang resulted in much=0A> fas= ter code than gcc.=0A>=0A>=0Ai missed this part of your message, but if i r= ead your code well,=0Ayou are running 100M iterations and the numbers below= are in seconds,=0Aso if you multiply the numbers by 10 you have the cost p= er hash in=0Ananoseconds.=0A=0AWhat CPU did you use for your tests ?=0A=0AA= lso some of the numbers (FNV and hash32) are suspiciously low.=0A=0AI=0A be= lieve that the compiler (both of them) have figure out that everything=0Ais= constant in these functions, and fnv_32_buf() and hash32_buf() are=0Ainlin= e,=0Ahence they can be optimized to just return a constant.=0AThis does not= happen for siphash and jenkins because they are defined=0Aexternally.=0A= =0ACan you please re-run the tests in a way that defeats the optimization ?= =0A(e.g. pass a non constant argument to the the hashes so you actually nee= d=0Ato run the code).=0A=0Acheers=0Aluigi=0A=0A=0Ahttp://people.freebsd.org= /~asomers/lagg_hash/=0A>=0A> [root@sm4u-4 /usr/home/alans/ctest/lagg_hash]#= ./lagg_hash-gcc-4.8=0A> FNV: 0.76=0A> hash32: 1.18=0A> SipHash24: 44.39=0A= > Jenkins: 6.20=0A> [root@sm4u-4 /usr/home/alans/ctest/lagg_hash]# ./lagg_h= ash-gcc-4.2.1=0A> FNV: 0.74=0A> hash32: 1.35=0A> SipHash24: 55.25=0A> Jenki= ns: 7.37=0A> [root@sm4u-4 /usr/home/alans/ctest/lagg_hash]# ./lagg_hash.cla= ng-3.3=0A> FNV: 0.30=0A> hash32: 0.30=0A> SipHash24: 55.97=0A> Jenkins: 6.4= 5=0A> [root@sm4u-4 /usr/home/alans/ctest/lagg_hash]# ./lagg_hash.clang-3.2= =0A> FNV: 0.30=0A> hash32: 0.30=0A> SipHash24: 44.52=0A> Jenkins: 6.48=0A>= =0A>=0A>=0A> > T.C.=0A> >=0A> > [1]=0A> >=0A> http://svnweb.freebsd.org/bas= e/head/sys/libkern/jenkins_hash.c?view=3Dmarkup=0A> _______________________= ________________________=0A> freebsd-net@freebsd.org mailing list=0A> http:= //lists.freebsd.org/mailman/listinfo/freebsd-net=0A> To unsubscribe, send a= ny mail to "freebsd-net-unsubscribe@freebsd.org"=0A>=0A____________________= ___________________________=0Afreebsd-net@freebsd.org mailing list=0Ahttp:/= /lists.freebsd.org/mailman/listinfo/freebsd-net=0ATo unsubscribe, send any = mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Sat Aug 31 15:05:23 2013 Return-Path: <owner-freebsd-net@FreeBSD.ORG> Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 701BDB81 for <freebsd-net@freebsd.org>; Sat, 31 Aug 2013 15:05:23 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm21-vm4.bullet.mail.ne1.yahoo.com (nm21-vm4.bullet.mail.ne1.yahoo.com [98.138.91.181]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1EA6D2656 for <freebsd-net@freebsd.org>; Sat, 31 Aug 2013 15:05:22 +0000 (UTC) Received: from [98.138.101.128] by nm21.bullet.mail.ne1.yahoo.com with NNFMP; 31 Aug 2013 15:03:08 -0000 Received: from [98.138.89.248] by tm16.bullet.mail.ne1.yahoo.com with NNFMP; 31 Aug 2013 15:03:08 -0000 Received: from [127.0.0.1] by omp1040.mail.ne1.yahoo.com with NNFMP; 31 Aug 2013 15:03:08 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 307911.32973.bm@omp1040.mail.ne1.yahoo.com Received: (qmail 38024 invoked by uid 60001); 31 Aug 2013 15:03:08 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1377961388; bh=Y7fnNQoRaB9Mi4aMrvBp20u0Jchwx0rrwWWy0Jz3lMw=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=mYfGjAfYimzkoy0S2YCm47jExxgCxB9/sGt60fEAmtyGc88ohT79tDls79W6OI59UNLaSGxQDL2h89St8qo4G4b//1wlq9zAvjlPQjyvTkdhS45pNJmGLMLgyNBRdfXaq4b4MHK9bNZ9gwjavxPmO+jM6DQB8+cbgEdSuV2En9Y= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=ExHGsnoQn35p3cmBt8J5dbDx+EClbaAS6h12mTmHDbzdPQgbYqYkn0v6paHO16ot1K+rn9e5lK+y5iGaa/ieFrTdJ8aocOSVSWIt1eHfXGqCYGI/HY3VKDu4GK/wNsN+i6G5SoW/tuofW8Tf2kvmYFq3Jz28Xb/hltoSgd7qjfY=; X-YMail-OSG: ob_reFgVM1lDffF6LE7hWbGzHbMGfKgIk0e6GWQyyC3FBT9 noVa6TSxl70hWFu9H3ep9GCTM0XrY0v.r8xdn0a3f4Dtv_9LvAI9UWM2rd8g o2TGdzGUYgdULWWfaxS_OgR1YU0XUcYRQRcCYViVK.V_89x6EWMqmJxUCvUk xp_l1ocXjkdwEp.6G5huw8624Wv0jchbKkoOXTPytM63KMLpwPV4aE_quPtS 94zfqrMowraoCynekyrfSnoW5xugQvykV4VKgpXP3VKlgC3in0uhOwLvEXCI .FhKTLbDrMeI.Wlv5PIypHX6TgnPJ_h91Lr4vByOHCEAF2jjIp4FoXtOQvoV hRmDmAS_1V9mezXJazZPjFEpeH63Wdemqi1HZnHzQY3hftGqvB5tIJ8V1Cw4 KN2EVxQIbhceL.pJN5kV_hxBtvzpBfwU_BKC7evZqMA5IeKjrnq7b1r07IVJ zripu_ZMp0Bi7QyunKQoaBSV5y_fxYi0EEHZIG4PFsNa.79N8Ck1ijCWvLrV Nu34TTEk7KdtQtx6e6gS41Zuy.fcwVtbtnF69Wcb0tyTAOYHVez2BVHNwokC L.VsnOocD1zqLDPIvlyQnT6ZAKENkgfWce8Via6y8n99ibWbQj1RG559YcJY lvoDJhCWhgjc- Received: from [98.203.118.124] by web121601.mail.ne1.yahoo.com via HTTP; Sat, 31 Aug 2013 08:03:08 PDT X-Rocket-MIMEInfo: 002.001, VGhhdCdzIHdheSB0b28gaGlnaC4gWW91ciBiYXNlIHJ4IHJlcXVpcmVtZW50IGlzwqAKClBvcnRzICogcXVldWVzICogcnhkwqAKCldpdGggYSBxdWFkIGNhcmQgeW91IHNob3VsZG4ndCBiZSB1c2luZyBtb3JlIHRoYW4gMiBxdWV1ZXMsIHNvIHlvdXIgcmVxdWlyZW1lbnQKd2l0aCA1IHBvcnRzIGlzIDEwLDI0MCBqdXN0IGZvciB0aGUgcmVjZWl2ZSBzZXR1cC4gSWYgeW91J3JlIHVzaW5nIDQgcXVldWVzIHRoYXQKbnVtYmVyIGRvdWJsZXMsIHdoaWNoIHdvdWxkIG1ha2UgMjUsNjAwIG5vdCBlbm91Z2guwqABMAEBAQE- X-Mailer: YahooMailWebService/0.8.156.576 References: <B966242F-A52D-43F7-A001-99942D53339E@ebureau.com> <CAFMmRNwAuwaGLSQ4P-y=Vzh63jpGXoDRCOXbxeWPoVb3ucy0kQ@mail.gmail.com> <71042F7C-5CBB-4494-B53A-EF4CE45B41BE@ebureau.com> Message-ID: <1377961388.28903.YahooMailNeo@web121601.mail.ne1.yahoo.com> Date: Sat, 31 Aug 2013 08:03:08 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> Subject: Re: Intel 4-port ethernet adaptor link aggregation issue To: Joe Moog <joemoog@ebureau.com>, freebsd-net <freebsd-net@freebsd.org> In-Reply-To: <71042F7C-5CBB-4494-B53A-EF4CE45B41BE@ebureau.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Barney Cordoba <barney_cordoba@yahoo.com> List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>; List-Post: <mailto:freebsd-net@freebsd.org> List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sat, 31 Aug 2013 15:05:23 -0000 That's way too high. Your base rx requirement is=A0=0A=0APorts * queues * r= xd=A0=0A=0AWith a quad card you shouldn't be using more than 2 queues, so y= our requirement=0Awith 5 ports is 10,240 just for the receive setup. If you= 're using 4 queues that=0Anumber doubles, which would make 25,600 not enoug= h.=A0=0A=0ANote that setting mbufs to a huge number doesn't allocate the bu= ffers; they'll be=0Aallocated as needed. It's a ceiling. The reason for the= ceiling is so that you don't=0Ablow up your memory. If your system is usin= g 2 million mbuf clusters then you=0Ahave much bigger problems than LAGG.= =0A=0AAnyone who recommends 2 million clearly has no idea what they're doin= g.=0A=0ABC=0A=0A=0A________________________________=0A From: Joe Moog <joem= oog@ebureau.com>=0ATo: freebsd-net <freebsd-net@freebsd.org> =0ASent: Wedne= sday, August 28, 2013 9:36 AM=0ASubject: Re: Intel 4-port ethernet adaptor = link aggregation issue=0A =0A=0AAll:=0A=0AThanks again to everybody for the= responses and suggestions to our 4-port lagg issue. The solution (for thos= e that may find the information of some value) was to set the value for ker= n.ipc.nmbclusters to a higher value than we had initially. Our previous tun= ing had this value set at 25600, but following a recommendation from the go= od folks at iXSystems we bumped this to a value closer to 2000000, and the = 4-port lagg is functioning as expected now.=0A=0AThank you all.=0A=0AJoe=0A= =0A_______________________________________________=0Afreebsd-net@freebsd.or= g mailing list=0Ahttp://lists.freebsd.org/mailman/listinfo/freebsd-net=0ATo= unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1377952913.44129.YahooMailNeo>