From owner-freebsd-performance@FreeBSD.ORG Tue Nov 21 05:50:40 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BBF9616A403; Tue, 21 Nov 2006 05:50:40 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0C9C243D4C; Tue, 21 Nov 2006 05:50:19 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.13.6/8.13.6) with ESMTP id kAL5ocmL011146; Tue, 21 Nov 2006 00:50:38 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAL5obPR089912 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Nov 2006 00:50:37 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611210550.kAL5obPR089912@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 21 Nov 2006 00:50:50 -0500 To: freebsd-performance@freebsd.org From: Mike Tancsa In-Reply-To: <200611200454.kAK4sdat083568@lava.sentex.ca> References: <2a41acea0611081719h31be096eu614d2f2325aff511@mail.gmail.com> <200611091536.kA9FaltD018819@lava.sentex.ca> <45534E76.6020906@samsco.org> <200611092200.kA9M0q1E020473@lava.sentex.ca> <200611102004.kAAK4iO9027778@lava.sentex.ca> <2a41acea0611101400w5b8cef40ob84ed6de181f3e2c@mail.gmail.com> <200611102221.kAAML6ol028630@lava.sentex.ca> <455570D8.6070000@samsco.org> <200611120412.kAC4CuIB035746@lava.sentex.ca> <45574ECA.4080207@samsco.org> <200611130040.kAD0etbp040637@lava.sentex.ca> <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV version 0.88.3, clamav-milter version 0.88.3 on clamscanner2 X-Virus-Status: Clean Cc: freebsd-net Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Nov 2006 05:50:40 -0000 I am moving this thread over to performance after this post as it makes more sense to continue there. At 11:54 PM 11/19/2006, Mike Tancsa wrote: >At 04:30 PM 11/13/2006, Scott Long wrote: >>Mike Tancsa wrote: >>>At 12:15 AM 11/13/2006, Scott Long wrote: >>> >>>>Is this with EM_INTR_FAST enabled also? >>> >>>Without it, the 2 streams are definitely lossy on the management interface >>> >>> ---Mike >> >>Ok, and would you be able to test the polling options as well? > > >Note about platforms. The HEAD w Patch is a patch >glebius@freebsd.org asked me to test. FastFWD is with >net.inet.ip.fastforwarding on. Also with FastFWD set to one, I >always used the kernel options ADAPTIVE_GIANT commented out and >added NO_ADAPTIVE_MUTEXES. INET6 was removed from all kernels as >well. With these kernel changes, and fast forwarding on, I was able >to keep the box r2 responsive from the console as while blasting >packets across its 2 interfaces. Otherwise, the box seemingly >livelocked. For the linux kernel config, it was pretty well the >default, except I removed INET6, IPSEC and disabled iptables. The >LINUX kernel was 2.6.18.2 on FC5. > >The first test is with UDP netperf. >/usr/local/netperf/netperf -l 60 -H 192.168.44.1 -i 10,2 -I 99,10 -t >UDP_STREAM -- -m 10 -s 32768 -S 32768 >/usr/local/netperf/netperf -l 60 -H 192.168.44.1 -i 10,2 -I 99,10 -t >UDP_STREAM -- -m 64 -s 32768 -S 32768 >/usr/local/netperf/netperf -l 60 -H 192.168.44.1 -i 10,2 -I 99,10 -t >UDP_STREAM -- -m 128 -s 32768 -S 32768 >/usr/local/netperf/netperf -l 60 -H 192.168.44.1 -i 10,2 -I 99,10 -t >UDP_STREAM -- -m 200 -s 32768 -S 32768 Again, this is the same setup as described at http://www.tancsa.com/blast.jpg My goals of this testing was to understand the following : 1) the new em driver to make sure it works well for me and give it a good shake out under load for RELENG_6 2) understand the implications of various configs on forwarding performance of SMP vs UP vs Polling vs Fast Interrupts and to avoid livelock when there is a lot of pps In this round of testing, I tried of RELENG_6 i386 in UP config as well. Although raw packets / second (pps) forwarding was faster, the box was pretty unresponsive and userland apps (i.e. routing) and made the config pretty unusable with fast_forwarding enabled. Once ipfw was loaded, the box would totally lock up and routing daemons was start to spin out of control as hold timers expired. Bottom line, there is slightly less raw pps performance out of an SMP config, but the box seems to be far more resilient against a high pps attack. RELENG_6 SMP with #define EM_FAST_INTR 1 in if_em.c net.inet.ip.fastforwarding=1 with ADAPTIVE_GIANT removed from the kernel and NO_ADAPTIVE_MUTEXES added gives decent pps forwarding rates, without the box becoming live locked. Note, in the real world, given an average packet size of ~600 bytes, a box in this config should be able to route and firewall gigabit speeds without much issue and can sustain moderate DDoS pps attacks over the net. For the routing test, I used 2 peers each sending ~ 195K routes. I re-tested the single UDP stream with 194k routes loaded in the kernel routing table and 2 bgp peers. Then, while blasting across the box, I cleared the session which had all the routes installed in the kernel so that the daemon would have to reinstall all the routes to point to the other peer. While this was happening (10 seconds on SMP boxes, MUCH longer ~ 1min on UP, sometimes total failure) I was measuring throughput. On UP it didnt drop too much, a bit more on SMP, but convergence was quite fast, about 10 seconds. Similarly, installing ipfw rules on the UP version made the box totally live lock in non polling mode, but seemed to perform well enough in polling mode. pf lagged quite far behind The biggest difference seems to be in the efficiency of the firewall rules in LINUX vs FreeBSD. Granted, the rules I inserted, are poorly written, but adding rules did seem to have a linearly negative impact on performance, where as rules via iptables did not significantly impact forwarding rates. However, in the LINUX test, I seemed to trigger some race in bgpd when doing the clear that took a little more proding with FreeBSD, but is there as well :( So it might be back to version .98 The table is also up at http://www.tancsa.com/blast.html which might be easier to read Straight Routing test One Stream 194K Routes bgp clear and single ipfw 5 ipfw ruipfw 10 pf 1 pf 5 Linux 581,310 590,584 579,833 582,963 575,718 579,442 FreeBSD HEAD Nov 20 +FastFWD 540,695 529,425 439,980 398,283 370,458 FreeBSD HEAD Nov 11 441,560 RELENG6 i386 407,403 RELENG6 i386 FastFWD 557,580 562,547 484,250 425,791 386,644 353,856 333,293 FreeBSD HEAD w Patch 422,294 FreeBSD HEAD w Patch FastFWD 567,290 564,340 482,093 436,205 381,459 359,248 361,458 AMD64 RELENG6 w FastFWD 574,590 549,233 486,737 400,050 320,129 296,760 273,824 AMD64 RELENG6 polling 285,917 AMD64 RELENG6 polling FastFWD 512,042 RELENG6 i386 polling FastFWD 558,600 550,041 RELENG6 i386 polling FastFWD HZ=2000 565,520 563,068 373,904 RELENG_6 UP i386 400,188 RELENG_6 UP i386 FastFWD 584,900 582,534 560,033 560,323 RELENG_6 UP i386 FastFWD Polling 585,934 476,629 422,633 393,301 Straight Routing test 2 streams opposite direction Linux 473,810 452,205 408,932 FreeBSD HEAD Nov 11 204,043 FreeBSD HEAD nov 20 + fastFWD 312,140 250,277 223,289 208,551 RELENG6 i386 165,461 RELENG6 i386 FastFWD 368,960 353,437 216,597 206,077 194,47669,50067,385 FreeBSD HEAD w Patch 127,832 FreeBSD HEAD w Patch FastFWD 346,220 404,785 249,617 234,047157,603 AMD64 RELENG6 w Polling 155,659 AMD64 RELENG6 w Polling FastFWD 231,541 RELENG6 i386 polling FastFWD 319,350 312,621 RELENG6 i386 polling FastFWD HZ=2000 300,280 299,018 RELENG_6 UP i386 FastFWD Polling 342,551 229,652 205,322 From owner-freebsd-performance@FreeBSD.ORG Wed Nov 22 02:47:12 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3F66F16A40F; Wed, 22 Nov 2006 02:47:12 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id CA68443D4C; Wed, 22 Nov 2006 02:46:45 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.13.6/8.13.6) with ESMTP id kAM2lACv066345; Tue, 21 Nov 2006 21:47:10 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAM2l9JP095066 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 21 Nov 2006 21:47:09 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611220247.kAM2l9JP095066@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Tue, 21 Nov 2006 21:47:09 -0500 To: freebsd-performance@freebsd.org From: Mike Tancsa In-Reply-To: <7.1.0.9.0.20061120160757.14d4a728@sentex.net> References: <2a41acea0611081719h31be096eu614d2f2325aff511@mail.gmail.com> <200611091536.kA9FaltD018819@lava.sentex.ca> <45534E76.6020906@samsco.org> <200611092200.kA9M0q1E020473@lava.sentex.ca> <200611102004.kAAK4iO9027778@lava.sentex.ca> <2a41acea0611101400w5b8cef40ob84ed6de181f3e2c@mail.gmail.com> <200611102221.kAAML6ol028630@lava.sentex.ca> <455570D8.6070000@samsco.org> <200611120412.kAC4CuIB035746@lava.sentex.ca> <45574ECA.4080207@samsco.org> <200611130040.kAD0etbp040637@lava.sentex.ca> <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV version 0.88.3, clamav-milter version 0.88.3 on clamscanner2 X-Virus-Status: Clean Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 02:47:12 -0000 At 12:50 AM 11/21/2006, Mike Tancsa wrote: >The table is also up at http://www.tancsa.com/blast.html which might >be easier to read Decided to test with RELENG_4 as a comparison. Quite a difference. With polling and fast forwarding on, I can use 2 routers to blast through at almost 1Mpps. Even with ipfw loaded, it performs as RELENG_6 and above does without ipfw. Updated stats in the table at the above URL. One other aspect I have not looked at yet are some of the compile time tunables for em. Looking at the stats from the dual port em nic below, does anyone have any suggestions on what to adjust ? This is a PCIe dual port Pro 1000 PT. em0: Excessive collisions = 0 em0: Symbol errors = 0 em0: Sequence errors = 0 em0: Defer count = 0 em0: Missed Packets = 271924 em0: Receive No Buffers = 1662344 em0: Receive Length Errors = 0 em0: Receive errors = 0 em0: Crc errors = 0 em0: Alignment errors = 0 em0: Carrier extension errors = 0 em0: RX overruns = 0 em0: watchdog timeouts = 0 em0: XON Rcvd = 0 em0: XON Xmtd = 149 em0: XOFF Rcvd = 0 em0: XOFF Xmtd = 272047 em0: Good Packets Rcvd = 175954557 em0: Good Packets Xmtd = 28676200 em1: Excessive collisions = 0 em1: Symbol errors = 0 em1: Sequence errors = 0 em1: Defer count = 0 em1: Missed Packets = 58484 em1: Receive No Buffers = 56645 em1: Receive Length Errors = 0 em1: Receive errors = 0 em1: Crc errors = 0 em1: Alignment errors = 0 em1: Carrier extension errors = 0 em1: RX overruns = 0 em1: watchdog timeouts = 0 em1: XON Rcvd = 0 em1: XON Xmtd = 589 em1: XOFF Rcvd = 0 em1: XOFF Xmtd = 59073 em1: Good Packets Rcvd = 28676216 em1: Good Packets Xmtd = 175954480 ---Mike From owner-freebsd-performance@FreeBSD.ORG Wed Nov 22 13:09:47 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A23D416A492 for ; Wed, 22 Nov 2006 13:09:47 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from smtp3-g19.free.fr (smtp3-g19.free.fr [212.27.42.29]) by mx1.FreeBSD.org (Postfix) with ESMTP id C4DDE43D45 for ; Wed, 22 Nov 2006 13:09:08 +0000 (GMT) (envelope-from tataz@tataz.chchile.org) Received: from tatooine.tataz.chchile.org (tataz.chchile.org [82.233.239.98]) by smtp3-g19.free.fr (Postfix) with ESMTP id 4796A4A20C; Wed, 22 Nov 2006 14:09:22 +0100 (CET) Received: from obiwan.tataz.chchile.org (unknown [192.168.1.25]) by tatooine.tataz.chchile.org (Postfix) with ESMTP id 403C29B46E; Wed, 22 Nov 2006 13:09:47 +0000 (UTC) Received: by obiwan.tataz.chchile.org (Postfix, from userid 1000) id 26CF2405B; Wed, 22 Nov 2006 14:09:47 +0100 (CET) Date: Wed, 22 Nov 2006 14:09:47 +0100 From: Jeremie Le Hen To: Mike Tancsa Message-ID: <20061122130947.GM20405@obiwan.tataz.chchile.org> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200611220247.kAM2l9JP095066@lava.sentex.ca> User-Agent: Mutt/1.5.13 (2006-08-11) Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 13:09:47 -0000 Hi Mike, Thank you for spending that much time for benchmarking, this is really interesting. Though this is a little bit off topic, I'm quite puzzled by the fact that having filtering rules on Linux or not doesn't change the result much. NetFitler keeps track of *all* connections even if there are no ruleset loaded -- you don't have to ask for it, so I guess you are simply wiping filtering rules, but you don't disable connection tracking. AFAIK, you can only disable it by either unloading the `conntrack'' module or recompiling the kernel without it, if built in. It would be interesting to know the real performance of Linux as a mere router if we want a true comparision with FreeBSD performances. Regards, -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org > From owner-freebsd-performance@FreeBSD.ORG Wed Nov 22 13:28:14 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CE48C16A407 for ; Wed, 22 Nov 2006 13:28:14 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id BBD3543D5F for ; Wed, 22 Nov 2006 13:27:45 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kAMDSDg4030806; Wed, 22 Nov 2006 08:28:13 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAMDSBlL097897 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 22 Nov 2006 08:28:11 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611221328.kAMDSBlL097897@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Wed, 22 Nov 2006 08:28:14 -0500 To: Jeremie Le Hen From: Mike Tancsa In-Reply-To: <20061122130947.GM20405@obiwan.tataz.chchile.org> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 13:28:14 -0000 At 08:09 AM 11/22/2006, Jeremie Le Hen wrote: >Hi Mike, > >Thank you for spending that much time for benchmarking, this is really >interesting. Hi, More to come, and if you can think of other tests let me know. Next is VLAN performance. >Though this is a little bit off topic, I'm quite puzzled by the fact >that having filtering rules on Linux or not doesn't change the result >much. NetFitler keeps track of *all* connections even if there are no >ruleset loaded -- you don't have to ask for it Not sure, but I would unload iptables from the kernel when testing. I will check again today as I want to go back and test the LINUX kernel in UP mode to see what difference it makes. >It would be interesting to know the real performance of Linux as a mere >router if we want a true comparision with FreeBSD performances. As just a router, they seem fairly close without any firewall rules. RELENG_4 seems to be the only clear leader in terms of raw pps, at least in these tests. I am still puzzled by the fact that ipfw does relatively poorly compared to RELENG_4. I also have some PCIe bge nics I will try and test. There are some patches that Bruce Evans posted and I would like to see how they perform. ---Mike From owner-freebsd-performance@FreeBSD.ORG Wed Nov 22 13:52:48 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AE56216A49E; Wed, 22 Nov 2006 13:52:48 +0000 (UTC) (envelope-from marcelo@registro.br) Received: from clone.registro.br (clone.registro.br [200.160.2.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 582B243D62; Wed, 22 Nov 2006 13:52:19 +0000 (GMT) (envelope-from marcelo@registro.br) Received: by clone.registro.br (Postfix, from userid 1014) id 3981A2A4BC; Wed, 22 Nov 2006 11:52:47 -0200 (BRST) Date: Wed, 22 Nov 2006 11:52:47 -0200 From: Marcelo Gardini do Amaral To: "O. Hartmann" Message-ID: <20061122135247.GA87427@registro.br> References: <20061030192702.GG76994@registro.br> <20061111091844.I63959@fledge.watson.org> <20061116164053.GR57732@registro.br> <455F1021.6040004@mail.zedat.fu-berlin.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <455F1021.6040004@mail.zedat.fu-berlin.de> User-Agent: Mutt/1.4.2.1i Cc: freebsd-performance@freebsd.org, Robert Watson Subject: Re: DNS Performance Numbers X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 13:52:48 -0000 On Sat, Nov 18, 2006 at 02:52:33PM +0100, O. Hartmann wrote: > These results looks very puzzling to me. > As far as I know, multithreading and/or multiprocessors should perform > better anyway than a single threaded application within other > applications on an UP box. Strange results ...And more strange than this > is the result taken from the FBSD 4.11 box! Is there an explanation why > FreeBSD performs so bad beyond 4.X and on SMP boxes? Please show me > threads ... The results were discussed in the following threads: http://lists.freebsd.org/pipermail/freebsd-net/2006-September/011748.html http://lists.freebsd.org/pipermail/freebsd-net/2006-September/011756.html http://lists.freebsd.org/pipermail/freebsd-net/2006-September/011767.html http://lists.freebsd.org/pipermail/freebsd-net/2006-September/011771.html ... -- Att., Marcelo Gardini From owner-freebsd-performance@FreeBSD.ORG Wed Nov 22 14:27:41 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D3E7C16A415; Wed, 22 Nov 2006 14:27:41 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id DDAA243D58; Wed, 22 Nov 2006 14:27:09 +0000 (GMT) (envelope-from ohartman@zedat.fu-berlin.de) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.62) with esmtp (envelope-from ) id <1Gmt4b-0006gn-EL>; Wed, 22 Nov 2006 15:27:37 +0100 Received: from telesto.geoinf.fu-berlin.de ([130.133.86.198]) by inpost2.zedat.fu-berlin.de (Exim 4.62) with esmtpsa (envelope-from ) id <1Gmt4b-00049h-DH>; Wed, 22 Nov 2006 15:27:37 +0100 Message-ID: <45645E4B.9010308@zedat.fu-berlin.de> Date: Wed, 22 Nov 2006 15:27:23 +0100 From: "O. Hartmann" Organization: Freie =?ISO-8859-1?Q?Universit=E4t_Berlin?= User-Agent: Thunderbird 1.5.0.8 (X11/20061110) MIME-Version: 1.0 To: Marcelo Gardini do Amaral References: <20061030192702.GG76994@registro.br> <20061111091844.I63959@fledge.watson.org> <20061116164053.GR57732@registro.br> <455F1021.6040004@mail.zedat.fu-berlin.de> <20061122135247.GA87427@registro.br> In-Reply-To: <20061122135247.GA87427@registro.br> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: 130.133.86.198 Cc: "O. Hartmann" , Robert Watson , freebsd-performance@freebsd.org Subject: Re: DNS Performance Numbers X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 14:27:41 -0000 Marcelo Gardini do Amaral wrote: > On Sat, Nov 18, 2006 at 02:52:33PM +0100, O. Hartmann wrote: > >> These results looks very puzzling to me. >> As far as I know, multithreading and/or multiprocessors should perform >> better anyway than a single threaded application within other >> applications on an UP box. Strange results ...And more strange than this >> is the result taken from the FBSD 4.11 box! Is there an explanation why >> FreeBSD performs so bad beyond 4.X and on SMP boxes? Please show me >> threads ... > > > The results were discussed in the following threads: > > http://lists.freebsd.org/pipermail/freebsd-net/2006-September/011748.html > > http://lists.freebsd.org/pipermail/freebsd-net/2006-September/011756.html > > http://lists.freebsd.org/pipermail/freebsd-net/2006-September/011767.html > > http://lists.freebsd.org/pipermail/freebsd-net/2006-September/011771.html > > ... > > Thank you very much for the search. To be honest, it looks very disappointing to me. At this moment I regret not having more substantle opportunities to test, but with the end of this year or with the beginning of next year my department starts using Opteron blade systems for our research purposes and I may have the chance to test, too (SUN X4100 or SUN X1200 M2 are discussed, but I'm not sure). My hope is still that FreeBSD evolves to be worth being used in science, especially for clustering and high performance computing. It seems that Linux has made it due its support by many Compiler vendors (Pathscale, Intel) and, obviously, its much better network und SMP performance. Regards, Oliver From owner-freebsd-performance@FreeBSD.ORG Wed Nov 22 15:17:33 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0A43216A403 for ; Wed, 22 Nov 2006 15:17:33 +0000 (UTC) (envelope-from mark@gaiahost.coop) Received: from biodiesel.gaiahost.coop (biodiesel.gaiahost.coop [64.95.78.120]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B5DB43D7B for ; Wed, 22 Nov 2006 15:17:02 +0000 (GMT) (envelope-from mark@gaiahost.coop) Received: from gaiahost.coop (host-64-65-195-19.spr.choiceone.net [::ffff:64.65.195.19]) (AUTH: LOGIN mark@hubcapconsulting.com) by biodiesel.gaiahost.coop with esmtp; Wed, 22 Nov 2006 10:17:29 -0500 id 00534030.45646A09.0000022B Received: by gaiahost.coop (sSMTP sendmail emulation); Wed, 22 Nov 2006 10:17:30 -0500 Date: Wed, 22 Nov 2006 10:17:29 -0500 From: Mark Bucciarelli To: freebsd-performance@freebsd.org Message-ID: <20061122151729.GA564@rabbit> Mail-Followup-To: freebsd-performance@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline User-Agent: Mutt/1.4.2.1i Subject: Re: DNS Performance Numbers X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 15:17:33 -0000 On Wed, Nov 22, 2006 at 11:52:47AM -0200, Marcelo Gardini do Amaral wrote: > > The results were discussed in the following threads: > I see the speed differences are major, but don't have a good idea of what 15,000 DNS queries per second means. Is the following interpretation correct? 15,000 DNS queries per second is: - 1,000 hosted domains, each getting 15 hits/second or - 1,000 mail domains, each getting 15 deliveries/s or - local cache for mailserver with 15,000 incoming messages/s or - some linear combination of the above. I guess something like SPF would adds an extra DNS query for each incoming message. I'm going to be building a DNS server soon and I'm trying to judge if it really matters that 6.1 is so slow. But it's very hard to pass up the fourfold increase you see with NSD on 4.11 over 6.1. Thanks, m From owner-freebsd-performance@FreeBSD.ORG Wed Nov 22 15:45:29 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EC9EB16A4CA for ; Wed, 22 Nov 2006 15:45:28 +0000 (UTC) (envelope-from kreios@gmail.com) Received: from nz-out-0102.google.com (nz-out-0102.google.com [64.233.162.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5BA8D43D78 for ; Wed, 22 Nov 2006 15:43:46 +0000 (GMT) (envelope-from kreios@gmail.com) Received: by nz-out-0102.google.com with SMTP id i11so140218nzh for ; Wed, 22 Nov 2006 07:44:09 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:in-reply-to:references:mime-version:content-type:message-id:cc:content-transfer-encoding:from:subject:date:to:x-mailer; b=T0tTyxZeZYSYJKyx2jN7Kmd5JIRzzyq1pY4NeJSdMNl6aucCThWcaTnQKQlsSowCQJF4wMbnWbL0512hiTKo4iwMqk6T+7AOcyR5ZaRliE/NRjleX917cWGmmHDi2Bbkxn4G2/Uf+pq1gpPlKFtBlY2mKJBmHOO0+GyAhRfuwRQ= Received: by 10.65.154.10 with SMTP id g10mr12684726qbo.1164210249495; Wed, 22 Nov 2006 07:44:09 -0800 (PST) Received: from ?192.168.1.198? ( [71.113.235.243]) by mx.google.com with ESMTP id e19sm14043405qbe.2006.11.22.07.44.08; Wed, 22 Nov 2006 07:44:09 -0800 (PST) In-Reply-To: <455F1021.6040004@mail.zedat.fu-berlin.de> References: <20061030192702.GG76994@registro.br> <20061111091844.I63959@fledge.watson.org> <20061116164053.GR57732@registro.br> <455F1021.6040004@mail.zedat.fu-berlin.de> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <77EFE6C7-96D4-4509-AC03-63F1AAE9BBA4@gmail.com> Content-Transfer-Encoding: 7bit From: Dave Date: Wed, 22 Nov 2006 09:44:05 -0600 To: O. Hartmann X-Mailer: Apple Mail (2.752.2) Cc: freebsd-performance@freebsd.org, Marcelo Gardini do Amaral , Robert Watson Subject: Re: DNS Performance Numbers X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 15:45:29 -0000 On Nov 18, 2006, at 7:52 AM, O. Hartmann wrote: > Marcelo Gardini do Amaral wrote: >>> FYI: In response to feedback from ISC, there are UDP transmit >>> optimizations >>> in FreeBSD 7.x. These have a relatively minor performance impact >>> for >>> single-threaded applications, but in the special case of BIND >>> accessing a >>> single UDP socket from many different threads, it significantly >>> improves >>> performance. I'll look at MFC'ing these to 6.x after 6.2-RELEASE >>> (especially if reminded in a month or so :-). >>> >>> With regard to the possible bge issue -- I would encourage you to >>> test >>> using a 7.x kernel, ideally with all the debugging disabled, and >>> see if >>> there's been any improvement (or regression). There has been a >>> lot of >>> change in these areas, and it would be helpful to know what, if >>> any, impact >>> this has had. >>> >> >> >> I made some tests using 7.x with all the debugging disabled: >> >> queries / s >> >> Int bind (d_t) bind (e_t) nsd (1_s) nsd (2_s) >> --- ---------- ---------- --------- --------- >> >> bge 15439 14733 12910 10946 >> em 37655 34092 42411 41974 >> >> >> d_t: disable threads >> e_t: enable threads (libpthread) >> 1_s: 1 server forked >> 2_s: 2 server forked >> >> Bind: 9.2.3 >> NSD: 3.0.2 >> em: Dell 1950, Intel NIC, SMP kernel >> bge: HP Blade BL35p, Broadcom NIC, SMP kernel >> Client: Dell 1750, Intel NIC, FreeBSD 4.11 UP, running queryperf >> >> >> >> The results are very good for em NIC, better than my numbers [1] with >> FreeBSD 6.1 some months ago. So I guess that we had an >> improvement :-) >> >> But I got the same poor performance with the bge interface. The >> problem remains. >> >> [1] http://lists.freebsd.org/pipermail/freebsd-net/2006-September/ >> 011767.html >> >> Cheers, >> Marcelo >> >> > > > These results looks very puzzling to me. > As far as I know, multithreading and/or multiprocessors should perform > better anyway than a single threaded application within other > applications on an UP box. Strange results ...And more strange than > this > is the result taken from the FBSD 4.11 box! Is there an explanation > why > FreeBSD performs so bad beyond 4.X and on SMP boxes? Please show me > threads ... > > Thanks and regards, > Oliver The FreeBSD pthread library and BIND don't work well together. If you use the libthr library, performance goes up. -- Dave From owner-freebsd-performance@FreeBSD.ORG Wed Nov 22 15:45:49 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 519C916A4FB for ; Wed, 22 Nov 2006 15:45:49 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9583E43DC5 for ; Wed, 22 Nov 2006 15:44:19 +0000 (GMT) (envelope-from ohartman@zedat.fu-berlin.de) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.62) for freebsd-performance@freebsd.org with esmtp (envelope-from ) id <1GmuHE-0004v7-IP>; Wed, 22 Nov 2006 16:44:44 +0100 Received: from telesto.geoinf.fu-berlin.de ([130.133.86.198]) by inpost2.zedat.fu-berlin.de (Exim 4.62) for freebsd-performance@freebsd.org with esmtpsa (envelope-from ) id <1GmuHE-0006TQ-HP>; Wed, 22 Nov 2006 16:44:44 +0100 Message-ID: <45647035.7020401@zedat.fu-berlin.de> Date: Wed, 22 Nov 2006 16:43:49 +0100 From: "O. Hartmann" Organization: Freie =?ISO-8859-1?Q?Universit=E4t_Berlin?= User-Agent: Thunderbird 1.5.0.8 (X11/20061110) MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: <20061122151729.GA564@rabbit> In-Reply-To: <20061122151729.GA564@rabbit> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: 130.133.86.198 Subject: Re: DNS Performance Numbers X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 15:45:49 -0000 Mark Bucciarelli wrote: > On Wed, Nov 22, 2006 at 11:52:47AM -0200, Marcelo Gardini do Amaral wrote: >> The results were discussed in the following threads: >> > > I see the speed differences are major, but don't have a good idea > of what 15,000 DNS queries per second means. Is the following > interpretation correct? > > 15,000 DNS queries per second is: > > - 1,000 hosted domains, each getting 15 hits/second > > or > > - 1,000 mail domains, each getting 15 deliveries/s > > or > > - local cache for mailserver with 15,000 incoming > messages/s > > or > > - some linear combination of the above. > > I guess something like SPF would adds an extra DNS query for each > incoming message. > > I'm going to be building a DNS server soon and I'm trying to > judge if it really matters that 6.1 is so slow. But it's very > hard to pass up the fourfold increase you see with NSD on 4.11 > over 6.1. > > Thanks, > > m Ok, at the moment, your questions reveal some musunderstandings in my way of looking at the subject, so I should asure myself talking about the same problems as others do. Means: as I see the 'numbers', they roughly show the performance of an untuned TCP/IP stack and/or of the speed the kernel can handle the 'subject of being tested for'. At the conclusion, the still means Linux, especially their new kernel 2.6.XX outperforms both FreeBSD 4.11/UP and the new branch 6.1. And it was said that FreeBSD 4.11 outperforms FreeBSD 6.X or the new generation branch at all. My purposes for the specific OS may be not very common and not that specific to DNS. Next year, in Feb, I would like to start build a number crunching system for my work. Due to money limitations, this would be done with standard SoHo or smaller server-like hardware - connection will be 1GbE. So, if performance of SMP capable OS and their network performance differ that much, a OS choice reconsideration would be necessary. In the past 10 years I'm now with FreeBSD, it wasn't very science-friendly due to some lack of suitable support by compiler vendors, as I wrote, i did and do not care about 10% or 15% of performance gain or drainage if I could stay with the familiar OS (in my opinion FreeBSD was the 'better' OS in the past). From owner-freebsd-performance@FreeBSD.ORG Wed Nov 22 17:00:47 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2498216A5EC for ; Wed, 22 Nov 2006 17:00:47 +0000 (UTC) (envelope-from mark@gaiahost.coop) Received: from biodiesel.gaiahost.coop (biodiesel.gaiahost.coop [64.95.78.120]) by mx1.FreeBSD.org (Postfix) with ESMTP id 70A0543D67 for ; Wed, 22 Nov 2006 16:59:34 +0000 (GMT) (envelope-from mark@gaiahost.coop) Received: from gaiahost.coop (host-64-65-195-19.spr.choiceone.net [::ffff:64.65.195.19]) (AUTH: LOGIN mark@hubcapconsulting.com) by biodiesel.gaiahost.coop with esmtp; Wed, 22 Nov 2006 12:00:01 -0500 id 00534068.45648212.00007EB4 Received: by gaiahost.coop (sSMTP sendmail emulation); Wed, 22 Nov 2006 12:00:03 -0500 Date: Wed, 22 Nov 2006 12:00:02 -0500 From: Mark Bucciarelli To: "O. Hartmann" Message-ID: <20061122170002.GA2988@rabbit> Mail-Followup-To: "O. Hartmann" , freebsd-performance@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline User-Agent: Mutt/1.4.2.1i Cc: freebsd-performance@freebsd.org Subject: Re: DNS Performance Numbers X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 17:00:47 -0000 On Wed, Nov 22, 2006 at 04:43:49PM +0100, O. Hartmann wrote: > Mark Bucciarelli wrote: > > > > On Wed, Nov 22, 2006 at 11:52:47AM -0200, Marcelo Gardini do Amaral wrote: > > > > > > The results were discussed in the following threads: > > > > > > > I see the speed differences are major, but don't have a good > > idea of what 15,000 DNS queries per second means. Is the > > following interpretation correct? > > > Means: as I see the 'numbers', they roughly show the > performance of an untuned TCP/IP stack and/or of the speed the > kernel can handle the 'subject of being tested for'. [...] > My purposes for the specific OS may be not very common and not that > specific to DNS. I'm going to build a DNS server in the next couple weeks, and the difference between 6.1 and 4.11 in Marcelo's tests was dramatic. Are you saying Marcelo's test results do not apply to my situation? [...] > i did and do not care about 10% or 15% of > performance gain or > drainage if I could stay with the familiar OS I agree 100% with this idea, and that is why I am trying to get a better gut sense if this performance difference really matters for my application (web and email servers). Thanks, m From owner-freebsd-performance@FreeBSD.ORG Wed Nov 22 17:19:58 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6D33616A492 for ; Wed, 22 Nov 2006 17:19:58 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5338743D46 for ; Wed, 22 Nov 2006 17:19:24 +0000 (GMT) (envelope-from ohartman@zedat.fu-berlin.de) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost1.zedat.fu-berlin.de (Exim 4.62) with esmtp (envelope-from ) id <1GmvlI-0003rA-MG>; Wed, 22 Nov 2006 18:19:52 +0100 Received: from telesto.geoinf.fu-berlin.de ([130.133.86.198]) by inpost2.zedat.fu-berlin.de (Exim 4.62) with esmtpsa (envelope-from ) id <1GmvlI-0000bm-LG>; Wed, 22 Nov 2006 18:19:52 +0100 Message-ID: <45648649.4000408@zedat.fu-berlin.de> Date: Wed, 22 Nov 2006 18:18:01 +0100 From: "O. Hartmann" Organization: Freie =?ISO-8859-1?Q?Universit=E4t_Berlin?= User-Agent: Thunderbird 1.5.0.8 (X11/20061110) MIME-Version: 1.0 To: "O. Hartmann" , freebsd-performance@freebsd.org References: <20061122170002.GA2988@rabbit> In-Reply-To: <20061122170002.GA2988@rabbit> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: 130.133.86.198 Cc: Subject: Re: DNS Performance Numbers X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Nov 2006 17:19:58 -0000 Mark Bucciarelli wrote: > On Wed, Nov 22, 2006 at 04:43:49PM +0100, O. Hartmann wrote: >> Mark Bucciarelli wrote: >>> On Wed, Nov 22, 2006 at 11:52:47AM -0200, Marcelo Gardini do Amaral wrote: >>>> The results were discussed in the following threads: >>>> >>> I see the speed differences are major, but don't have a good >>> idea of what 15,000 DNS queries per second means. Is the >>> following interpretation correct? >>> >> Means: as I see the 'numbers', they roughly show the >> performance of an untuned TCP/IP stack and/or of the speed the >> kernel can handle the 'subject of being tested for'. > > [...] > >> My purposes for the specific OS may be not very common and not that >> specific to DNS. > > I'm going to build a DNS server in the next couple weeks, and the > difference between 6.1 and 4.11 in Marcelo's tests was dramatic. > > Are you saying Marcelo's test results do not apply to my > situation? > > Thanks, > > m No, I complain about the dramatic performance drainage of FreeBSD and would also say, that my main purpose for an UNIX driven box isn't the service for network like routing, DNS and others. But I'm frightened by the poor network performance when I have MPI in vi> > Thanks, > > m ew, for cheaper clusters GbE is still the common way of linking nodes and this will be my 'bottleneck'. So performance of the above shown tests may have the same dramatic impact to my purposes as to yours, b> > Thanks, > > m ut I do not know exactly. It is only a guess and it should in both cases be subject of more investigations. > > [...] > >> i did and do not care about 10% or 15% of > performance gain or >> drainage if I could stay with the familiar OS > > I agree 100% with this idea, and that is why I am trying to get a > better gut sense if this performance difference really matters > for my application (web and email servers). In some aspects I think this test we saw lead to some major problems and I would guess if the performance drainage of the DNS/network service is substantl, there will also be a overall drainage of performance, not even in one particular application aspect. Look at BIND and NDS, do they really suffer from the same systematic problem? If not, the performance drainage comapring UP/SMP and 4.11/6.X seems to be representative. Changing to Linux is an option, but the last one, because it will end a ten years 'friendship' with Berkeley UNIXes. Ok, don't want to be offending or a troll, stopping here ... Serious: I miss performance tests, test suits on which some operating system performance competitions could be made. In the past nearly every new release of any operating system was benchmarked but nowadays this seems to be subject of the specific persons who need to make decissions about the OS choice. Regards, Oliver From owner-freebsd-performance@FreeBSD.ORG Thu Nov 23 00:23:03 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4DE3116A412 for ; Thu, 23 Nov 2006 00:23:03 +0000 (UTC) (envelope-from rnsanchez@wait4.org) Received: from spunkymail-a18.dreamhost.com (mailbigip.dreamhost.com [208.97.132.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6BC9343D5A for ; Thu, 23 Nov 2006 00:22:31 +0000 (GMT) (envelope-from rnsanchez@wait4.org) Received: from sauron.lan.box (unknown [200.180.161.150]) by spunkymail-a18.dreamhost.com (Postfix) with ESMTP id 1B7D95B530; Wed, 22 Nov 2006 16:22:58 -0800 (PST) Date: Wed, 22 Nov 2006 22:22:58 -0200 From: Ricardo Nabinger Sanchez To: "O. Hartmann" Message-Id: <20061122222258.3bcae487.rnsanchez@wait4.org> In-Reply-To: <45648649.4000408@zedat.fu-berlin.de> References: <20061122170002.GA2988@rabbit> <45648649.4000408@zedat.fu-berlin.de> Organization: SYS_WAIT4 X-Mailer: Sylpheed version 2.3.0beta2 (GTK+ 2.10.6; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re: DNS Performance Numbers X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 00:23:03 -0000 On Wed, 22 Nov 2006 18:18:01 +0100 "O. Hartmann" wrote: > No, I complain about the dramatic performance drainage of FreeBSD and > would also say, that my main purpose for an UNIX driven box isn't the > service for network like routing, DNS and others. But I'm frightened by > the poor network performance when I have MPI in vi> OK, but DNS queries are short (less than 1 KB). You probably know much better than me (I also work with scientific computing, but it's been only 4 years) that you can't expect terrific performance with thousands of messages per second -- this will always be your bottleneck. As such, I believe it's unfair (and imprecise) to use DNS performance results for a group communication (MPI) environment comparison. I had good experiences with MPI + FreeBSD, although don't have numbers to prove. Regards. :) -- Ricardo Nabinger Sanchez Powered by FreeBSD "Left to themselves, things tend to go from bad to worse." From owner-freebsd-performance@FreeBSD.ORG Thu Nov 23 12:13:37 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D1C4216A4CE; Thu, 23 Nov 2006 12:13:37 +0000 (UTC) (envelope-from marcelo@registro.br) Received: from clone.registro.br (clone.registro.br [200.160.2.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id D279743D88; Thu, 23 Nov 2006 12:12:55 +0000 (GMT) (envelope-from marcelo@registro.br) Received: by clone.registro.br (Postfix, from userid 1014) id 02FB22A455; Thu, 23 Nov 2006 10:13:29 -0200 (BRST) Date: Thu, 23 Nov 2006 10:13:28 -0200 From: Marcelo Gardini do Amaral To: Dave Message-ID: <20061123121328.GC9038@registro.br> References: <20061030192702.GG76994@registro.br> <20061111091844.I63959@fledge.watson.org> <20061116164053.GR57732@registro.br> <455F1021.6040004@mail.zedat.fu-berlin.de> <77EFE6C7-96D4-4509-AC03-63F1AAE9BBA4@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <77EFE6C7-96D4-4509-AC03-63F1AAE9BBA4@gmail.com> User-Agent: Mutt/1.4.2.1i Cc: "O. Hartmann" , Robert Watson , freebsd-performance@freebsd.org Subject: Re: DNS Performance Numbers X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 12:13:37 -0000 > The FreeBSD pthread library and BIND don't work well together. If > you use the libthr library, performance goes up. I still didn't make tests with this lib, but I rebuilt everything on FreeBSD 6.0 and got nice numbers: DNS performance numbers on FreeBSD 6.0 queries / s Int bind (d_t) bind (e_t) nsd (1_s) nsd (2_s) --- ---------- ---------- --------- --------- bge-UP 39682 26004 42070 41983 bge-SMP 37401 18272 42313 41092 d_t: disable threads e_t: enable threads (libpthread) 1_s: 1 server forked 2_s: 2 server forked Bind: 9.2.3 NSD: 3.0.2 bge: HP Blade BL35p, Broadcom NIC, FreeBSD 6.0 Client: Dell 1750, Intel NIC, FreeBSD 4.11 UP running queryperf And running Bind 9.4.0b4 on the same setup: queries / s Int bind (d_t) bind (e_t) bind (e_a) --- ---------- ---------- ---------- bge-SMP 37470 39063 39035 d_t: disable threads e_t: enable threads (libpthread) e_a: enable threads and enable atomic Well, bge 'get along' in UDP traffic on FreeBSD 6.0. -- Att., Marcelo Gardini NIC .br From owner-freebsd-performance@FreeBSD.ORG Thu Nov 23 16:52:31 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 73F1116A412 for ; Thu, 23 Nov 2006 16:52:31 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 49B9C43D8C for ; Thu, 23 Nov 2006 16:51:44 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kANGqKDK010758; Thu, 23 Nov 2006 11:52:20 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kANGqJsr005016 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 23 Nov 2006 11:52:19 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611231652.kANGqJsr005016@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 23 Nov 2006 11:52:22 -0500 To: Jeremie Le Hen From: Mike Tancsa In-Reply-To: <20061122130947.GM20405@obiwan.tataz.chchile.org> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 16:52:31 -0000 At 08:09 AM 11/22/2006, Jeremie Le Hen wrote: >It would be interesting to know the real performance of Linux as a mere >router if we want a true comparision with FreeBSD performances. Re-tested, this time with a LINUX UP kernel and there is not that much difference in overall speeds. I added a few IPTABLES rules which loaded a few of the modules. [root@r2 ~]# iptables -A OUTPUT -p tcp -s 10.90.2.1 --dport 135:139 -j DROP [root@r2 ~]# iptables -A OUTPUT -p tcp -s 11.90.3.1 --dport 445 -j DROP [root@r2 ~]# iptables -A OUTPUT -p tcp -s 0.0.0.0/0 --dport 448 -j REJECT [root@r2 ~]# [root@r2 ~]# lsmod Module Size Used by ipt_REJECT 5248 1 xt_tcpudp 3456 3 iptable_filter 3328 1 ip_tables 12872 1 iptable_filter x_tables 14212 3 ipt_REJECT,xt_tcpudp,ip_tables autofs4 21508 2 sunrpc 153660 1 acpi_cpufreq 8204 0 dm_mirror 21968 0 dm_multipath 18568 0 dm_mod 57112 2 dm_mirror,dm_multipath video 17028 0 sbs 16192 0 i2c_ec 5376 1 sbs button 7184 0 battery 10500 0 asus_acpi 16792 0 ac 5636 0 pcspkr 3456 0 i2c_nforce2 7552 0 i2c_core 21504 2 i2c_ec,i2c_nforce2 serio_raw 7300 0 tg3 99972 0 e1000 114752 0 ide_cd 38432 0 cdrom 34848 1 ide_cd forcedeth 39940 0 sata_nv 11652 0 libata 98068 1 sata_nv sd_mod 20864 0 scsi_mod 133672 2 libata,sd_mod ext3 129416 1 jbd 58152 1 ext3 ehci_hcd 30984 0 ohci_hcd 19844 0 uhci_hcd 23052 0 [root@r2 ~]# Stopping it does seem to unload the klds [root@r2 ~]# /etc/rc.d/init.d/iptables stop Flushing firewall rules: [ OK ] Setting chains to policy ACCEPT: filter [ OK ] Unloading iptables modules: [ OK ] [root@r2 ~]# [root@r2 ~]# lsmod Module Size Used by ipt_REJECT 5248 0 xt_tcpudp 3456 0 x_tables 14212 2 ipt_REJECT,xt_tcpudp autofs4 21508 2 sunrpc 153660 1 acpi_cpufreq 8204 0 dm_mirror 21968 0 dm_multipath 18568 0 dm_mod 57112 2 dm_mirror,dm_multipath video 17028 0 sbs 16192 0 i2c_ec 5376 1 sbs button 7184 0 battery 10500 0 asus_acpi 16792 0 ac 5636 0 pcspkr 3456 0 i2c_nforce2 7552 0 i2c_core 21504 2 i2c_ec,i2c_nforce2 serio_raw 7300 0 tg3 99972 0 e1000 114752 0 ide_cd 38432 0 cdrom 34848 1 ide_cd forcedeth 39940 0 sata_nv 11652 0 libata 98068 1 sata_nv sd_mod 20864 0 scsi_mod 133672 2 libata,sd_mod ext3 129416 1 jbd 58152 1 ext3 ehci_hcd 30984 0 ohci_hcd 19844 0 uhci_hcd 23052 0 [root@r2 ~]# RELENG_4 still seems to be king, speed wise for raw forwarding and firewalling performance. I tried Dragon Fly, but the em NICs lock up with a few packets blasted at it both in SMP and UP mode. Also tried booting one of their development kernels, but no difference. I might give OpenBSD a quick try as a reference. I also tried a few variations in the kernel from CURRENT. Removing things like KTRACE and PREMPTION seem to have very little impact, but some. Also tried changing the hardware timer to TSC, but that didnt have any impact. From owner-freebsd-performance@FreeBSD.ORG Thu Nov 23 17:43:51 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6292416A494 for ; Thu, 23 Nov 2006 17:43:51 +0000 (UTC) (envelope-from dudu@dudu.ro) Received: from nz-out-0102.google.com (nz-out-0102.google.com [64.233.162.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3359843D5A for ; Thu, 23 Nov 2006 17:43:08 +0000 (GMT) (envelope-from dudu@dudu.ro) Received: by nz-out-0102.google.com with SMTP id i11so336219nzh for ; Thu, 23 Nov 2006 09:43:44 -0800 (PST) Received: by 10.65.219.9 with SMTP id w9mr9899866qbq.1164303824085; Thu, 23 Nov 2006 09:43:44 -0800 (PST) Received: by 10.65.110.19 with HTTP; Thu, 23 Nov 2006 09:43:44 -0800 (PST) Message-ID: Date: Thu, 23 Nov 2006 19:43:44 +0200 From: "Vlad Galu" To: freebsd-performance@freebsd.org, freebsd-net@freebsd.org In-Reply-To: <200611231652.kANGqJsr005016@lava.sentex.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4557CECD.2000609@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> Cc: Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 17:43:51 -0000 On 11/23/06, Mike Tancsa wrote: > At 08:09 AM 11/22/2006, Jeremie Le Hen wrote: > > >It would be interesting to know the real performance of Linux as a mere > >router if we want a true comparision with FreeBSD performances. > > Re-tested, this time with a LINUX UP kernel and there is not that > much difference in overall speeds. I added a few IPTABLES rules which > loaded a few of the modules. > Can you please completely remove the iptables support from your Linux configuration, as well as removing support for any packet filter in FreeBSD? Also, please enable fast_forwarding. -- If it's there, and you can see it, it's real. If it's not there, and you can see it, it's virtual. If it's there, and you can't see it, it's transparent. If it's not there, and you can't see it, you erased it. From owner-freebsd-performance@FreeBSD.ORG Thu Nov 23 18:03:24 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B452B16A415; Thu, 23 Nov 2006 18:03:24 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 708F143D70; Thu, 23 Nov 2006 18:02:46 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kANI3Lg4019588; Thu, 23 Nov 2006 13:03:21 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kANI3KBf005315 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 23 Nov 2006 13:03:20 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611231803.kANI3KBf005315@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 23 Nov 2006 13:03:24 -0500 To: "Vlad Galu" , freebsd-performance@freebsd.org, freebsd-net@freebsd.org From: Mike Tancsa In-Reply-To: References: <4557CECD.2000609@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 18:03:24 -0000 At 12:43 PM 11/23/2006, Vlad Galu wrote: > Can you please completely remove the iptables support from your >Linux configuration, as well as removing support for any packet filter >in FreeBSD? Also, please enable fast_forwarding. I did that a while ago. See http://www.tancsa.com/blast.html ---Mike From owner-freebsd-performance@FreeBSD.ORG Thu Nov 23 18:20:22 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5C7BB16A416 for ; Thu, 23 Nov 2006 18:20:22 +0000 (UTC) (envelope-from dudu@dudu.ro) Received: from nz-out-0102.google.com (nz-out-0102.google.com [64.233.162.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6BBDB43D5C for ; Thu, 23 Nov 2006 18:19:44 +0000 (GMT) (envelope-from dudu@dudu.ro) Received: by nz-out-0102.google.com with SMTP id s18so413478nze for ; Thu, 23 Nov 2006 10:20:20 -0800 (PST) Received: by 10.65.219.3 with SMTP id w3mr10020508qbq.1164306020425; Thu, 23 Nov 2006 10:20:20 -0800 (PST) Received: by 10.65.110.19 with HTTP; Thu, 23 Nov 2006 10:20:20 -0800 (PST) Message-ID: Date: Thu, 23 Nov 2006 20:20:20 +0200 From: "Vlad Galu" To: "Mike Tancsa" In-Reply-To: <200611231803.kANI3KBf005315@lava.sentex.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4557CECD.2000609@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <200611231803.kANI3KBf005315@lava.sentex.ca> Cc: freebsd-net@freebsd.org, freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 18:20:22 -0000 On 11/23/06, Mike Tancsa wrote: > At 12:43 PM 11/23/2006, Vlad Galu wrote: > > > Can you please completely remove the iptables support from your > >Linux configuration, as well as removing support for any packet filter > >in FreeBSD? Also, please enable fast_forwarding. > > I did that a while ago. See I'm sorry, I had too much coffee :( > > http://www.tancsa.com/blast.html > > > ---Mike > > -- If it's there, and you can see it, it's real. If it's not there, and you can see it, it's virtual. If it's there, and you can't see it, it's transparent. If it's not there, and you can't see it, you erased it. From owner-freebsd-performance@FreeBSD.ORG Thu Nov 23 22:14:13 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 14D3116A416 for ; Thu, 23 Nov 2006 22:14:13 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 88B4D43D62 for ; Thu, 23 Nov 2006 22:13:34 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.13.6/8.13.6) with ESMTP id kANME5VC017630; Thu, 23 Nov 2006 17:14:05 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kANMDt6C006356 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 23 Nov 2006 17:14:05 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611232214.kANMDt6C006356@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 23 Nov 2006 17:13:59 -0500 To: freebsd-performance@freebsd.org From: Mike Tancsa In-Reply-To: References: <4557CECD.2000609@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <200611231803.kANI3KBf005315@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV version 0.88.3, clamav-milter version 0.88.3 on clamscanner2 X-Virus-Status: Clean Subject: Re: Broadcom bge forwarding performance (was em forwarding performance ) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Nov 2006 22:14:13 -0000 >>test setup description at >>http://www.tancsa.com/blast.html More testing :) This time with a pair of PCIe 1x bge nics, as well as using the patch at http://lists.freebsd.org/pipermail/freebsd-net/2006-November/012389.html As I switched to bge, I could test against Dragonfly as well since its em driver is broken with the dual port PT1000 Couple of interesting notes. Unlike the em nics, I dont see the box being overwhelmed with packets to the point of netstat -ni showing input errors. I dont know if this is just a measurement issue, or the driver is smarter about throttling back the sender rate with the switch ? I also noticed that the initial blast, as seen by the routing box, sees pps rates in around 500k, but then slowly goes down to about 460k and settles in around that rate but then eventually go back up towards 500kpps. 26479 0 1588740 2 0 332 0 511592 0 30695562 3 0 386 0 508043 0 30482580 2 0 332 0 508026 0 30481560 2 0 332 0 input (Total) output packets errs bytes packets errs bytes colls 506675 0 30400628 2 0 332 0 511805 0 30708402 3 0 498 0 477792 0 28667520 2 0 332 0 454527 0 27271620 2 0 332 0 454815 0 27288900 2 0 332 0 453920 0 27235200 2 0 444 0 449027 0 26941620 2 0 332 0 449383 0 26962980 2 0 332 0 461838 0 27710280 2 0 332 0 455068 0 27304080 2 0 332 0 455014 0 27300840 2 0 332 0 411416 0 24684960 4 0 664 0 3 0 180 2 0 332 0 2 0 120 2 0 332 0 4 0 293 2 0 332 0 2 0 120 2 0 332 0 443378 0 26602680 2 0 332 0 455782 0 27346920 2 0 332 0 453432 0 27205920 2 0 332 0 453816 0 27228960 2 0 332 0 450561 0 27033660 2 0 332 0 input (Total) output packets errs bytes packets errs bytes colls 450317 0 27019020 2 0 332 0 454149 0 27248940 2 0 444 0 With the above patch and from current as of today (including http://lists.freebsd.org/pipermail/cvs-all/2006-November/197096.html) the box is *very* responsive even when two blasts of packets are going in opposite directions! Here initially is one blast coming from 192.168.88.176 to 192.168.44.1, then I start up another blast from 192.168.44.244 to 192.168.88.206. The box is totally responsive from the management interface [r2-current]# vmstat -i interrupt total rate irq4: sio0 929 0 irq14: ata0 1489 0 irq15: ata1 67 0 irq16: bge1 3158402 1980 irq17: bge0 4918 3 irq19: bge2 2358601 1478 cpu0: timer 3187013 1998 irq256: em0 4 0 irq257: em1 4 0 cpu1: timer 3186657 1997 Total 11898084 7459 [r2-current]# [r2-current]# ifstat -b bge0 bge1 bge2 Kbps in Kbps out Kbps in Kbps out Kbps in Kbps out 0.47 1.67 222466.8 0.00 0.00 174265.6 0.94 1.17 233668.8 0.00 0.00 183040.6 0.47 1.17 212013.7 0.00 0.00 166077.4 0.94 1.17 220571.9 0.00 0.00 172781.3 0.47 1.17 240600.0 0.00 0.00 188470.0 0.94 1.17 239261.9 0.00 0.00 187421.8 0.47 1.17 210286.3 0.00 0.00 164724.2 32.20 46.60 236021.0 0.00 0.00 184883.1 9.18 8.46 240039.5 0.00 0.00 188030.9 7.31 7.54 237144.6 0.00 0.00 185763.3 7.59 7.62 231464.9 0.00 0.00 181314.2 0.47 2.47 210767.9 0.00 0.00 165101.5 1.40 2.47 216543.6 0.00 0.00 169625.8 0.94 2.47 213539.4 0.00 0.00 167272.6 1.40 2.47 214488.8 0.00 0.00 168016.2 0.94 2.47 197169.2 21056.33 24296.24 154449.2 1.40 2.47 117003.6 101840.6 117507.9 91652.81 2.20 2.89 119036.2 100521.9 115987.3 93245.06 1.87 2.47 117262.4 101812.5 117475.5 91855.57 0.94 2.47 118018.5 101145.1 116706.3 92447.80 1.40 2.47 116557.3 102412.8 118168.2 91303.23 0.94 2.47 117745.3 101736.3 117388.0 92233.84 1.40 2.47 117146.3 102174.3 117893.4 91764.60 0.94 2.47 116071.5 102631.6 118421.1 90922.66 0.94 2.47 116262.2 102233.0 117961.1 91072.04 0.94 2.47 116043.0 102018.6 117713.2 90899.94 1.87 2.47 115890.5 101801.2 117463.4 90780.86 0.94 2.47 117120.8 101347.2 116939.1 91744.60 1.40 2.47 118051.4 100909.8 116434.4 92473.56 0.94 2.47 118767.3 102499.5 118268.6 93034.42 1.40 2.47 116915.7 101836.6 117503.8 91583.94 1.40 3.34 116517.4 102566.9 118346.0 91271.97 1.73 2.89 116316.7 102750.2 118558.4 91114.77 0.94 2.47 117356.1 101465.1 117074.6 91928.93 1.40 2.47 117906.8 101133.9 116693.0 92360.30 1.82 2.47 116012.6 102530.9 118305.3 90876.51 1.40 2.47 115859.8 103404.1 119311.9 90756.86 0.94 2.47 117364.7 102034.7 117732.3 91935.72 1.40 2.47 117721.1 100830.2 116342.5 92214.86 1.40 2.47 116521.8 102289.6 118026.9 91275.45 0.94 2.47 117061.8 101941.4 117624.2 91698.38 bge0 bge1 bge2 Kbps in Kbps out Kbps in Kbps out Kbps in Kbps out 1.87 2.47 91718.83 119571.1 137967.1 71846.42 0.94 3.47 0.00 182065.9 210075.6 0.00 1.40 2.47 0.00 150718.8 173906.3 0.00 0.47 2.47 0.00 0.00 0.00 0.00 1.40 2.47 0.00 0.00 0.00 0.00 0.94 2.47 0.00 0.00 0.00 0.00 1.40 2.47 0.00 0.00 0.00 0.00 [r2-current]# netstat -ni 1 input (Total) output packets errs bytes packets errs bytes colls 449655 0 26979300 449622 0 21134055 0 462036 0 27722160 462018 0 21715820 0 457761 0 27465660 457814 0 21514989 0 454539 0 27272340 454446 0 21363555 0 469598 0 28175880 469277 0 22175141 0 501175 0 30070500 501315 0 24809563 0 500932 0 30055920 500938 0 24777688 0 502038 0 30122322 501827 0 24855812 0 501276 0 30076560 501521 0 24804540 0 501378 0 30082680 501418 0 24821941 0 501099 0 30065940 501067 0 24806099 0 502164 0 30129840 501890 0 24862704 0 501911 0 30114660 502006 0 24856474 0 499672 0 29980320 499854 0 24742438 0 499575 0 29974502 499586 0 24738495 0 498042 0 29882520 498027 0 24659622 0 500104 0 30006240 500065 0 24751670 0 500569 0 30034140 500653 0 24768087 0 506240 0 30374400 506113 0 25066903 0 500648 0 30038880 500563 0 24777197 0 input (Total) output packets errs bytes packets errs bytes colls 501650 0 30099000 501554 0 24845586 0 501332 0 30079962 501550 0 24825352 0 500664 0 30039840 500713 0 24778844 0 501305 0 30078300 501235 0 24812561 0 500147 0 30008873 499945 0 24769644 0 501150 0 30069000 501284 0 24827402 0 503208 0 30192420 503357 0 24909495 0 499578 0 29974740 499528 0 24717105 0 500846 0 30050760 500855 0 24799047 0 501295 0 30077700 501261 0 24825246 0 496749 0 29804940 496919 0 24691116 0 449463 0 26967780 449513 0 23372416 0 427765 0 25665900 427857 0 22244044 0 2 0 120 2 0 316 0 Results along with ipfw rule tests at http://www.tancsa.com/blast.html I also compared Dragonfly using the bge nics and their behaviour is quite different as compared to FreeBSD. Out of the box its "slower" but it keeps its forwarding speed with or without ipfw loaded. Development kernel results are up there as well and it behaves much like the other BSDs but is notably faster on the firewall rule processing. However, FreeBSD HEAD when in UP mode will live lock with 2 blasts of packets going in opposite directions. The management interface as well as the serial console is not responsive at all. [r2-dragonfly]# netstat -ni Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll bge0 1500 00:10:18:14:15:12 600 0 232 0 0 bge0 1500 fe80:1::210 fe80:1::210:18ff: 0 - 0 - - bge0 1500 192.168.43 192.168.43.224 280 - 223 - - bge1 1500 00:10:18:14:27:d5 76475440 1271 8606533 0 0 bge1 1500 192.168.88 192.168.88.223 4 - 4 - - bge1 1500 fe80:2::210 fe80:2::210:18ff: 0 - 0 - - bge2 1500 00:10:18:14:38:d2 8631578 2465 76061495 0 0 bge2 1500 192.168.44 192.168.44.223 1 - 1 - - bge2 1500 fe80:3::210 fe80:3::210:18ff: 0 - 0 - - From owner-freebsd-performance@FreeBSD.ORG Fri Nov 24 08:28:21 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C88D116A407 for ; Fri, 24 Nov 2006 08:28:21 +0000 (UTC) (envelope-from massimo@cedoc.mo.it) Received: from insomma.datacode.it (ip-174-86.sn2.eutelia.it [83.211.174.86]) by mx1.FreeBSD.org (Postfix) with ESMTP id E931C43D6D for ; Fri, 24 Nov 2006 08:27:39 +0000 (GMT) (envelope-from massimo@cedoc.mo.it) Received: from localhost (localhost.datacode.it [127.0.0.1]) by insomma.datacode.it (Postfix) with SMTP id 9B2E12C90B for ; Fri, 24 Nov 2006 09:28:16 +0100 (CET) Received: from insomma.datacode.it (localhost.datacode.it [127.0.0.1]) by insomma.datacode.it (Postfix) with ESMTP id 78D602C90A; Fri, 24 Nov 2006 09:28:15 +0100 (CET) Received: from massimo.datacode.it (massimo.datacode.it [192.168.1.13]) by insomma.datacode.it (Postfix) with ESMTP id 24C812C906; Fri, 24 Nov 2006 09:28:15 +0100 (CET) From: Massimo Lusetti To: Mike Tancsa In-Reply-To: <200611231652.kANGqJsr005016@lava.sentex.ca> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> Content-Type: text/plain Organization: CEDOC - Modena Date: Fri, 24 Nov 2006 09:28:14 +0100 Message-Id: <1164356894.4306.1.camel@massimo.datacode.it> Mime-Version: 1.0 X-Mailer: Evolution 2.0.4 (2.0.4-7) Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP Cc: freebsd-performance@freebsd.org, Jeremie Le Hen Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 08:28:21 -0000 On Thu, 2006-11-23 at 11:52 -0500, Mike Tancsa wrote: > I might give OpenBSD a quick try as a reference. That would be very interesting. BTW you really did a good and very compete job, thanks! Regards -- Massimo.run(); From owner-freebsd-performance@FreeBSD.ORG Fri Nov 24 20:27:38 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A592616A416 for ; Fri, 24 Nov 2006 20:27:38 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D68CD43D69 for ; Fri, 24 Nov 2006 20:26:54 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kAOKRaAb057808; Fri, 24 Nov 2006 15:27:36 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAOKRYZg012113 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 24 Nov 2006 15:27:34 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611242027.kAOKRYZg012113@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 24 Nov 2006 15:27:40 -0500 To: Massimo Lusetti From: Mike Tancsa In-Reply-To: <1164356894.4306.1.camel@massimo.datacode.it> References: <4557CECD.2000609@samsco.org> <200611130158.kAD1wdKE040908@lava.sentex.ca> <4557EF13.9060305@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-performance@freebsd.org, Jeremie Le Hen Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 20:27:38 -0000 At 03:28 AM 11/24/2006, Massimo Lusetti wrote: >On Thu, 2006-11-23 at 11:52 -0500, Mike Tancsa wrote: > > > I might give OpenBSD a quick try as a reference. > >That would be very interesting. OpenBSD 4.0 i386 panics on boot. I also posted some results with PMC compiled into the kernel ipfw compiled into the kernel, with 1 rule http://www.tancsa.com/pmc/ ---Mike From owner-freebsd-performance@FreeBSD.ORG Fri Nov 24 21:03:10 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 885E816A416 for ; Fri, 24 Nov 2006 21:03:10 +0000 (UTC) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (eva.fit.vutbr.cz [147.229.176.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4DD4543D46 for ; Fri, 24 Nov 2006 21:02:25 +0000 (GMT) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (localhost [127.0.0.1]) by eva.fit.vutbr.cz (envelope-from xdivac02@eva.fit.vutbr.cz) (8.13.8/8.13.7) with ESMTP id kAOL35VL050293 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 24 Nov 2006 22:03:06 +0100 (CET) Received: (from xdivac02@localhost) by eva.fit.vutbr.cz (8.13.8/8.13.3/Submit) id kAOL35J2050292; Fri, 24 Nov 2006 22:03:05 +0100 (CET) Date: Fri, 24 Nov 2006 22:03:05 +0100 From: Divacky Roman To: Mike Tancsa Message-ID: <20061124210305.GA49228@stud.fit.vutbr.cz> References: <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611242027.kAOKRYZg012113@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200611242027.kAOKRYZg012113@lava.sentex.ca> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.57 on 147.229.176.14 Cc: freebsd-performance@freebsd.org, Massimo Lusetti , Jeremie Le Hen Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 21:03:10 -0000 On Fri, Nov 24, 2006 at 03:27:40PM -0500, Mike Tancsa wrote: > At 03:28 AM 11/24/2006, Massimo Lusetti wrote: > >On Thu, 2006-11-23 at 11:52 -0500, Mike Tancsa wrote: > > > >> I might give OpenBSD a quick try as a reference. > > > >That would be very interesting. > > OpenBSD 4.0 i386 panics on boot. > > I also posted some results with PMC compiled into the kernel > > ipfw compiled into the kernel, with 1 rule > > http://www.tancsa.com/pmc/ I see generic_bzero/bcopy used quite often. why dont you define cpu I586_CPU in your kernel config? also... bde@ just commited some bzero-related optimizations to routing code.. might be work trying.. just an idea roman From owner-freebsd-performance@FreeBSD.ORG Fri Nov 24 21:17:59 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CDCEC16A416 for ; Fri, 24 Nov 2006 21:17:59 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 930BF43D75 for ; Fri, 24 Nov 2006 21:17:15 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kAOLHweO068263; Fri, 24 Nov 2006 16:17:58 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAOLHuBP012313 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 24 Nov 2006 16:17:57 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611242117.kAOLHuBP012313@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 24 Nov 2006 16:18:03 -0500 To: Divacky Roman From: Mike Tancsa In-Reply-To: <20061124210305.GA49228@stud.fit.vutbr.cz> References: <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611242027.kAOKRYZg012113@lava.sentex.ca> <20061124210305.GA49228@stud.fit.vutbr.cz> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 21:17:59 -0000 At 04:03 PM 11/24/2006, Divacky Roman wrote: >On Fri, Nov 24, 2006 at 03:27:40PM -0500, Mike Tancsa wrote: > > At 03:28 AM 11/24/2006, Massimo Lusetti wrote: > > >On Thu, 2006-11-23 at 11:52 -0500, Mike Tancsa wrote: > > > > > >> I might give OpenBSD a quick try as a reference. > > > > > >That would be very interesting. > > > > OpenBSD 4.0 i386 panics on boot. > > > > I also posted some results with PMC compiled into the kernel > > > > ipfw compiled into the kernel, with 1 rule > > > > http://www.tancsa.com/pmc/ > >I see generic_bzero/bcopy used quite often. why dont you define >cpu I586_CPU >in your kernel config? Hi, I had cpu I486_CPU cpu I586_CPU cpu I686_CPU Are you saying cpu I586_CPU cpu I686_CPU or just have cpu I586_CPU >also... bde@ just commited some bzero-related optimizations to routing >code.. might be work trying.. I think that was already in there, but I will cvsup upto today and see if there is any difference. ---Mike From owner-freebsd-performance@FreeBSD.ORG Fri Nov 24 22:51:44 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BDB3A16A407 for ; Fri, 24 Nov 2006 22:51:44 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 25B2C43D62 for ; Fri, 24 Nov 2006 22:50:59 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kAOMpfmt088243; Fri, 24 Nov 2006 17:51:42 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAOMpfv6012738 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 24 Nov 2006 17:51:41 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611242251.kAOMpfv6012738@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 24 Nov 2006 17:51:47 -0500 To: Divacky Roman From: Mike Tancsa In-Reply-To: <20061124210305.GA49228@stud.fit.vutbr.cz> References: <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611242027.kAOKRYZg012113@lava.sentex.ca> <20061124210305.GA49228@stud.fit.vutbr.cz> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-performance@freebsd.org, Massimo Lusetti , Jeremie Le Hen Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 22:51:44 -0000 At 04:03 PM 11/24/2006, Divacky Roman wrote: >I see generic_bzero/bcopy used quite often. why dont you define >cpu I586_CPU >in your kernel config? Hi, I cvsup'd to todays kernel and re-ran some of the tests, controlling for CPU defs in the kernel. Posted at http://www.tancsa.com/blast.html Statistically, I think the results are too close to say they are different. ---Mike From owner-freebsd-performance@FreeBSD.ORG Fri Nov 24 23:40:59 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F3F7216A415 for ; Fri, 24 Nov 2006 23:40:58 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from multiplay.co.uk (core6.multiplay.co.uk [85.236.96.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id 13E2943D58 for ; Fri, 24 Nov 2006 23:40:13 +0000 (GMT) (envelope-from killing@multiplay.co.uk) Received: from vader ([212.135.219.179]) by multiplay.co.uk (multiplay.co.uk [85.236.96.23]) (MDaemon PRO v9.0.1) with ESMTP id md50003251329.msg for ; Fri, 24 Nov 2006 23:41:02 +0000 Message-ID: <001b01c71021$f85ce070$b3db87d4@multiplay.co.uk> From: "Steven Hartland" To: "Divacky Roman" , "Mike Tancsa" References: <4557FF7A.8020704@samsco.org><200611132054.kADKsFvK045726@lava.sentex.ca><4558E3DC.6080800@samsco.org><200611200454.kAK4sdat083568@lava.sentex.ca><7.1.0.9.0.20061120160757.14d4a728@sentex.net><200611220247.kAM2l9JP095066@lava.sentex.ca><20061122130947.GM20405@obiwan.tataz.chchile.org><200611231652.kANGqJsr005016@lava.sentex.ca><1164356894.4306.1.camel@massimo.datacode.it><200611242027.kAOKRYZg012113@lava.sentex.ca><20061124210305.GA49228@stud.fit.vutbr.cz> <200611242251.kAOMpfv6012738@lava.sentex.ca> Date: Fri, 24 Nov 2006 23:40:42 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2869 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2962 X-Spam-Processed: multiplay.co.uk, Fri, 24 Nov 2006 23:41:02 +0000 (not processed: message from valid local sender) X-MDRemoteIP: 212.135.219.179 X-Return-Path: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-performance@freebsd.org X-MDAV-Processed: multiplay.co.uk, Fri, 24 Nov 2006 23:41:03 +0000 Cc: freebsd-performance@freebsd.org, Massimo Lusetti , Jeremie Le Hen Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Nov 2006 23:40:59 -0000 Mike Tancsa wrote: > I cvsup'd to todays kernel and re-ran some of the tests, controlling > for CPU defs in the kernel. Posted at > http://www.tancsa.com/blast.html > > Statistically, I think the results are too close to say they are > different. Whats wrong with that web page the display is totally broken :( Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-performance@FreeBSD.ORG Sat Nov 25 01:20:39 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6902016A415 for ; Sat, 25 Nov 2006 01:20:39 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 48F3743D46 for ; Sat, 25 Nov 2006 01:19:54 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kAP1KbK5010190; Fri, 24 Nov 2006 20:20:38 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAP1KbfJ013275 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 24 Nov 2006 20:20:37 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611250120.kAP1KbfJ013275@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Fri, 24 Nov 2006 20:20:43 -0500 To: "Steven Hartland" From: Mike Tancsa In-Reply-To: <001b01c71021$f85ce070$b3db87d4@multiplay.co.uk> References: <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611242027.kAOKRYZg012113@lava.sentex.ca> <20061124210305.GA49228@stud.fit.vutbr.cz> <200611242251.kAOMpfv6012738@lava.sentex.ca> <001b01c71021$f85ce070$b3db87d4@multiplay.co.uk> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Nov 2006 01:20:39 -0000 At 06:40 PM 11/24/2006, Steven Hartland wrote: >Whats wrong with that web page the display is totally broken :( Try it now. ---Mike From owner-freebsd-performance@FreeBSD.ORG Sat Nov 25 08:36:09 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C145E16A407 for ; Sat, 25 Nov 2006 08:36:09 +0000 (UTC) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (eva.fit.vutbr.cz [147.229.176.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF9F643D46 for ; Sat, 25 Nov 2006 08:35:22 +0000 (GMT) (envelope-from xdivac02@stud.fit.vutbr.cz) Received: from eva.fit.vutbr.cz (localhost [127.0.0.1]) by eva.fit.vutbr.cz (envelope-from xdivac02@eva.fit.vutbr.cz) (8.13.8/8.13.7) with ESMTP id kAP8a5Es017740 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 25 Nov 2006 09:36:05 +0100 (CET) Received: (from xdivac02@localhost) by eva.fit.vutbr.cz (8.13.8/8.13.3/Submit) id kAP8a5o8017739; Sat, 25 Nov 2006 09:36:05 +0100 (CET) Date: Sat, 25 Nov 2006 09:36:05 +0100 From: Divacky Roman To: Mike Tancsa Message-ID: <20061125083605.GA17350@stud.fit.vutbr.cz> References: <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611242027.kAOKRYZg012113@lava.sentex.ca> <20061124210305.GA49228@stud.fit.vutbr.cz> <200611242117.kAOLHuBP012313@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200611242117.kAOLHuBP012313@lava.sentex.ca> User-Agent: Mutt/1.4.2.2i X-Scanned-By: MIMEDefang 2.57 on 147.229.176.14 Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Nov 2006 08:36:09 -0000 On Fri, Nov 24, 2006 at 04:18:03PM -0500, Mike Tancsa wrote: > At 04:03 PM 11/24/2006, Divacky Roman wrote: > >On Fri, Nov 24, 2006 at 03:27:40PM -0500, Mike Tancsa wrote: > >> At 03:28 AM 11/24/2006, Massimo Lusetti wrote: > >> >On Thu, 2006-11-23 at 11:52 -0500, Mike Tancsa wrote: > >> > > >> >> I might give OpenBSD a quick try as a reference. > >> > > >> >That would be very interesting. > >> > >> OpenBSD 4.0 i386 panics on boot. > >> > >> I also posted some results with PMC compiled into the kernel > >> > >> ipfw compiled into the kernel, with 1 rule > >> > >> http://www.tancsa.com/pmc/ > > > >I see generic_bzero/bcopy used quite often. why dont you define > >cpu I586_CPU > >in your kernel config? > > Hi, > > I had > > cpu I486_CPU > cpu I586_CPU > cpu I686_CPU hm.. now I am confused. the rule is that having I586_CPU improves performance because optimized bzero/bcopy is included (its not included if you only have I686_CPU). I dont understand why the generic version is used. From owner-freebsd-performance@FreeBSD.ORG Sat Nov 25 09:37:35 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 282CB16A403 for ; Sat, 25 Nov 2006 09:37:35 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id E4E1743D5D for ; Sat, 25 Nov 2006 09:36:47 +0000 (GMT) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1GntyX-0005Ho-Dg for freebsd-performance@freebsd.org; Sat, 25 Nov 2006 10:37:33 +0100 Received: from 83-131-111-147.adsl.net.t-com.hr ([83.131.111.147]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 25 Nov 2006 10:37:33 +0100 Received: from ivoras by 83-131-111-147.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 25 Nov 2006 10:37:33 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Sat, 25 Nov 2006 10:37:37 +0100 Lines: 10 Message-ID: References: <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611242027.kAOKRYZg012113@lava.sentex.ca> <20061124210305.GA49228@stud.fit.vutbr.cz> <200611242117.kAOLHuBP012313@lava.sentex.ca> <20061125083605.GA17350@stud.fit.vutbr.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 83-131-111-147.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.8 (Windows/20061025) In-Reply-To: <20061125083605.GA17350@stud.fit.vutbr.cz> Sender: news Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Nov 2006 09:37:35 -0000 Divacky Roman wrote: > hm.. now I am confused. the rule is that having I586_CPU improves > performance because optimized bzero/bcopy is included (its not > included if you only have I686_CPU). > > I dont understand why the generic version is used. I believe the consensus was that I486 line disables it, i.e. to leave just I586 and I686 in. From owner-freebsd-performance@FreeBSD.ORG Sat Nov 25 11:22:19 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 104F216A417 for ; Sat, 25 Nov 2006 11:22:19 +0000 (UTC) (envelope-from fullermd@over-yonder.net) Received: from optimus.centralmiss.com (ns.centralmiss.com [206.156.254.79]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5AB5243D6D for ; Sat, 25 Nov 2006 11:21:30 +0000 (GMT) (envelope-from fullermd@over-yonder.net) Received: from draco.over-yonder.net (adsl-072-148-013-213.sip.jan.bellsouth.net [72.148.13.213]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by optimus.centralmiss.com (Postfix) with ESMTP id 1731A284E9; Sat, 25 Nov 2006 05:22:17 -0600 (CST) Received: by draco.over-yonder.net (Postfix, from userid 100) id 89C3261C36; Sat, 25 Nov 2006 05:22:16 -0600 (CST) Date: Sat, 25 Nov 2006 05:22:16 -0600 From: "Matthew D. Fuller" To: Divacky Roman Message-ID: <20061125112216.GC91673@over-yonder.net> References: <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> <1164356894.4306.1.camel@massimo.datacode.it> <200611242027.kAOKRYZg012113@lava.sentex.ca> <20061124210305.GA49228@stud.fit.vutbr.cz> <200611242117.kAOLHuBP012313@lava.sentex.ca> <20061125083605.GA17350@stud.fit.vutbr.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20061125083605.GA17350@stud.fit.vutbr.cz> X-Editor: vi X-OS: FreeBSD User-Agent: Mutt/1.5.11-fullermd.3 Cc: freebsd-performance@freebsd.org, Mike Tancsa Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Nov 2006 11:22:19 -0000 On Sat, Nov 25, 2006 at 09:36:05AM +0100 I heard the voice of Divacky Roman, and lo! it spake thus: > > hm.. now I am confused. the rule is that having I586_CPU improves > performance because optimized bzero/bcopy is included (its not > included if you only have I686_CPU). Haven't we been by this before? It's not included even if you have I586_CPU either. See src/sys/i386/isa/npx.c, line 432: #ifdef I586_CPU_XXX <-------- ^^^ This has been disabled since r1.95, in 2001 (in 5-CURRENT days). There may be SOMETHING about including I586_CPU that speeds things up, but it ain't that. -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. From owner-freebsd-performance@FreeBSD.ORG Sat Nov 25 19:12:48 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CB93C16A403 for ; Sat, 25 Nov 2006 19:12:48 +0000 (UTC) (envelope-from linicks@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.190]) by mx1.FreeBSD.org (Postfix) with ESMTP id CADF743D55 for ; Sat, 25 Nov 2006 19:11:58 +0000 (GMT) (envelope-from linicks@gmail.com) Received: by nf-out-0910.google.com with SMTP id x37so1549572nfc for ; Sat, 25 Nov 2006 11:12:46 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=cMUN6fFTBi/rjz6jMMj9RcyHWsjr1QDo7p9usRxqfojqRjc10vriOX56pIk6/4PJvAxahA8jXMGL3GdDcqfdq6ham3e1Hndg9zBu/Iv/V4MIVB3qPAGF7JiJ0wAYVn8GlwG/ztKv1lJzdVZoWlT8PIyfbbMXQrj7R/yvVAxVLQ4= Received: by 10.82.142.9 with SMTP id p9mr1743428bud.1164481966192; Sat, 25 Nov 2006 11:12:46 -0800 (PST) Received: by 10.82.174.13 with HTTP; Sat, 25 Nov 2006 11:12:45 -0800 (PST) Message-ID: Date: Sun, 26 Nov 2006 02:12:45 +0700 From: "Nick Pavlica" To: "Mike Tancsa" In-Reply-To: <200611231652.kANGqJsr005016@lava.sentex.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <4557CECD.2000609@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> Cc: freebsd-performance@freebsd.org, Jeremie Le Hen Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Nov 2006 19:12:48 -0000 > I might give OpenBSD a quick try as a reference. Mike, Have you done any testing on Solaris 10, or OpenSolaris? I understand that it has a very robust IP stack. It would be interesting to see how the three stack up against each other (FBSD, LINUS, SunOS). --Nick From owner-freebsd-performance@FreeBSD.ORG Sat Nov 25 22:31:21 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BF3E516A403 for ; Sat, 25 Nov 2006 22:31:21 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2A62C43D46 for ; Sat, 25 Nov 2006 22:30:30 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.13.6/8.13.6) with ESMTP id kAPMVKGU069232; Sat, 25 Nov 2006 17:31:20 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAPMVJKX018453 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 25 Nov 2006 17:31:19 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611252231.kAPMVJKX018453@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Sat, 25 Nov 2006 17:31:27 -0500 To: "Nick Pavlica" From: Mike Tancsa In-Reply-To: References: <4557CECD.2000609@samsco.org> <200611130454.kAD4sZwe041556@lava.sentex.ca> <4557FF7A.8020704@samsco.org> <200611132054.kADKsFvK045726@lava.sentex.ca> <4558E3DC.6080800@samsco.org> <200611200454.kAK4sdat083568@lava.sentex.ca> <7.1.0.9.0.20061120160757.14d4a728@sentex.net> <200611220247.kAM2l9JP095066@lava.sentex.ca> <20061122130947.GM20405@obiwan.tataz.chchile.org> <200611231652.kANGqJsr005016@lava.sentex.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV version 0.88.3, clamav-milter version 0.88.3 on clamscanner2 X-Virus-Status: Clean Cc: freebsd-performance@freebsd.org Subject: Re: em forwarding performance (was Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Nov 2006 22:31:21 -0000 At 02:12 PM 11/25/2006, Nick Pavlica wrote: >>I might give OpenBSD a quick try as a reference. > >Mike, > Have you done any testing on Solaris 10, or OpenSolaris? I >understand that it has a very robust IP stack. It would be >interesting to see how the three stack up against each other (FBSD, >LINUS, SunOS). I installed OpenSolaris elsewhere this week. Yeah, might be interesting to try out. I will give it a test on Monday if the NICs are supported. ipfilter seems to be the firewall software. ---Mike