From owner-freebsd-stable@FreeBSD.ORG Sun Jul 3 00:38:17 2005 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CAAB416A545; Sun, 3 Jul 2005 00:38:01 +0000 (GMT) (envelope-from ps@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8AFC7440EB; Sun, 3 Jul 2005 00:22:43 +0000 (GMT) (envelope-from ps@mu.org) Received: by elvis.mu.org (Postfix, from userid 1000) id CCEBA625B0; Sat, 2 Jul 2005 17:20:16 -0700 (PDT) X-Original-To: ps@mu.org Delivered-To: ps@mu.org Received: from mx2.freebsd.org (mx2.freebsd.org [216.136.204.119]) by elvis.mu.org (Postfix) with ESMTP id 35E645C985 for ; Thu, 18 Nov 2004 13:31:34 -0800 (PST) Received: from hub.freebsd.org (hub.freebsd.org [216.136.204.18]) by mx2.freebsd.org (Postfix) with ESMTP id 99DFD58220; Thu, 18 Nov 2004 21:31:27 +0000 (GMT) (envelope-from owner-freebsd-current@freebsd.org) Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 6307216A4F2; Thu, 18 Nov 2004 21:31:26 +0000 (GMT) Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 996E716A4CF; Thu, 18 Nov 2004 21:31:22 +0000 (GMT) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D81343D48; Thu, 18 Nov 2004 21:31:22 +0000 (GMT) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) iAILVKcW032004; Thu, 18 Nov 2004 13:31:20 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.9p2/8.12.9/Submit) id iAILVJ1T032003; Thu, 18 Nov 2004 13:31:19 -0800 (PST) (envelope-from dillon) From: Matthew Dillon Message-Id: <200411182131.iAILVJ1T032003@apollo.backplane.com> To: Wilko Bulte References: <200411172357.47735.Emanuel.Strobl@gmx.net> <20041118124902.GC75559@freebie.xs4all.nl> X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Sender: owner-freebsd-current@freebsd.org Errors-To: owner-freebsd-current@freebsd.org X-Spam-Checker-Version: SpamAssassin 3.0.0 (2004-09-13) on elvis.mu.org X-Spam-Status: No, score=-2.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.0.0 X-Spam-Level: Cc: Emanuel Strobl , Robert Watson , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: serious networking (em) performance (ggate and NFS) problem X-BeenThere: freebsd-stable@freebsd.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Date: Sun, 03 Jul 2005 00:38:17 -0000 X-Original-Date: Thu, 18 Nov 2004 13:31:19 -0800 (PST) X-List-Received-Date: Sun, 03 Jul 2005 00:38:17 -0000 Polling should not produce any improvement over interrupts for EM0. The EM0 card will aggregate 8-14+ packets per interrupt, or more. which is only around 8000 interrupts/sec. I've got a ton of these cards installed. # mount_nfs -a 4 dhcp61:/home /mnt # dd if=/mnt/x of=/dev/null bs=32k # netstat -in 1 input (Total) output packets errs bytes packets errs bytes colls 66401 0 93668746 5534 0 962920 0 66426 0 94230092 5537 0 1007108 0 66424 0 93699848 5536 0 963268 0 66422 0 94222372 5536 0 1007290 0 66391 0 93654846 5534 0 962746 0 66375 0 94154432 5532 0 1006404 0 zfod Interrupts Proc:r p d s w Csw Trp Sys Int Sof Flt cow 8100 total 1 9 6211 7 75 8100 4 12 88864 wire 7873 mux irq10 10404 act ata0 irq14 19.2%Sys 0.0%Intr 0.0%User 0.0%Nice 80.8%Idl 864476 inact ata1 irq15 | | | | | | | | | | 58152 cache mux irq11 ========== 2992 free 227 clk irq0 Note that the interrupt rate is only 7873 interrupts per second while I am transfering 94 MBytes/sec over NFS (UDP) and receiving over 66000 packets per second (~8 packets per interrupt). If I use a TCP mount I get just about the same thing: # mount_nfs -T -a 4 dhcp61:/home /mnt # dd if=/mnt/x of=/dev/null bs=32k # netstat -in 1 input (Total) output packets errs bytes packets errs bytes colls 61752 0 93978800 8091 0 968618 0 61780 0 93530484 8098 0 904370 0 61710 0 93917880 8093 0 968128 0 61754 0 93491260 8095 0 903940 0 61756 0 93986320 8097 0 968336 0 Proc:r p d s w Csw Trp Sys Int Sof Flt cow 8145 total 5 8 22828 13 5490 8146 13 11 141556 wire 7917 mux irq10 7800 act ata0 irq14 26.4%Sys 0.0%Intr 0.0%User 0.0%Nice 73.6%Idl 244872 inact ata1 irq15 | | | | | | | | | | 8 cache mux irq11 ============= 630780 free 228 clk irq0 In this case around 8000 interrupts per second with 61700 packet per second incoming on the interface (around ~8 packets per interrupt). The extra interrupts are due to the additional outgoing TCP ack traffic. If I look at the systat -vm 1 output on the NFS server it also sees only around 8000 interrupts per second, which isn't saying much other then it's transmit path (61700 pps outoging) is not creating an undue interrupt burden relative to the receive path. -Matt _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"