From owner-freebsd-net@FreeBSD.ORG Sun Mar 18 22:40:57 2012 Return-Path: <owner-freebsd-net@FreeBSD.ORG> Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 18E93106566B for <freebsd-net@freebsd.org>; Sun, 18 Mar 2012 22:40:57 +0000 (UTC) (envelope-from longwitz@incore.de) Received: from dss.incore.de (dss.incore.de [195.145.1.138]) by mx1.freebsd.org (Postfix) with ESMTP id 9D1078FC08 for <freebsd-net@freebsd.org>; Sun, 18 Mar 2012 22:40:56 +0000 (UTC) Received: from inetmail.dmz (inetmail.dmz [10.3.0.3]) by dss.incore.de (Postfix) with ESMTP id D73DA5D29F; Sun, 18 Mar 2012 23:40:48 +0100 (CET) X-Virus-Scanned: amavisd-new at incore.de Received: from dss.incore.de ([10.3.0.3]) by inetmail.dmz (inetmail.dmz [10.3.0.3]) (amavisd-new, port 10024) with LMTP id FnDmzidkQTzB; Sun, 18 Mar 2012 23:40:47 +0100 (CET) Received: from mail.incore (fwintern.dmz [10.0.0.253]) by dss.incore.de (Postfix) with ESMTP id DC2AE5D29D; Sun, 18 Mar 2012 23:40:47 +0100 (CET) Received: from bsdmhs.longwitz (unknown [192.168.99.6]) by mail.incore (Postfix) with ESMTP id 38CDA450A8; Sun, 18 Mar 2012 23:40:47 +0100 (CET) Message-ID: <4F66646E.4080603@incore.de> Date: Sun, 18 Mar 2012 23:40:46 +0100 From: Andreas Longwitz <longwitz@incore.de> User-Agent: Thunderbird 2.0.0.19 (X11/20090113) MIME-Version: 1.0 To: pyunyh@gmail.com References: <4F594856.3030303@incore.de> <20120312211907.GC3671@michelle.cdnetworks.com> <4F5E0AF7.30302@incore.de> <20120313202559.GA3360@michelle.cdnetworks.com> In-Reply-To: <20120313202559.GA3360@michelle.cdnetworks.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org Subject: Re: Intel 82550 Pro/100 Ethernet and Microcode X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net> List-Post: <mailto:freebsd-net@freebsd.org> List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sun, 18 Mar 2012 22:40:57 -0000 YongHyeon PYUN wrote: > > The microcode is normally used to reduce high number of interrupts > under heavy network load by bundling multiple RX frames. ok, I understand. But I have DEVICE_POLLING and HZ=1000 in the kernel source, so interrupts do not occur. > However > your reason to use microcode for i82550C looks weird since the > microcode used for i82550C does not have a fix for TCO bug. > The microcode for i82550(fxp2 in your system) indeed has fix for > TCO bug and includes additional feature for bundling. According to the comments in rcvbundl.h it is just directly opposed. > If you're > suffering from TCO bug of i82550, NFS over UDP issue should happen > only on i82550(fxp2). Can you check whether the NFS issue happens > on i82550C (fxp0 and fxp1) without loading the microcode? The NFS issue happens on fxp0 (i82550C) without loading of microcode: Mar 16 16:12:52 <kern.info> dssresv1 kernel: nfs server dsscam:/prod/mobotix: not responding Mar 16 16:13:24 <kern.info> dssresv1 kernel: nfs server dsscam:/prod/mobotix: not responding Mar 16 16:13:56 <kern.info> dssresv1 kernel: nfs server dsscam:/prod/mobotix: not responding Mar 16 16:14:28 <kern.info> dssresv1 kernel: nfs server dsscam:/prod/mobotix: not responding Mar 16 16:14:59 <kern.info> dssresv1 kernel: nfs server dsscam:/prod/mobotix: is alive again The last message appears immediately after loading the microcode with ifconfig fxp0 link0. > I still can't explain why your i82550C with the loaded microcode > does not generates SCB timeouts because mine always shows the error > right after loading the microcode. In FreeBSD 6 and 8 I need my bzero-patch to get fxp working, In FreeBSD 4 - on the same hardware - this was not necessary. > Are you actively using fxp0 or fxp1 after loading the microcode? Yes I have 6 server with fxp0/fxp1 i82550C and 2 server with fxp0/fxp1 i82550 on motherboard in production, additionally some fxp2 i82550 external cards. All run with microcode loaded and polling. > If yes, could you check whether > the CPU Saver feature of the microcode really works on i82550C? > You may be able to use netperf UDP stream test to verify that. I did some tests with netperf 2.5.0, At first I had to patch some format strings in netperf from %d to %lld for uint64_t variables to get readable output from netperf on my i386 systems. The test command netperf -H host_with_fxp -t UDP_STREAM gives nearly always the same output Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 9216 9216 10.00 13069 1515880 96.34 41600 10.00 13069 96.34 And this output did not considerably change for the test cases fxp-type i82550C or i82550, microcode loaded or not and polling yes or no. But I can answer your question concerning the CPU Saver funktion on i82550C and its a little suprise (for me). In all my tests without polling I checked the irq's on host_with_fxp using "vmstat -i". The result is that the CPU Saver feature of the microcode does not work on i82250C and it works on i82250. On i82250C the value of dev.fxp.0.bundle_max is irrelevant, the i82250C always needs ca. 91000 irq's for the netperf test. The same number of irq's needs the i82250 with bundle_max=1, but when I set bundle_max=6 (the default), then the number of irq's for the same test goes down to 91000/7 = 13000. Regards Andreas Longwitz