From owner-freebsd-net@FreeBSD.ORG Mon Mar 19 12:17:01 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F37E1065670 for ; Mon, 19 Mar 2012 12:17:01 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6BF4C8FC16 for ; Mon, 19 Mar 2012 12:17:01 +0000 (UTC) Received: by dald2 with SMTP id d2so10603852dal.13 for ; Mon, 19 Mar 2012 05:17:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=DOagoC1IXI1AfQtoaI2qfcSRcIKVETG7HsGV8Hsdv2U=; b=Uv/eRMVD2mkC4l4qrv8rdqHuoRp+dESRm1n+CqWRHYid6rRruPYPjY2PW4Fb7sEDYJ 66UojHXxkBC/ggyZ5kLv/qZ40+Q5o1BHHA0hh9IWRxg1mSGbixpdlOsanGRe8SeBSc7i oENHl7AlaZ9gyObvcg9tvzCWDM5Kfv+WJMkaOpnh9OXCoaPPm4PcVM+c/+A8rWW5CTkC e/PgZgSzpqdmhxj4uqbNyI64UqWUWjPFPkjhu/QKM3RnhHk7kanCuxS+Zuf6iQ8fHDQC IPdni2fuqUcFeM9RByJ2A23DxuO6urmOQ4LJY5ZcG/ha+HDqYrpilNAbJ+aoohUQqZbu GC/g== Received: by 10.68.232.162 with SMTP id tp2mr39258064pbc.165.1332159420882; Mon, 19 Mar 2012 05:17:00 -0700 (PDT) Received: from pyunyh@gmail.com ([114.111.62.249]) by mx.google.com with ESMTPS id z5sm11248274pbn.35.2012.03.19.05.16.56 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 19 Mar 2012 05:17:00 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Mon, 19 Mar 2012 21:16:47 -0700 From: YongHyeon PYUN Date: Mon, 19 Mar 2012 21:16:47 -0700 To: Andreas Longwitz Message-ID: <20120320041647.GE7518@michelle.cdnetworks.com> References: <4F594856.3030303@incore.de> <20120312211907.GC3671@michelle.cdnetworks.com> <4F5E0AF7.30302@incore.de> <20120313202559.GA3360@michelle.cdnetworks.com> <4F66646E.4080603@incore.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F66646E.4080603@incore.de> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org Subject: Re: Intel 82550 Pro/100 Ethernet and Microcode X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2012 12:17:01 -0000 On Sun, Mar 18, 2012 at 11:40:46PM +0100, Andreas Longwitz wrote: > YongHyeon PYUN wrote: > > > > The microcode is normally used to reduce high number of interrupts > > under heavy network load by bundling multiple RX frames. > > ok, I understand. But I have DEVICE_POLLING and HZ=1000 in the kernel > source, so interrupts do not occur. > Good data point. I'll try that on my box. > > However > > your reason to use microcode for i82550C looks weird since the > > microcode used for i82550C does not have a fix for TCO bug. > > The microcode for i82550(fxp2 in your system) indeed has fix for > > TCO bug and includes additional feature for bundling. > > According to the comments in rcvbundl.h it is just directly opposed. > > > If you're > > suffering from TCO bug of i82550, NFS over UDP issue should happen > > only on i82550(fxp2). Can you check whether the NFS issue happens > > on i82550C (fxp0 and fxp1) without loading the microcode? > > The NFS issue happens on fxp0 (i82550C) without loading of microcode: > Mar 16 16:12:52 dssresv1 kernel: nfs server > dsscam:/prod/mobotix: not responding > Mar 16 16:13:24 dssresv1 kernel: nfs server > dsscam:/prod/mobotix: not responding > Mar 16 16:13:56 dssresv1 kernel: nfs server > dsscam:/prod/mobotix: not responding > Mar 16 16:14:28 dssresv1 kernel: nfs server > dsscam:/prod/mobotix: not responding > Mar 16 16:14:59 dssresv1 kernel: nfs server > dsscam:/prod/mobotix: is alive again > I didn't ever try NFS on i82550C. If it can't handle fragmented IP datagrams, it would also have failed netperf UDP stream test since all UDP datagrams are fragmented. > The last message appears immediately after loading the microcode with > ifconfig fxp0 link0. > > > I still can't explain why your i82550C with the loaded microcode > > does not generates SCB timeouts because mine always shows the error > > right after loading the microcode. > > In FreeBSD 6 and 8 I need my bzero-patch to get fxp working, In FreeBSD > 4 - on the same hardware - this was not necessary. > > > Are you actively using fxp0 or fxp1 after loading the microcode? > > Yes I have 6 server with fxp0/fxp1 i82550C and 2 server with fxp0/fxp1 > i82550 on motherboard in production, additionally some fxp2 i82550 > external cards. All run with microcode loaded and polling. > > > If yes, could you check whether > > the CPU Saver feature of the microcode really works on i82550C? > > You may be able to use netperf UDP stream test to verify that. > > I did some tests with netperf 2.5.0, At first I had to patch some format > strings in netperf from %d to %lld for uint64_t variables to get > readable output from netperf on my i386 systems. > Yeah, that is one of bug of netperf. > The test command > netperf -H host_with_fxp -t UDP_STREAM > gives nearly always the same output > > Socket Message Elapsed Messages > Size Size Time Okay Errors Throughput > bytes bytes secs # # 10^6bits/sec > > 9216 9216 10.00 13069 1515880 96.34 > 41600 10.00 13069 96.34 > > And this output did not considerably change for the test cases fxp-type > i82550C or i82550, microcode loaded or not and polling yes or no. > > But I can answer your question concerning the CPU Saver funktion on > i82550C and its a little suprise (for me). In all my tests without > polling I checked the irq's on host_with_fxp using "vmstat -i". The > result is that the CPU Saver feature of the microcode does not work on > i82250C and it works on i82250. On i82250C the value of > dev.fxp.0.bundle_max is irrelevant, the i82250C always needs ca. 91000 > irq's for the netperf test. The same number of irq's needs the i82250 > with bundle_max=1, but when I set bundle_max=6 (the default), then the > number of irq's for the same test goes down to 91000/7 = 13000. Thanks for the detailed information. This indicates the microcode for i82550C does not have bundling feature. By chance, did you ever update your controller firmware to newest one? I don't remember whether I also updated controller firmware for i82550C but I used to update several i82559 controllers for PXE. > > > Regards > > Andreas Longwitz >