From owner-freebsd-usb@FreeBSD.ORG Mon Oct 4 03:45:03 2010 Return-Path: Delivered-To: freebsd-usb@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C357106564A; Mon, 4 Oct 2010 03:45:03 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from out-0.mx.aerioconnect.net (outb.internet-mail-service.net [216.240.47.225]) by mx1.freebsd.org (Postfix) with ESMTP id 042DF8FC12; Mon, 4 Oct 2010 03:45:02 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id o943WwWp010893; Sun, 3 Oct 2010 20:32:58 -0700 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id AAF1D2D6011; Sun, 3 Oct 2010 20:32:57 -0700 (PDT) Message-ID: <4CA94B13.7020809@freebsd.org> Date: Sun, 03 Oct 2010 20:33:39 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.9) Gecko/20100915 Thunderbird/3.1.4 MIME-Version: 1.0 To: pyunyh@gmail.com References: <20101002001100.GL10521@michelle.cdnetworks.com> <201010020841.57474.hselasky@freebsd.org> <20101003235423.GB1135@michelle.cdnetworks.com> In-Reply-To: <20101003235423.GB1135@michelle.cdnetworks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Hans Petter Selasky , freebsd-usb@freebsd.org Subject: Re: Network TX/RX fairness is not honored by USB stack X-BeenThere: freebsd-usb@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD support for USB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Oct 2010 03:45:03 -0000 On 10/3/10 4:54 PM, Pyun YongHyeon wrote: > On Sat, Oct 02, 2010 at 08:41:57AM +0200, Hans Petter Selasky wrote: >> On Saturday 02 October 2010 02:11:00 Pyun YongHyeon wrote: >>> Hi, >>> >>> I don't know how long it had been there but it seems current USB >>> stack does not honor fairness of TX/RX on USB ethernet controller. >>> Unidirectional performance test(UDP) or most-unidirectional >>> performance(TCP) test works well without problems. However if heavy >>> TX/RX traffic hits controller at the same time either TX or RX is >>> not served at all. I'm under the impression that whenever TX work >>> is done it seems USB reschedules next pending TX again instead of >>> processing RX such that RX is starved to death. This can be easily >>> reproduced on two hosts with the netperf performance test. >>> Whenever both hosts send tiny UDP datagrams to the other host >>> either TX or RX packet counters are not increasing until the end >>> of the UDP torture test. The number of EHCI interrupt is about 8K/s >>> while test is in progress so I think it reached its maximum >>> processing limit. After netperf testing, it can still process TX/RX >>> packets even though it dropped too many RX packets. But these >>> dropped packets are not counted so netstat(1) shows 0 dropped >>> frames even though it lost millions of packets. >>> >>> Hans, do you have any idea what's going on here? >>> You can use the following netperf command on both hosts after >>> running netserver. >>> %netperf -c -H ip_addr_of_other_host -tUDP_STREAM -l 300 -- -m 1 >>> >>> Another odd thing I noticed is number of interrupts does not go >>> down to 0 after the testing. It constantly generates 1k/s >>> interrupts after that. >> Maybe we are triggering a bug. Can you enable USB debugging to figure out what >> data lengths are transmitted or received. >> > In the middle of testing? If yes, that would be meaningless as it > would generate bunch of messages. The test case generates payload > size 1 UDP datagrams with full speed so enabling debug messages > will change timing. Note, I'm exercising number of packets per > second, not number of bytes per second. > >> USB EHCI uses round robin, so this is either USB device problem or a test- >> program software failure. >> > I'm pretty sure the benchmark program is not broken, so either > axe(4) or USB stack could be wrong here. I see three issues from > the UDP torture test. > - Either TX or RX could be starved to death. If you start TX test > first, RX would be stuck. If you start RX test first, TX would > be stuck. > - The number of packets sent or received are much lower than > expected. > For TX case, the number of packets sent per second is exactly 8k > which is much less than that of non-USB controllers. For gigabit that is a big clue. the USB hardware uses an 8 thousand time per second clock for it's internal polling. > controllers number of TX packets could be several hundred > thousands per second. For RX packets it shows 14K/s packets with > 8K/s interrupts. I thought USB ethernet controllers can send > more than 8k packets per second. Because the number of > interrupts per second and 8k packets per second is the same, > this also make me wonder there could be some relations there. > - Number of interrupts does not go back to 0 after the testing. > > I'll let you know if I find some clue but it may take long time as > I'm not familiar with USB stack. :-( > >> Check the CPU usage of the host computer during the test. Do you see anything? >> > I didn't notice odd thing except 8k/s interrupts. > >>> The only way I stop that interrupts was to >>> down the ue0 interface with "ifconfig ue0 down" command. >> --HPS > _______________________________________________ > freebsd-usb@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-usb > To unsubscribe, send any mail to "freebsd-usb-unsubscribe@freebsd.org" >