From owner-freebsd-net@FreeBSD.ORG Wed Dec 21 15:55:14 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A1061065678 for ; Wed, 21 Dec 2011 15:55:14 +0000 (UTC) (envelope-from clint@servedomains.com) Received: from loki.servedomains.com (loki.servedomains.com [66.133.105.45]) by mx1.freebsd.org (Postfix) with ESMTP id 40A398FC1B for ; Wed, 21 Dec 2011 15:55:12 +0000 (UTC) Received: from loki.servedomains.com (mail [192.168.0.60]) by loki.servedomains.com (Postfix) with ESMTP id E2149278975; Wed, 21 Dec 2011 08:55:11 -0700 (MST) Received: from mail.servedomains.com (localhost [127.0.0.1]) by loki.servedomains.com (Postfix) with ESMTP id B0925278951 for ; Wed, 21 Dec 2011 08:55:11 -0700 (MST) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 21 Dec 2011 08:55:11 -0700 From: Clint To: In-Reply-To: <20111221153456.GA30629@onelab2.iet.unipi.it> References: <2f23773b5165887ac51d19a612bf4c93@servedomains.com> <20111220224814.GA7715@michelle.cdnetworks.com> <5b246a7cf0bceedf39ff5718336b0753@servedomains.com> <91610121f172c87d5095c8ac03ea9837@servedomains.com> <20111221153456.GA30629@onelab2.iet.unipi.it> Message-ID: <49223118e5e2ca35a395959ef8ce4e91@servedomains.com> X-Sender: clint@servedomains.com User-Agent: Roundcube Webmail/0.5.3 X-Virus-Scanned: ClamAV via loki.servedomains.com Subject: Re: igb performance/load udp issue X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Dec 2011 15:55:14 -0000 On Wed, 21 Dec 2011 16:34:56 +0100, Luigi Rizzo wrote: > On Wed, Dec 21, 2011 at 08:04:54AM -0700, Clint wrote: >> On Tue, 20 Dec 2011 22:46:46 -0800, Jack Vogel wrote: >> >I was fighting with UDP issues before the latest checkin, so you >> >should >> >look at THAT version, 2.3.1 in HEAD please. >> > >> >Jack >> > >> >> I'm actually running the HEAD version now, but my problems still >> persist. I did notice that the excessive load is now spread across >> multiple cores where before it was isolated to one core at a time. >> This >> version also doesn't suffer from the horrible performance I >> experienced >> with v2.2.5. > > as someone mentioned, it may have to do with the sender > using multiple queues thus causing reordering in the frames, > which then the receiver has to sort (and often, the receive > queue is a list so O(N) complexity per packet.) > > Can you run the igb driver with just a single queue ? > In the bootloader, or before loading the module using "kenv", you > should set > > hw.igb.num_queues = 1 > > This should prevent gratuitous reordering and perhaps fix > your performance. > > cheers > luigi I've already tried that as well, but with the v2.2.3 driver, didn't help. I'll try it again with the HEAD driver tonight. Thanks, Clint