From owner-freebsd-net@FreeBSD.ORG Mon Apr 18 21:15:49 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C50651065674 for ; Mon, 18 Apr 2011 21:15:49 +0000 (UTC) (envelope-from if@xip.at) Received: from chile.gbit.at (ns1.xip.at [193.239.188.99]) by mx1.freebsd.org (Postfix) with ESMTP id 2ACD68FC24 for ; Mon, 18 Apr 2011 21:15:48 +0000 (UTC) Received: (qmail 26109 invoked from network); 18 Apr 2011 23:09:06 +0200 Received: from unknown (HELO filebunker.xip.at) (89.207.145.147) by chile.gbit.at with (DHE-RSA-AES256-SHA encrypted) SMTP; 18 Apr 2011 23:09:06 +0200 Date: Mon, 18 Apr 2011 23:09:06 +0200 (CEST) From: Ingo Flaschberger To: "K. Macy" In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (LRH 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Cc: freebsd-net@freebsd.org Subject: Re: Routing enhancement - reduce routing table locking X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Apr 2011 21:15:49 -0000 > It occurred to me that I should add a couple of qualifications to the > previous statements. 1.6Mpps is line rate for GigE and I only know of > it to be achievable by igb hardware. The most I've seen em hardware > achieve is 1.1Mpps. Furthermore, in order to achieve that you would > have to enable IFNET_MULTIQUEUE in the driver, because by default the > driver uses the traditional (slow) IFQ as opposed overloading > if_transmit and doing its own queueing when needed. Support for > efficient multi-queue software queueing is provided by buf_ring, a > lock-free multi-producer ring buffer written just for this purpose. > > Thus, the fairly low transmit rate may be attributable to driver locking. Currently the quad core hardware is in production, I can only test with the single core 1,2ghz pentiumM. Also no igb cards. em cards, 82541GI, with polling 8.2 i386 with patch rmlock-copy, 400k /32 routes, 64byte packets: fastfw standard flowtable 1 dest: 85607pps 57189pps 57216pps rand. dest: 83945pps 54976pps 55007pps standard routing seems to be as fast as flowtable. flowtable does not support fastforward, rmlock-copy-patch supports fastforward. 8.2 i386 w/o patch, 400k /32 routes, 64byte packets: fastfw standard flowtable 1 dest: 84792pps 55357pps 55515pps rand. dest: 80156pps 52320pps 52300pps so even on a single cpu system less locking improve performance, but as you mentioned above, the bottlenecks of this system are the "desktop" pci network cards. I would really like to see some compareable tests with 10gbe hardware. Kind regards, Ingo Flaschberger