From owner-freebsd-net@FreeBSD.ORG Wed Oct 3 01:48:38 2007 Return-Path: Delivered-To: freebsd-net@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F70816A418 for ; Wed, 3 Oct 2007 01:48:38 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail14.syd.optusnet.com.au (mail14.syd.optusnet.com.au [211.29.132.195]) by mx1.freebsd.org (Postfix) with ESMTP id D534213C469 for ; Wed, 3 Oct 2007 01:48:37 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c220-239-235-248.carlnfd3.nsw.optusnet.com.au (c220-239-235-248.carlnfd3.nsw.optusnet.com.au [220.239.235.248]) by mail14.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id l931mXSj028618 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 3 Oct 2007 11:48:35 +1000 Date: Wed, 3 Oct 2007 11:48:33 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Vladimir Ivanov In-Reply-To: <470280F6.9070009@yandex-team.ru> Message-ID: <20071003111737.U14276@delplex.bde.org> References: <46B07931.3080300@yandex-team.ru> <2a41acea0708010923m7b21095ajc2ee84c37e0d5354@mail.gmail.com> <470280F6.9070009@yandex-team.ru> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: "freebsd-net@freebsd.org" , Jack Vogel Subject: Re: SMPable version of EM driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2007 01:48:38 -0000 On Tue, 2 Oct 2007, Vladimir Ivanov wrote: > Main improvement of this version: driver does not use TX interrupts at all. > So, interrupt rate reduced significantly. Polling for anything is a bug IMO. Buggy hardware may work better with it, but em is not buggy :-). For bge, I tune the interrupt moderation parameters to reduce the tx interrupt rate to almost as low as possible without doing polling. The rate is either 1 interrupt per second if the tx is almost inactive or 1 interrupt every 384 packets if the tx is active. -current mistunes these parameters to 150 (microseconds) and 10 (descriptos). Old tuning of 150 and 128 only loses a little compared with 1000000 and 384. (150 gives 6667 interrupts per second under load. This interrupt rate is quite manageable and is about the same rate as you have to use with polling to get the same throughput but lower efficiency as with interrupts. 128 for the descriptor limit causes in a max interrupt rate of only a few hundred per second except with tiny packets, but 10 is excessively small and requires a rate of up to 140000 per second to keep up with tiny packets. 140000 isn't manageable.) em has more/better interrupt parameters with non-broken defaults so I haven't needed to tune them. For bge, I implement dynamic rx interrupt moderation in software where em has it in hardware. 10000 interrupts/second for rx is a good limit. IIRC, em uses 8000 which is a bit low for a max, and is missing a sysctl for easy tuning. Bruce