From owner-cvs-src@FreeBSD.ORG Sun Dec 24 09:43:31 2006 Return-Path: X-Original-To: cvs-src@freebsd.org Delivered-To: cvs-src@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DC10C16A417; Sun, 24 Dec 2006 09:43:31 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1-3.pacific.net.au [61.8.2.210]) by mx1.freebsd.org (Postfix) with ESMTP id 76B0A13C48D; Sun, 24 Dec 2006 09:43:31 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout1.pacific.net.au (Postfix) with ESMTP id 1C3595A3CC6; Sun, 24 Dec 2006 20:43:30 +1100 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (Postfix) with ESMTP id A70572740D; Sun, 24 Dec 2006 20:43:28 +1100 (EST) Date: Sun, 24 Dec 2006 20:43:22 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Scott Long In-Reply-To: <458E10B0.5030506@samsco.org> Message-ID: <20061224185620.M25303@delplex.bde.org> References: <20061223213014.U35809@fledge.watson.org> <20061224120307.P24444@delplex.bde.org> <458E10B0.5030506@samsco.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: cvs-src@freebsd.org, src-committers@freebsd.org, Robert Watson , cvs-all@freebsd.org, John Polstra Subject: Re: cvs commit: src/sys/dev/bge if_bge.c X-BeenThere: cvs-src@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Dec 2006 09:43:32 -0000 On Sun, 24 Dec 2006, Scott Long wrote: > Bruce Evans wrote: >> On Sat, 23 Dec 2006, Robert Watson wrote: >> >>> On Sat, 23 Dec 2006, John Polstra wrote: >>> >>>>> That said, dropping and regrabbing the driver lock in the rxeof routine >>>>> of any driver is bad. It may be safe to do, but it incurs horrible >>>>> performance penalties. It essentially allows the time-critical, high >>>>> priority RX path to be constantly preempted by the lower priority >>>>> if_start or if_ioctl paths. Even without this preemption and priority >>>>> inversion, you're doing an excessive number of expensive lock ops in the >>>>> fast path. >> >> It's not very time-critical or high priority for bge or any other device >> that has a reasonably large rx ring. With a ring size of 512 and an rx >> interrupt occuring not too near the end (say at half way), you have 256 >> packet times to finish processing the interrupt. For normal 1518 byte >> packets at 1Gbps, 256 packet times is about 3 mS. bge's rx ring size >> is actually larger than 512 for most hardware. > > Speed testing full size packets doesn't tax the hardware or the OS, nor > does it represent real world scenarios. Testing minimum sized packets > isn't terribly real-world either, but it represents the worst-case for > the hardware and the OS and thus is a good standard for extrapolating > performance potential. And, 1Gb isn't terribly interesting these days > either, 10Gb is. I left calculating the timing for minimum size packets as an exercise. It's also not very interesting, because there are no device/bus/CPU combinations that can keep up with it, even at 1Gbps, due to overhead issues that push the latency issues further off. The maximum packet rate in theory for 1Gbps is 2700 kpps (46 byte packets). I believe the maximum packet rate in practice for anyGbps with a single 3-4GHz CPU is about 1000 kpps. With an rx ring size of 1024 and an interrupt when the ring has 24 descriptors in it (the default is 10, which is far too small, but that is another bug), this leaves 1 mS of latency to space. 3 mS was eternity, and 1 mS is still eternity. Bruce