From owner-freebsd-performance@FreeBSD.ORG Tue Feb 1 11:37:27 2011 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB9151065673; Tue, 1 Feb 2011 11:37:27 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 2DE008FC17; Tue, 1 Feb 2011 11:37:27 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1PkEY0-000GWj-WF; Tue, 01 Feb 2011 14:37:25 +0300 Date: Tue, 1 Feb 2011 14:37:24 +0300 From: Slawa Olhovchenkov To: Bruce Evans Message-ID: <20110201113724.GS18170@zxy.spb.ru> References: <20110128143355.GD18170@zxy.spb.ru> <22E77EED-6455-4164-9115-BBD359EC8CA6@moneybookers.com> <20110128161035.GF18170@zxy.spb.ru> <4D42F87C.7020909@freebsd.org> <20110128172516.GG18170@zxy.spb.ru> <20110129070205.Q7034@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110129070205.Q7034@besplex.bde.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: freebsd-performance@freebsd.org, Julian Elischer , Stefan Lambrev Subject: Re: Interrupt performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Feb 2011 11:37:27 -0000 On Sat, Jan 29, 2011 at 07:52:11AM +1100, Bruce Evans wrote: > >> there are profiling tools that you may decide to run. > > > > What tools I can use on amd64? > > > > I boot kernel configured with 'config -p'. > > Most time in spinlock_exit and acpi_cpu_c1. > > Normal profiling works poorly (I see you found my old mail about high > resolution profiling). Linux might be misreporting the overhead for > exactly the same reasons that normal profiling works poorly: > - the profiling clock frequency of ~1 KHz was adequate for 5 MHz machines > in 1998, but is now too slow. Statistics clocks are even slower (128 > Hz in FreeBSD, and possibly 100 Hz (?) jiffies in Linux). > - the statistics clock might be too synchronized with other interrupts. > The above spinlock_exit and acpi_cpu_c1 times indicate that the > statistics clock almost always fires on exit from another spinlock > and/or inside ACPI, for waking up from idle for the latter. Seeing > lots of exits from spinlocks may indicated that spinlocks are being > used too much. > But FreeBSD will report interrupt times and system for non-fast-interrupts > to an accuracy of about 1 microsecond, since it doesn't use the > statistics clock much for this. OTOH, for fast interrupts it is typical > behaviour in FreeBSD and Linux to not see them at all from the statistics > clock interrupt, since they mask all interrupts so they mask the > statistics clock interrupt in particular. In FreeBSD, lots of time > apparently spent in spinlock_exit is a typical result of this, or at > least similar things, since spinlock_enter masks all interrupts (except > in my version of course). Linux doesn't have fast interrupts in the > same way that FreeBSD does, but at least in old versions almost all of > its interrupts masked other interrupts a lot. I do some more test and build kernel with KTR. Now I don't think that inetrrupt overhead on FreeBSD weight: I try polling and don't see any difference. I see many reported by netperf send errors. I found this http://docs.freebsd.org/cgi/mid.cgi?E1Aice9-0002by-00. After insert into src/nettest_bsd.c usleep(1000) if ENOBUF I see 53% idle and ./loop 2000000000 "Elapsed 15188006 us" -- this near to linux (Elapsed 14107670 us). 10% of difference may be by more weight network stack (only 32104 ticks from 126136 in interrupt handler and task switching, and 94032 -- UDP processing in network stack and passing datagram to driver). May be weight SOCKBUF_LOCK/SOCKBUF_UNLOCK and/or INP_INFO_RUNLOCK/INP_RUNLOCK.