From owner-freebsd-hackers@FreeBSD.ORG Tue Aug 3 13:57:14 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ACCA11065687 for ; Tue, 3 Aug 2010 13:57:14 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 7FE698FC22 for ; Tue, 3 Aug 2010 13:57:14 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 38CE146B8C; Tue, 3 Aug 2010 09:57:14 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C20808A04E; Tue, 3 Aug 2010 09:57:08 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Tue, 3 Aug 2010 09:41:30 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100217; KDE/4.4.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201008030941.30649.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 03 Aug 2010 09:57:08 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Guy Helmer Subject: Re: Puzzling performance X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Aug 2010 13:57:14 -0000 On Monday, August 02, 2010 6:13:50 pm Guy Helmer wrote: > On a FreeBSD 7.1 SCHED_ULE kernel, I have a large number of files opened and > mmapped (with MAP_NOSYNC option) for shared-memory communication between > processes. Normally, memcpy() copies data into these shared-memory buffers > in a reasonable amount of time closely related to the size of the copy > (roughly 10us per 10KB). However, due to performance issues I've found that > sometimes a memcpy() takes an abnormally long time (10ms for 40KB, and I > suspect longer times occurring when I have not had monitoring enabled). The > system doesn't seem to be in memory overcommit -- there is just a minor > amount of swap in use, and I've not seen page-ins or page-outs while > watching systat or vmstat. > > Since I'm using MAP_NOSYNC, I would not expect the pager to flush dirty > pages to disk and cause add delays. Any ideas where to look? Might it help > to pin threads to CPUs in case a thread is getting moved to a different > core? Pinning might help yes. You might also want to ensure there aren't any interrupts on that CPU. Currently there isn't a good way to figure that out short of kgdb though. :( -- John Baldwin