From owner-freebsd-hackers@FreeBSD.ORG Mon Aug 2 22:44:15 2010 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AE121065676 for ; Mon, 2 Aug 2010 22:44:15 +0000 (UTC) (envelope-from guy.helmer@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id E8CFD8FC19 for ; Mon, 2 Aug 2010 22:44:14 +0000 (UTC) Received: by wyj26 with SMTP id 26so4631466wyj.13 for ; Mon, 02 Aug 2010 15:44:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=SQLsGG7d2VK7fwKvxSwkEARxReiCYgkRuYNO9+fipEU=; b=uH/UEal6N8qRvk/XC5McjYT9YJben6LRNjplCVVB8ODF6OUpWpR8aTIPHxCxSuP/oV HfOfIFRj+C5uDFu1V7nKIwSqmP9BfZOWd+HIHkF+2zSElEwIvRlfoige/BRoizvNg6G9 oLvS49ihqO/R3ZAWZe/PaWNC+vMjeNwp0jYBk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=SetNAmXjibFCD3DS49kWfBGjeiUmRJwph1bU0Sz9tVptKkJFyBNfinFdWd+lBUdiK/ MnRL6FFr3WtjALYBolqH8r0t7lmG0KT+m7HfsXEMM2dS8EjanXGHhU1eP92b/+iV1tdd 4bNpGW3joNb1/FB+ap2zpBqQexpCQnYZKm4Y0= MIME-Version: 1.0 Received: by 10.216.89.149 with SMTP id c21mr5505061wef.73.1280787230435; Mon, 02 Aug 2010 15:13:50 -0700 (PDT) Received: by 10.216.126.13 with HTTP; Mon, 2 Aug 2010 15:13:50 -0700 (PDT) Date: Mon, 2 Aug 2010 17:13:50 -0500 Message-ID: From: Guy Helmer To: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Mon, 02 Aug 2010 22:53:01 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Puzzling performance X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Aug 2010 22:44:15 -0000 On a FreeBSD 7.1 SCHED_ULE kernel, I have a large number of files opened and mmapped (with MAP_NOSYNC option) for shared-memory communication between processes. Normally, memcpy() copies data into these shared-memory buffers in a reasonable amount of time closely related to the size of the copy (roughly 10us per 10KB). However, due to performance issues I've found that sometimes a memcpy() takes an abnormally long time (10ms for 40KB, and I suspect longer times occurring when I have not had monitoring enabled). The system doesn't seem to be in memory overcommit -- there is just a minor amount of swap in use, and I've not seen page-ins or page-outs while watching systat or vmstat. Since I'm using MAP_NOSYNC, I would not expect the pager to flush dirty pages to disk and cause add delays. Any ideas where to look? Might it help to pin threads to CPUs in case a thread is getting moved to a different core? Thanks, Guy