From owner-freebsd-arch@FreeBSD.ORG Sat Apr 25 16:31:32 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54790EBF for ; Sat, 25 Apr 2015 16:31:32 +0000 (UTC) Received: from mail-ig0-x230.google.com (mail-ig0-x230.google.com [IPv6:2607:f8b0:4001:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 21BE71080 for ; Sat, 25 Apr 2015 16:31:32 +0000 (UTC) Received: by igbpi8 with SMTP id pi8so45466962igb.0 for ; Sat, 25 Apr 2015 09:31:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=tIa+mu/cO/iNmtzmsjft++fAa/aSgJUkEFjwJR6LJKU=; b=n9STZDhqTfUE6KleAF3oPHiQOpUVhXfdiS30V4Z19iXuDJnnwZlUjlKx8n+dCuWc6w lNtrJjY/IEQqr8588dNxfqlyQW5xwX9pZipolvbvIjvHRdANINvq8Z/azIGxBoYblffU kwnH5FkRoxOb71zULxZBWtnAO8AdXzl8Yb8RcR2vwkekUR/ZmatE/j+KDT+kq+Sz9d2l ph/lzLnB1DgHbquiC5sKfB/NioS3Fpuh5x926x0GHK26Saupbo8w5qkjJgFnouUDVS3t PbHJPVcZPtDsIgeT9NG0SpoyMQNMWKYvpz2VsrtGnGJSP2WDvf9IJFkLuQ2ryWTm7RjN hYXA== MIME-Version: 1.0 X-Received: by 10.107.155.13 with SMTP id d13mr4564178ioe.29.1429979491562; Sat, 25 Apr 2015 09:31:31 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.38.133 with HTTP; Sat, 25 Apr 2015 09:31:31 -0700 (PDT) Date: Sat, 25 Apr 2015 09:31:31 -0700 X-Google-Sender-Auth: lVzMIvQwk5kpojJj4C_FbI7La4g Message-ID: Subject: RFC: setting performance_cx_lowest=C2 in -HEAD to avoid lock contention on many-CPU boxes From: Adrian Chadd To: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Apr 2015 16:31:32 -0000 Hi! I've been doing some NUMA testing on large boxes and I've found that there's lock contention in the ACPI path. It's due to my change a while ago to start using sleep states above ACPI C1 by default. The ACPI C3 state involves a bunch of register fiddling in the ACPI sleep path that grabs a serialiser lock, and on an 80 thread box this is costly. I'd like to drop performance_cx_lowest to C2 in -HEAD. ACPI C2 state doesn't require the same register fiddling (to disable bus mastering, if I'm reading it right) and so it doesn't enter that particular serialised path. I've verified on Westmere-EX, Sandybridge, Ivybridge and Haswell boxes that ACPI C2 does let one drop down into a deeper CPU sleep state (C6 on each of these). I think is still a good default for both servers and desktops. If no-one has a problem with this then I'll do it after the weekend. Thanks! -adrian