From owner-freebsd-arch@FreeBSD.ORG Sat Apr 25 18:18:08 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 05181750; Sat, 25 Apr 2015 18:18:08 +0000 (UTC) Received: from mail-lb0-x229.google.com (mail-lb0-x229.google.com [IPv6:2a00:1450:4010:c04::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 815AA1B4A; Sat, 25 Apr 2015 18:18:07 +0000 (UTC) Received: by lbbqq2 with SMTP id qq2so57013677lbb.3; Sat, 25 Apr 2015 11:18:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=GHeXNbqzfUuv+fBZ5fmKo4vyza8SH9sFKTYoUEekyzs=; b=zxeuEuh+zjP7zQ3o90/ipSnKAmy4q3khcj6Jk/D9or3gpI0ee/aa2r4MP04HebRqkC Sk6EkjJiJ86LWLwqPG4Bgqq3Qjr8rMEqUIIIqqr759wl+4MAX26MJljAuaUbKRFWXExH s1eVt6H/hGq6aAcIpUgtdboXUcuasFAaopRPK/0qj2P4WYwUIlP0eJfFUhrjM104XtbP 9EQI5dAtU/RHn6Iiqik7DQQSAmuuAwUS3nM01GFc2j+Z1Yg1SWoQJ5tV6TLHHPaOqL6D hGHKe71n+/k6JXz6OHApEiRVO2I18seTg6ZXlmwKl0PkE2MCT5aTtZ8tqc458dxfqe1I LkxQ== MIME-Version: 1.0 X-Received: by 10.152.234.139 with SMTP id ue11mr3597627lac.28.1429985885670; Sat, 25 Apr 2015 11:18:05 -0700 (PDT) Sender: davide.italiano@gmail.com Received: by 10.25.88.77 with HTTP; Sat, 25 Apr 2015 11:18:05 -0700 (PDT) In-Reply-To: References: Date: Sat, 25 Apr 2015 11:18:05 -0700 X-Google-Sender-Auth: 4sDDFijODMBR-lRmfxQUMCV5F4M Message-ID: Subject: Re: RFC: setting performance_cx_lowest=C2 in -HEAD to avoid lock contention on many-CPU boxes From: Davide Italiano To: Adrian Chadd Cc: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Apr 2015 18:18:08 -0000 On Sat, Apr 25, 2015 at 9:31 AM, Adrian Chadd wrote: > Hi! > > I've been doing some NUMA testing on large boxes and I've found that > there's lock contention in the ACPI path. It's due to my change a > while ago to start using sleep states above ACPI C1 by default. The > ACPI C3 state involves a bunch of register fiddling in the ACPI sleep > path that grabs a serialiser lock, and on an 80 thread box this is > costly. > > I'd like to drop performance_cx_lowest to C2 in -HEAD. ACPI C2 state > doesn't require the same register fiddling (to disable bus mastering, > if I'm reading it right) and so it doesn't enter that particular > serialised path. I've verified on Westmere-EX, Sandybridge, Ivybridge > and Haswell boxes that ACPI C2 does let one drop down into a deeper > CPU sleep state (C6 on each of these). I think is still a good default > for both servers and desktops. > > If no-one has a problem with this then I'll do it after the weekend. > This sounds to me just a way to hide a problem. Very few people nowaday run on NUMA and they can tune the machine as they like when they do testing. If there's a lock contention problem, it needs to be fixed and not hidden under another default. Also, as already noted this is a problem on 80-core machines but probably not on a 2-core Atom. I think you need to understand factors better and come up with a more sensible relation. In other words, your bet needs to be proven before changing a default useful for frew that can impact many. -- Davide "There are no solved problems; there are only problems that are more or less solved" -- Henri Poincare