From owner-freebsd-stable@freebsd.org Mon Aug 14 16:48:09 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DC12DD6717 for ; Mon, 14 Aug 2017 16:48:09 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-vk0-x235.google.com (mail-vk0-x235.google.com [IPv6:2607:f8b0:400c:c05::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 33F0E65059; Mon, 14 Aug 2017 16:48:09 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-vk0-x235.google.com with SMTP id d124so33319859vkf.2; Mon, 14 Aug 2017 09:48:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=rxZWWNDCnCnoEl3ZLCbvenvQI+AJxE660EEynETUFlI=; b=QsniFvR9ZU5UBXjE/OsdvYUgdp+HA5jxMRqBsxmXAOQj+DEMsvlgr1lkLIel7X2WkH OO2VwkTp+zWg46buuzxRY93Cu2AYyuykoefqFPU35drTyAcQ3DSk10Z9WN9cVwlFJlAe D1+rHsYdxpmBUU78l+Fgra6CcDn9Wn7gL0HboyBc1t7SfOgrh3aYX17zvRGpHc+uWvsF A4jKpYeHwLDG2ulAI3ZuGiN2rOJ8mZgrOWauw5/Y7kCx6FmFRxIw4srNQvqxMkPU7gqd yy5xs3Tml/ooLNutbYvJR7khReLUz3F1FUfe7AbUNXLdixDhZRCP9P/11NcYEKxuRUMh IxLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=rxZWWNDCnCnoEl3ZLCbvenvQI+AJxE660EEynETUFlI=; b=hxbL2nGlyjzcFpSb79JvE5zTDkzmLjddMsd2n8MSwU1yE1PHCIGYqlVfGannPdHmdD OzTaekchvyLxdvJdrKGFk+R5zeQNfJzL/HVz0KbWia0OvxMYOd6GIE77lDu+mbSjvoDR +0QDW272eRqn133pThrGjfKVXdfiCJ1ufVkAJ1rSRZFjlLm4hFIDwVY2yylOSz9q6gHz ets/xzpxIPgXvsuoGyTkuqCf6NSIL5RwJv5kCeL+zKbTHQjmFQPFWGO4voIxouzLiZrZ uFOw7JZtxhr/m2ZNH2IMa3Bu9Jcq6hSqx2ZGzGkGddWvW87z/qzN2gJgscoPeylUk7hr SvvA== X-Gm-Message-State: AHYfb5jjVyKopwiat97u3G4fhXmER5vreEse5e9IFlPba/HRd80dQ0zN WRB0PpXC90bh/Iv8aqRD42lbJt6MHCXm X-Received: by 10.31.69.8 with SMTP id s8mr15792590vka.102.1502729288195; Mon, 14 Aug 2017 09:48:08 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.103.76.143 with HTTP; Mon, 14 Aug 2017 09:48:07 -0700 (PDT) In-Reply-To: <20170815010943.B12950@sola.nimnet.asn.au> References: <2ef99f5e-46f5-a185-2ac3-67d6afe68c89@ish.com.au> <20170815010943.B12950@sola.nimnet.asn.au> From: Kevin Oberman Date: Mon, 14 Aug 2017 09:48:07 -0700 X-Google-Sender-Auth: 4vjWzoF1YrnmUqvjOaKZ7S4KjRY Message-ID: Subject: Re: TSC timekeeping and cpu states To: Ian Smith Cc: Aristedes Maniatis , freebsd-stable , Alexander Motin Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Aug 2017 16:48:09 -0000 On Mon, Aug 14, 2017 at 8:38 AM, Ian Smith wrote: > On Mon, 14 Aug 2017 17:16:22 +1000, Aristedes Maniatis wrote: > > > On 14/8/17 3:08PM, Kevin Oberman wrote: > > > Again, the documentation lags reality. The default was changed for > > > 11.0. It is still conservative. In ALMOST all cases, Cmax will yield > > > the bast results. However, on large systems with many cores, Cmax > > > will trigger very poor results, so the default is C2, just to be > > > safe. > > Given it's a server, anything beyond C2 is likely not worth trying. > OTOH, C2 is perhaps not worth avoiding; it's probably low latency and > should result in lower power consumption, so heat, and unlikely to hurt. > > Or at least, I suspect that's the case .. cc'ing Alexander, as the wiki > article you referenced was his doing, so he's among those best placed. > > > > As far as possible TSC impact, I think older processors had TSC > > > issues when not all cores ran with the same clock speed. That said, > > > I am not remotely expert on such issues, so don't take this too > > > seriously. > > I wasn't aware that FreeBSD could yet do different freqs on different > cores? But I'm less expert than Kevin, and certainly behind the times. > > > Thanks Kevin > > > > What does 'large' and 'many cores' mean here? Is 24 cores large or > > small? For a server do we ever want the CPU to enter states other > > than C1? > > If C2 works well on your box, I don't see why you wouldn't want to use > it .. but others might. I have no personal experience beyond 2 cores, > but I'm perennially curious about such issues, as Kevin knows :) > > Are you using powerd? And what says, for example: > > sysctl -a | egrep 'cx|available|_freq|freq_|choice' | grep -v net\. > > which should include your timecounter and eventtimer setup too. > > cheers, Ian > I guess I need to clarify. No, FreeBSD does not have the ability to tun different cores at different frequencies. I seem to recall that TCC on some processors could adjust the frequency of a core exceeding a defined temperature, skipping N of every 8 clock cycles to slow the processor and reduce the temperature. This is what TCC was designed for. It is entirely possible that I am not correctly remembering the details of the issue, but it could only be resolved by switching from TCC to another clocking system. If memory serves, and it may not, there was an issue a few years ago (jhb@ worked the issue) where TSC was varying with frequency and that caused clock drift. I believe all "modern" processors do not have this issue and it seems unlikely that any system running 24 cores is old enough that this might be an issue. Sorry for any confusion I may have caused.