From owner-freebsd-stable@FreeBSD.ORG Sat Mar 21 17:21:57 2015 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A4703A79; Sat, 21 Mar 2015 17:21:57 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 752FCFC8; Sat, 21 Mar 2015 17:21:57 +0000 (UTC) Received: from John-Baldwins-MacBook-Pro.local (d-69-161-105-82.cpe.metrocast.net [69.161.105.82]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5962CB93C; Sat, 21 Mar 2015 13:21:56 -0400 (EDT) Message-ID: <550DA8B3.8080803@FreeBSD.org> Date: Sat, 21 Mar 2015 13:21:55 -0400 From: John Baldwin User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: RELENG_10 performance regression (was Re: 35-40% performance drop releng9 vs releng10 openvpn References: <5506250A.2000506@sentex.net> <20150316132055.GQ32288@funkthat.com> <5509D6C6.4050204@sentex.net> <20150318211457.GL51048@funkthat.com> <550B6950.8060806@sentex.net> <550C5AAF.9060502@sentex.net> <550C8AEE.4090408@sentex.net> <550CB306.7030405@delphij.net> <20150321001559.GB2379@kib.kiev.ua> <550CBF80.6030809@sentex.net> <550D93C7.9080709@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Sat, 21 Mar 2015 13:21:56 -0400 (EDT) Cc: Konstantin Belousov , FreeBSD-STABLE Mailing List , John-Mark Gurney , Xin LI X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Mar 2015 17:21:57 -0000 On 3/21/15 12:31 PM, Adrian Chadd wrote: > On 21 March 2015 at 08:52, John Baldwin wrote: >> On 3/20/15 8:46 PM, Mike Tancsa wrote: >>> On 3/20/2015 8:15 PM, Konstantin Belousov wrote: >>>>> >>>>> For the purpose of devfs, does it make sense to bump timestamps like >>>>> normal filesystems for each read/write operation? Looks like Mac OS X >>>>> will bump timestamps for each operation but Debian don't. >>>> >>>> First question is, what timecounter hardware is used. I would accept >>>> some slowdown from hardware like HPET, but it is indeed surprising >>>> if caused by TSC. >>>> >>>> >>> >>> David Wolfskill suggested trying the problem commit with >>> >>> vfs.timestamp_precision=0 >>> >>> and it does indeed restore performance to what it was. The raw dtrace >>> files are available and FlameGraphs can all be found at >>> >>> http://tancsa.com/time/ >> >> Do you know why you are using the HPET instead of TSC for timestamping? >> Using the TSC can make a non-trivial performance difference since userland >> can calculate timestamps without using system calls when it is used. >> (That is not related to this case, but switching to the TSC in general is >> preferable.) >> >> There are a few generations of Intel CPUs where you can't mix deeper sleep >> states with the TSC as timecounter, but those CPUs are getting to be a bit >> older at this point. > > What about various VMs? It depends on the hypervisor. bryanv@ is working on bits to allow us to use very cheap timecounters under KVM for example (if that isn't already in the tree). I think bhyve permits guests to use the TSC already. I think when we talked about this on arch@ before the change was made folks felt that even many embedded systems would have some sort of relatively cheap cycle counter, especially going forward. It may be that we end up picking a different default for guests as we do for 'hz' (though that has its downsides. Luigi has noted that one of the things he has to do to fix network performance in VMs is undo that and raise hz back to 1000). However, for bare metal I'd like to figure out why folks aren't using the TSC and fix those if possible. -- John Baldwin