From owner-freebsd-stable@freebsd.org Sun Aug 21 06:45:17 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7625BC0EA0 for ; Sun, 21 Aug 2016 06:45:17 +0000 (UTC) (envelope-from erichsfreebsdlist@alogt.com) Received: from alogt.com (alogt.com [69.36.191.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C7138169E; Sun, 21 Aug 2016 06:45:17 +0000 (UTC) (envelope-from erichsfreebsdlist@alogt.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=alogt.com; s=default; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References: In-Reply-To:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=0OPmye6iTjJuj/ozJTk1IPSKJnt1JVS66wZmgT2ls4M=; b=gLMTEWb+66asPNCklt49xNqvqa HpkcyEDLumUTrnaVXuad9lPZkma9I8R2KeonS64EjS+doF2b88znlc9nBTX868fFUXnlab4TM0U5q x28GnH95Cf0udtmPgNU4jG0FFa+4VBQNaETtw6efxh8VJqRMBVqPIUGGAyy0DoJUSm4I=; Received: from [114.121.234.203] (port=18697 helo=X220.alogt.com) by sl-508-2.slc.westdc.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.87) (envelope-from ) id 1bbMVK-0016nS-G6; Sun, 21 Aug 2016 00:45:11 -0600 Date: Sun, 21 Aug 2016 14:45:05 +0800 From: Erich Dollansky To: Kubilay Kocak Cc: freebsd-stable Subject: Re: Benchmarks results for FreeBSD 11 Message-ID: <20160821144505.27c0f55d@X220.alogt.com> In-Reply-To: References: <20160819073422.4292997b@X220.alogt.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - sl-508-2.slc.westdc.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - alogt.com X-Get-Message-Sender-Via: sl-508-2.slc.westdc.net: authenticated_id: erichsfreebsdlist@alogt.com X-Authenticated-Sender: sl-508-2.slc.westdc.net: erichsfreebsdlist@alogt.com X-Source: X-Source-Args: X-Source-Dir: X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Aug 2016 06:45:18 -0000 Hi, On Sun, 21 Aug 2016 15:21:01 +1000 Kubilay Kocak wrote: > On 19/08/2016 9:34 AM, Erich Dollansky wrote: > > > > I am sure that some know of this site: > > > > http://www.phoronix.com/scan.php?page=article&item=2bsd-7linux-bench&num=4 > > > > I wonder about the results for FreeBSD. As I do not have 11 on my > > machines, a stupid question. Are there still some debugging aids > > enabled in 11? > > They're off in those versions, but did note compiler (and compiler > args) differences between within most tests (See attachments) as you > mentioned. > the benchmark then compares the off-the-shelve distributions. > > > I know that some of the results are caused by the use of CLang and > > some of the results test applications/compilers and not operating > > systems. > > gcc/clang tests and defaults in upstream build systems are almost > certainly contributors. At a minimum it would be nice to see an > attempt to standardise (force) compiler args across all OS runs for > the same test, even if this doesn't prove to be perfect. Yes, without, he compares compilers more than operating systems. > > Separating or adding tests for the same tests using non-default > compilers (in particular latest GCC versions from ports) so they match > across OS's would also be valuable. > > At a minimum it would be worth Michael highlighting the differences, > and ideally removing these variables from the tests, even if they > aren't default configurations. Though a test of out of the box > configurations is still valuable, it can serve to muddy the > underlying differences and make them tougher to isolate. He never does this. I have no idea what he wants to achieve with his benchmarking all the while as the results are very difficult to compare with real life workloads. Erich