From owner-freebsd-current@FreeBSD.ORG Tue Dec 1 13:25:34 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E50C1065692 for ; Tue, 1 Dec 2009 13:25:34 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id DD6FD8FC1B for ; Tue, 1 Dec 2009 13:25:33 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 8ECEA46B03; Tue, 1 Dec 2009 08:25:33 -0500 (EST) Date: Tue, 1 Dec 2009 13:25:33 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: David Naylor In-Reply-To: <200911301117.37950.naylor.b.david@gmail.com> Message-ID: References: <200911301117.37950.naylor.b.david@gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@freebsd.org Subject: Re: FreeBSD 8.0 Performance (at Phoronix) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2009 13:25:34 -0000 On Mon, 30 Nov 2009, David Naylor wrote: > Phoronix recently published a comparative benchmark[1] for FreeBSD 7.2/8.0 > against Linux and OpenSolaris. I would like to bring some of the good and > bad to light (in the hopes that the developers with the correct expertise > will be intrigued). > > The tests were performed with a 'standard' installation of FreeBSD on a > Lenovo ThinkPad T61. > > I've tried to eliminate tests who's performance is a result of compiler > differences and/or 3rd party applications and tests who's statistical > significance are not so strong (subjective guess). It would be nice to see the same tests rerun with more attention to well-understood benchmarking methodology (i.e., multiple runs, characterization of variance, statistical tests to decide if two results are likely to be from the same set, and what statistically significant difference exists). At the very least, three or five runs of each test so we can consider spread would be very useful. However, I agree with your characterizaitons that these are the likely areas of issue :-). For tests that depend on file system namespace operations, seeing a measurement comparing ZFS directly would be interesting; likewise, perhaps using async file system mounts on FreeBSD without soft updates. Jeff's forthcoming journaling changes would also be interesting to consider, but I'm not sure to what extent he'll see performance improvements, or whether it's just about the consistency improvements for his SU+J work. Looking at compiler differences would also make a lot of sense for things like compression/ray tracing/etc. Another area of comparison would simply be the version and contents of gzip, particularly it might be interesting to try the Linux gzip under FreeBSD's Linux emulation (and for a few other of the CPU-centric tests: use a Linux chroot or Debian/kFreeBSD to do side-by-side comparisons on the same kernel). The threaded I/O is interesting and possiby the greatest concern. We should try to reproduce these in "lab conditions" and attempt to understand the workloads more. We do have known "strong" synchronization in the vnode write path, such that writes to the same vnode may be serialized in a strong way -- it could be that these tests show that up. Robert > > Improvements for FreeBSD 8.0 vs 7.2: > - 7-Zip Compression (page 3) > - Timed MAFFT Alignment (page 5) > - GraphicsMagick (page 5) > - Threaded IO (64MB Random Write - 32 threads) (page 7) > - Threaded IO (64MB Read - 32 threads) (page 7) > > Regressions for FreeBSD 8.0 vs 7.2: > - Gzip compressions of a 2GB file (page 3) > - C-Ray (page 4) > - Threaded IO (64MB Write - 4 threads) (page 7) > - Threaded IO (64MB Write - 32 threads) (page 7) > > Poor performance relative to Linux and OpenSolaris > - Threaded IO (especially random writes) (page 7) > - OpenSSL (RSA 4096bit) (page 8) > - PostMark (disk transaction) (page 8) > > It appears that threaded activity on UFS does not fair well against Linux/ext4 > and OpenSolaris/ZFS. Phoronix intends to do a comparative test against > FreeBSD and OpenSolaris on ZFS. > > Regards, > > David > > 1) > http://www.phoronix.com/scan.php?page=article&item=freebsd8_benchmarks&num=1 >