Date: Tue, 1 Dec 2009 13:25:33 +0000 (GMT) From: Robert Watson <rwatson@FreeBSD.org> To: David Naylor <naylor.b.david@gmail.com> Cc: freebsd-current@freebsd.org Subject: Re: FreeBSD 8.0 Performance (at Phoronix) Message-ID: <alpine.BSF.2.00.0912011320290.1155@fledge.watson.org> In-Reply-To: <200911301117.37950.naylor.b.david@gmail.com> References: <200911301117.37950.naylor.b.david@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 30 Nov 2009, David Naylor wrote: > Phoronix recently published a comparative benchmark[1] for FreeBSD 7.2/8.0 > against Linux and OpenSolaris. I would like to bring some of the good and > bad to light (in the hopes that the developers with the correct expertise > will be intrigued). > > The tests were performed with a 'standard' installation of FreeBSD on a > Lenovo ThinkPad T61. > > I've tried to eliminate tests who's performance is a result of compiler > differences and/or 3rd party applications and tests who's statistical > significance are not so strong (subjective guess). It would be nice to see the same tests rerun with more attention to well-understood benchmarking methodology (i.e., multiple runs, characterization of variance, statistical tests to decide if two results are likely to be from the same set, and what statistically significant difference exists). At the very least, three or five runs of each test so we can consider spread would be very useful. However, I agree with your characterizaitons that these are the likely areas of issue :-). For tests that depend on file system namespace operations, seeing a measurement comparing ZFS directly would be interesting; likewise, perhaps using async file system mounts on FreeBSD without soft updates. Jeff's forthcoming journaling changes would also be interesting to consider, but I'm not sure to what extent he'll see performance improvements, or whether it's just about the consistency improvements for his SU+J work. Looking at compiler differences would also make a lot of sense for things like compression/ray tracing/etc. Another area of comparison would simply be the version and contents of gzip, particularly it might be interesting to try the Linux gzip under FreeBSD's Linux emulation (and for a few other of the CPU-centric tests: use a Linux chroot or Debian/kFreeBSD to do side-by-side comparisons on the same kernel). The threaded I/O is interesting and possiby the greatest concern. We should try to reproduce these in "lab conditions" and attempt to understand the workloads more. We do have known "strong" synchronization in the vnode write path, such that writes to the same vnode may be serialized in a strong way -- it could be that these tests show that up. Robert > > Improvements for FreeBSD 8.0 vs 7.2: > - 7-Zip Compression (page 3) > - Timed MAFFT Alignment (page 5) > - GraphicsMagick (page 5) > - Threaded IO (64MB Random Write - 32 threads) (page 7) > - Threaded IO (64MB Read - 32 threads) (page 7) > > Regressions for FreeBSD 8.0 vs 7.2: > - Gzip compressions of a 2GB file (page 3) > - C-Ray (page 4) > - Threaded IO (64MB Write - 4 threads) (page 7) > - Threaded IO (64MB Write - 32 threads) (page 7) > > Poor performance relative to Linux and OpenSolaris > - Threaded IO (especially random writes) (page 7) > - OpenSSL (RSA 4096bit) (page 8) > - PostMark (disk transaction) (page 8) > > It appears that threaded activity on UFS does not fair well against Linux/ext4 > and OpenSolaris/ZFS. Phoronix intends to do a comparative test against > FreeBSD and OpenSolaris on ZFS. > > Regards, > > David > > 1) > http://www.phoronix.com/scan.php?page=article&item=freebsd8_benchmarks&num=1 >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?alpine.BSF.2.00.0912011320290.1155>