Date: Fri, 22 Oct 2010 16:17:30 -0700 From: David Wolfskill <david@catwhisker.org> To: freebsd-performance@freebsd.org Subject: Re: Possible evidence of performance regression for 8.1-S (vs. 7.1) Message-ID: <20101022231730.GP52404@albert.catwhisker.org> In-Reply-To: <4CBF8032.8000609@freebsd.org> References: <20101020174854.GZ21226@albert.catwhisker.org> <4CBF8032.8000609@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--AwNVUpjOmSj7UnwZ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 20, 2010 at 04:50:10PM -0700, Julian Elischer wrote: > ... > try the 7.x machine but running the 8.x kernel.. i.e. change nothing,=20 > but boot the new kernel. I just started that test, to run over the weekend. I did run another test (3 iterations) with the 7.1 OS (kernel+userland) on the physical machine that normally runs 8.x. Here is a summary of the earlier results, along with the new ones: start stop real user sys host os 1287525363 1287546846 21483.13 82628.20 21703.09 ref-7.x 7.1-R+ 1287548005 1287569100 21094.63 82853.19 22185.02 ref-7.x 7.1-R+ 1287570300 1287591371 21071.33 82756.81 21943.22 ref-7.x 7.1-R+ 1287592592 1287614103 21511.23 82637.30 21849.90 ref-7.x 7.1-R+ 1287615323 1287636770 21446.42 82715.81 21708.97 ref-7.x 7.1-R+ 1287436357 1287461948 25590.99 81502.22 18115.07 ref-8.x 8.1-S@r214029 1287462797 1287488766 25969.26 81452.14 17920.14 ref-8.x 8.1-S@r214029 1287489641 1287515287 25645.84 81548.40 18256.52 ref-8.x 8.1-S@r214029 1287516151 1287541481 25329.64 81546.23 18294.10 ref-8.x 8.1-S@r214029 1287542355 1287568599 26244.59 81431.47 17902.39 ref-8.x 8.1-S@r214029 1287710312 1287732046 21733.20 82688.01 22108.95 ref-8.x 7.1-R+ 1287733360 1287754549 21188.88 82869.09 21890.83 ref-8.x 7.1-R+ 1287755881 1287777566 21684.09 82772.50 21933.74 ref-8.x 7.1-R+ I only ran 3 tests, as the purpose was merely a reality check -- to verify that we weren't actually seeing differences in hardware between the two machines. I believe that the above results provide adequate reason to believe that hardware is not the underlying issue: the 7.1 results on the ref-8.x machine are just about identical to the 7.1 results on the ref-7.x machine, and both are noticably different from the 8.1 results. > >FWIW, the workload is fairly CPU intensive during most of the run; the > >I/O done during (most of) the test appears to be very light, and the > >memory used is fairly modest. In each of the test machines, I have > >turned off HTT (HyperThreading Technology); hw.ncpu reports 8 for each. >=20 > try with HTT on for modern hardware.. > .... OK, I'll try that experiment at some point when I have the time and other resources, but what I'm trying to point out is that I believe I'm seeing something that appears to be a performance regression, comparing 7.1 (+ a few patches that were later committed to stable/7) to stable/8 @r214029, on identically configured hardware running the same workload (that happens to be critical for my employer). FWIW, I tried comparing the 7.1 (as above) vs. stable/7 a couple of months ago, and found no statistically significant difference in performance. Thus, I believe it would be possible for someone else to reproduce my results, comparing a fasirly recent stable/7 to a fairly recent stable/8. If someone does try that (with a workload primarily composed of building software), and gets results that differ significantly from what I'm reporting, I would appreciate an opportunity to compare notes, as it would then be apparent that I've managed to damage my stable/8 environment in some way. On the other hand, if someone else tries that, and gets results that corroborate what I'm seeing, we may have some work to do to get stable/8 whipped into shape before 8.2-RELEASE (for which the code freeze is currently scheduled for 28 Nov). Granted, performance isn't the only thing that drives releases, but I know that for the developers I support, having an 18% increase in elapsed time to perform a critical task, merely because of an "upgrade" to the OS, would be ... poorly received. And there's no way I could recommend doing that, unless the alternatives were demonstrably worse. I'm a bit concerned, here. Peace, david --=20 David H. Wolfskill david@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --AwNVUpjOmSj7UnwZ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkzCG4kACgkQmprOCmdXAD2lcACdGpkCS6H62SFB6yy8BtImiUtf GrEAnjXVVcVkRmATif+TsGBGXVLPlp+t =vp2g -----END PGP SIGNATURE----- --AwNVUpjOmSj7UnwZ--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20101022231730.GP52404>