Date: Sun, 28 Jan 1996 01:21:01 -0800 From: "Jordan K. Hubbard" <jkh@time.cdrom.com> To: Terry Lambert <terry@lambert.org> Cc: dyson@freefall.freebsd.org (John Dyson), hackers@freefall.freebsd.org Subject: Re: Good news -- pipe stuff Message-ID: <19327.822820861@time.cdrom.com> In-Reply-To: Your message of "Fri, 26 Jan 1996 11:51:24 MST." <199601261851.LAA04990@phaeton.artisoft.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> I read the Lai/Baker paper: in "Figure 1", FreeBSD kicks some serious > butt on context switches -- it appears to be both flat and linear past > 200 processes (the limit of the graph in the figure). Actually, at a later talk it came out that Linux had substantially improved this in the current release. However, both this talk and Larry McVoy's talk that followed it were wastes of paper and time for all concerned. First, the Lai/Baker talk spent 9/10ths of its time benchmarking the 3 various systems (Linux, FreeBSD, Solaris) and going into great and wonderful detail about how they strove to make each benchmark fair and complete. That was fine, and I had no problems with it up to that point when, in the last 1/10th of the talk, in a somewhat bizarre segue they threw all of the preceeding data out on its ear with the pat statement that "performance really didn't matter anyway." They chose Linux because they liked whichever mailing list or IRC channel they hung out in, and they made no effort to compare the level of support they'd received with FreeBSD or make any qualitative statements about it whatsoever. Small wonder, I suppose, since none of us could remember being contacted by any of the principles either way. >From where we were sitting, it was pretty clear that they had already decided on Linux at some earlier point in the game (for whatever reason) and everything that followed was a weak attempt at back-solving. Some aspects of this were downright surreal. In the network benchmark tests, for example, they used a Linux machine as the NFS server in their client benchmark despite acknowledging the fact that Linux had performed at 30% the speed of FreeBSD in general network performance and was thus entirely unsuitable for trying to prove the theoretical performance limits for a Linux or FreeBSD NFS client. One might as well attempt to measure the top speed of two runners while both are dragging anvils behind them! When later pressed on this point, they cited a resource shortage. They couldn't find a second PC and their only other NFS server was a Sparcstation I. C'mon, these people are doing benchmarking and publishing the results at USENIX and they can't even locate a lousy $3K PC to do their research correctly? I'm sorry, but this presentation was simply a shoddy affair with no attempt even made to adequately substantiate the conclusions it reached. One star. Then we had Larry McVoy discussing his lmbench suite, another presentation with a promising start and an utterly ludicrous conclusion. First Larry discussed the suite itself and his "microbenchmark" design, which was perfectly fine and dandy and we were all interested in knowing how he generated his numbers - no problems so far. His enthusiasm for Linux later led him entirely astray, however, as he took his own surreal trip into the twilight zone and actually attempted to *compare the numbers*. Now that might not sound like such a bad thing on the surface of it until you realize that when I say "compare the numbers" I mean "compare any numbers against any other numbers." He had collected together a complete hodge-podge of machine traces running under a wide variety of operating systems, no two matching OS/machine combinations among them, and "matched them up." Did you know, for example, that Linux 1.2.x running on a P6/200 out-performs a FreeBSD 2.1.0 machine running on a P5/133? I only knew it to be generally true before but Larry, by god, had the hard numbers to prove it! We also lost out to Linux on the ALPHA, I'm afraid, and it was with a heavy heart that I had to bow my head in shame when Larry pointed to the final numbers, which clearly showed $20K-$50K Linux boxes knocking the shorts off our little $4K FreeBSD box, and gleefully announced that "Linux kicked butt." It was almost as if our august researchers started out with the very best of intentions, thinking caps screwed squarely in place, and had completed a substantial portion of their presentations when they were suddenly stricken with severe cases of the Finnish Flu (reputedly linked with the GPV virus) - an ailment which causes the body temperature to rise until the very air is heated with every exhalation; the eyes go slack and the sufferer can only drool helplessly, twitching and murmering "Linus... Linus!" I dunno, one might say that I'm just angry that they didn't pick my horse and now I'm throwing a tantrum about it, but I'm genuinely not angry here so much as I'm simply *puzzled* and, yes, perhaps feeling a little short-changed at going to a conference to listen to a little good science (of which there was, thankfully, a considerable amount elsewhere) and encountering instead the same kind of knee-jerk, "stand on a soap-box in Hyde park" evangelism that I can get for free on USENET any time I'm stupid enough to tune into it. For a $400 conference, and from Lai and McVoy, I expected standards somewhat higher than that. Too bad. Jordan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19327.822820861>