From owner-freebsd-hackers Thu Dec 21 9:53:58 2000 From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 21 09:53:54 2000 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from herd.plethora.net (herd.plethora.net [205.166.146.1]) by hub.freebsd.org (Postfix) with ESMTP id 2732737B400 for ; Thu, 21 Dec 2000 09:53:53 -0800 (PST) Received: from guild.plethora.net (root@guild.plethora.net [205.166.146.8]) by herd.plethora.net (8.9.0/8.9.0) with ESMTP id LAA05694 for ; Thu, 21 Dec 2000 11:53:51 -0600 (CST) Received: from guild.plethora.net (seebs@localhost.plethora.net [127.0.0.1]) by guild.plethora.net (8.9.3/8.9.0) with ESMTP id LAA28708 for ; Thu, 21 Dec 2000 11:53:50 -0600 (CST) Message-Id: <200012211753.LAA28708@guild.plethora.net> From: seebs@plethora.net (Peter Seebach) Reply-To: seebs@plethora.net (Peter Seebach) To: freebsd-hackers@freebsd.org Subject: Re: Sitting on hands (no longer Re: FreeBSD vs Linux, Solaris, and NT) In-reply-to: Your message of "Thu, 21 Dec 2000 09:48:44 PST." Date: Thu, 21 Dec 2000 11:53:50 -0600 Sender: seebs@plethora.net Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , "SteveB" wri tes: >With commercial software (well at least the places I worked) nothing >could go out the door without a complete QA cycle performed on it. Yes. This is why the open systems have "releases" every so often; a release has been run through something more like a QA cycle. The QA cycle is where the naive fools run "-current" believing it will have "new features". :) >Even the smallest of bug fixes couldn't be released without a QA >cycle. A full QA cycle was time consuming and expensive, so fixes sat >until there was a stack of them to QA'd as a group or they had to wait >until next upgrade. That way we knew state of the product. Yes, the >state of the product would include known bugs. The key was a known bug >and a known documented bug was as valuable as a fix. Sure a bug is >bad, but if it is documented you don't waste trying to make something >work that is known to be broke. But you can't *do* anything. Imagine a known bug "doesn't run on Pentium or later systems". That's pretty much totally crippling now. The important point is that you get the choice. You can run a stable release, with known bugs, or you can run slightly less tested code which fixes them. >So who is testing these fixes in open source world? Just seeing if >the problem at hand is gone isn't real testing, even claiming >thousands of people are now using it isn't enough. There can still be >lurking potentially data destroying bugs lurking. Yes. But that's just as true of a full QA cycle. Safety, in software, is an analogue signal, not a digital one. My experience (and I admit, I'm mostly from a NetBSD background) is that -current releases are dramatically more reliable, and less buggy, than commercial software. Testing, alone, does not catch bugs. *Analysis* does, and one of the things the open source community shines at is having a fix *analyzed* by a number of people. >In the open source >world is there a official QA process or group. Is there a FreeBSD >test suite that releases go through. QA is unglamorous work, but >needs to be done. I don't know about the "official" process, but I will tell you that I'd rather have my life depend on FreeBSD-current than on Windows NT, despite the "QA cycle". There are many ways to do effective QA. -s To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message