Date: Thu, 21 Dec 2000 13:14:41 -0700 From: Drew Eckhardt <drew@PoohSticks.ORG> To: "SteveB" <admin@bsdfan.cncdsl.com> Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Sitting on hands (no longer Re: FreeBSD vs Linux, Solaris, and NT) Message-ID: <200012212014.eBLKEfh12072@chopper.Poohsticks.ORG> In-Reply-To: Your message of "Thu, 21 Dec 2000 09:48:44 PST." <NEBBIGOKKMNLOMOHMJNPMELBCNAA.admin@bsdfan.cncdsl.com>
next in thread | previous in thread | raw e-mail | index | archive | help
In message <NEBBIGOKKMNLOMOHMJNPMELBCNAA.admin@bsdfan.cncdsl.com>, admin@bsdfan .cncdsl.com writes: >Here's the thing about open software that still concerns me. My >background is with the major software development tools companies, so >that is my point of reference. It is great that code is available and >fixes are made and pushed out, but who is doing real testing of these >fixes. Sure the obvious problem is fixed, but what other problems has >it uncovered, what side effect has it created, and how about >compatibility with other software or drivers in this case. > >With commercial software (well at least the places I worked) nothing >could go out the door without a complete QA cycle performed on it. In a past life, I did half the design and implementation of the software tracking calls and letting the billing software know about them on a CDMA cellular base station. For hardware, we used machines from the biggest workstation vendor with a three letter name, running the latest production release of their Unix. Before booting the putz from our team who'd crippled our software with threads and excised the damage he'd done, we regularly dumped the machines out to the ROM monitor. I know people who work in several operating systems groups at that company, know a bit about their quality control process, and know that it was insufficient. I've yet to encounter a bug of that severity in any released version of free software (about the worst which wasn't hardware related is the FreeBSD Tulip driver not working correctly in full-duplex 100baseT mode). >So who is testing these fixes in open source world? Cygnus is/was doing automated regression testing on GCC. >Just seeing if >the problem at hand is gone isn't real testing, even claiming >thousands of people are now using it isn't enough. In theory, a standard suite of white and black box tests should be superior. Given inumerable bad experiences with Adobe, IBM, HP, Microsoft, Sun and other smaller companies, in practice it doesn't seem to work any better than the million-monkeys approach. >QA is unglamorous work, but needs to be done. Does this mean you're volunteering? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200012212014.eBLKEfh12072>