Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 21 Dec 2000 13:14:41 -0700
From:      Drew Eckhardt <drew@PoohSticks.ORG>
To:        "SteveB" <admin@bsdfan.cncdsl.com>
Cc:        freebsd-hackers@FreeBSD.ORG
Subject:   Re: Sitting on hands (no longer Re: FreeBSD vs Linux, Solaris, and NT) 
Message-ID:  <200012212014.eBLKEfh12072@chopper.Poohsticks.ORG>
In-Reply-To: Your message of "Thu, 21 Dec 2000 09:48:44 PST." <NEBBIGOKKMNLOMOHMJNPMELBCNAA.admin@bsdfan.cncdsl.com> 

next in thread | previous in thread | raw e-mail | index | archive | help
In message <NEBBIGOKKMNLOMOHMJNPMELBCNAA.admin@bsdfan.cncdsl.com>, admin@bsdfan
.cncdsl.com writes:
>Here's the thing about open software that still concerns me. My
>background is with the major software development tools companies, so
>that is my point of reference. It is great that code is available and
>fixes are made and pushed out, but who is doing real testing of these
>fixes.  Sure the obvious problem is fixed, but what other problems has
>it uncovered, what side effect has it created, and how about
>compatibility with other software or drivers in this case.
>
>With commercial software (well at least the places I worked) nothing
>could go out the door without a complete QA cycle performed on it.

In a past life, I did half the design and implementation of the
software tracking calls and letting the billing software know 
about them on a CDMA cellular base station.

For hardware, we used machines from the biggest workstation vendor
with a three letter name, running the latest production release of 
their Unix.

Before booting the putz from our team who'd crippled our software
with threads and excised the damage he'd done, we regularly dumped the 
machines out to the ROM monitor.

I know people who work in several operating systems groups at that
company, know a bit about their quality control process, and know that
it was insufficient.

I've yet to encounter a bug of that severity in any released version 
of free software (about the worst which wasn't hardware related is
the FreeBSD Tulip driver not working correctly in full-duplex 
100baseT mode).

>So who is testing these fixes in open source world?  

Cygnus is/was doing automated regression testing on GCC.

>Just seeing if
>the problem at hand is gone isn't real testing, even claiming
>thousands of people are now using it isn't enough.  

In theory, a standard suite of white and black box tests should
be superior.

Given inumerable bad experiences with Adobe, IBM, HP, Microsoft, Sun
and other smaller companies, in practice it doesn't seem to work any 
better than the million-monkeys approach.

>QA is unglamorous work, but needs to be done.

Does this mean you're volunteering?



To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200012212014.eBLKEfh12072>