Date: Mon, 11 Jun 2001 23:05:53 -0400 (EDT) From: Chris BeHanna <behanna@zbzoom.net> To: FreeBSD-Stable <stable@freebsd.org> Subject: Re: Why is the STABLE branch not so stable anymore? Message-ID: <Pine.BSF.4.32.0106112259510.32911-100000@topperwein.dyndns.org> In-Reply-To: <p05100e20b74afe263e07@[128.113.24.47]>
next in thread | previous in thread | raw e-mail | index | archive | help
(I'm not trying to be a pest, really.)
On Mon, 11 Jun 2001, Garance A Drosihn wrote:
> At 5:24 PM -0400 6/11/01, Mixtim wrote:
> > [...snip...]
> [...snip...]
>
> >I have a wild idea. How about making everyone who commits
> >to the STABLE branch actually:
> >
> > 1. Checkout a clean copy.
> > 2. make buildworld.
> > 3. Actually test the programs they changed.
>
> And guess what. That won't find all the problems.
> [...snip...]
>
> So, basically, you have to remember to test all the programs
> you are not changing, in addition to the ones you are changing.
> I believe this goes by the name of "regression testing". Yes,
> it would be very nice to do. No, it is not easy to do. It is
> very time-consuming to do a good job at it. It requires a lot
> more work than someone sitting down and writing a sarcastic
> 3-point list. I'm sure I could trivialize your life into a
> three-point list if I had the desire to.
Good testing is at least as hard as developing the original
software. To test software to developers' satisfaction requires the
same resources that went into developing the software in the first
place, and, preferably, the design is done in advance so that the
testers can build their test suites in parallel with the actual source
development.
(And even this leaves aside developers writing their own low-level
unit tests.)
If you're dissatisfied, then step up to the plate and donate your
time to build a test harness into which new tests (regression,
integrated, system, etc.) can easily be placed. You might use gABI,
SVVS, and/or eTET, or something completely different. Note that gABI
and SVVS are *HUGE*, and represent many man-years of development.
They were absolutely required for SysV porting partners to brand their
products, and their existence was, in addition to verifying API and
ABI compliance, for the purpose of regression testing.
I'm sure plenty of developers would fire off an automated
regression test suite if one existed. Building one is a nontrivial
exercise (and is also viewed as unglamorous, unfortunately).
And it *still* wouldn't catch all of the problems. :-(
--
Chris BeHanna
Software Engineer (Remove "bogus" before responding.)
behanna@bogus.zbzoom.net
I was raised by a pack of wild corn dogs.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.32.0106112259510.32911-100000>