Date: Sun, 24 Jun 2001 21:20:38 -0400 (EDT) From: Chris BeHanna <behanna@zbzoom.net> To: FreeBSD-Stable <stable@freebsd.org> Subject: RE: Staying *really stable* in FreeBSD Message-ID: <Pine.BSF.4.32.0106242057320.18238-100000@topperwein.dyndns.org> In-Reply-To: <15157.25654.158066.311481@guru.mired.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 23 Jun 2001, Mike Meyer wrote:
> FreeBSD Admin <freebsd@home.com> types:
> > I haven't posting anything in some time, so I'm making up for it now with
> > this tome. 8-) It says nothing important and means nothing, so skip as you
> > like.
>
> You do have some very good points, and some of them are being
> addressed already.
>
> > Don't think that the computer industry doesn't look at what goes on at the
> > FreeBSD jamboree. FreeBSD could be the next Linux, (stop booing) in terms
> > of gaining a much larger industry following since we all know it totally
> > trashes Linux. But and it's a big BUT, the industry PERCEPTION of the
> > FreeBSD community, it's response to security problems, it's ability to
> > recognize problems with stability, and reliability, seems to matter more
> > than what the reality is.
>
> This is all very true, but you have to consider what the goal of the
> project - or rather, the people working on it - is. While I don't
> speak for them, as far as I'm concerned getting lots of people to use
> FreeBSD is less important than continuing to provide a quality
> computing platform. One of the advantages of open source is that the
> developers can ignore the popularity contest of the market, and
> concentrate on quality. Not that I wouldn't like FreeBSD to be the
> most popular platform around, but if it has to become Windows - or
> even Linux - to do so, it clearly isn't worth it. I've seen enough
> tools go sour in pursuit of market share that I'd rather not see a
> single change that sacrifices quality for market share.
>
There's an answer for those who want FreeBSD and "professional"
support, testing, etc.: BSD/OS.
Short of folks stepping up and volunteering to certify -STABLE on
a given configuration (and of course, that's all that could be certified,
complicated hardware and software interactions being, well,
complicated), that's the best you're going to do.
I'm just thinking out loud here; none of what follows should be
read as "Somebody *should* do this." By way of background, I worked
on the SVR 4.2 MIPS port back in 1992, and was involved in some detail
with what is done to verify the quality of a commercial UNIX operating
system release. I've worked in the commercial software field ever
since, including for the world's major source of telecomm software,
and I've lived firsthand the CMM Level 5 lifestyle. I like FreeBSD,
and if things continue as they have been, I'll continue using it,
tracking the list and cvsup'ing at what appear to be opportune times,
tar-ing off my last good build beforehand, and saving my last known
good kernels. I am, to my great chagrin, NOT an experienced kernel
hacker, and I haven't spent much time in the depths of libc (apart
from the SYSV rtld code). Almost all of my experience is in
user-level code.
That said:
I don't have the time to write an entire "BVS" ("BSD Verification
Suite"), nor to keep it up-to-date; however, I'd be willing to
contribute (and maintain) pieces here and there if a test framework
and/or harness (read: API/coding convention) were put into place.
There's DejaGNU, but it suffers from the GPL. eTET might be a
possibility. I'll be happy to investigate this in my (limited) spare
time if Jordan (or whomever else) is willing to help me find a place
to put it (be kind! :-).
Ideally, the authors of a given piece of FreeBSD would write and
maintain the code that tested their piece, and would add tests to
reproduce bugs as they attacked those bugs. Inevitably, some parts of
the test suite would lag behind, and would emit spurious failure
reports. The code freeze on FreeBSD proper prior to a release might
be a good time to update those parts of the test suite.
Once an (up-to-date) test suite is in place, regression will help
to verify that -STABLE really is stable--but again, only on the
platform and for the configuration on which the test suite is run. A
list of volunteers could submit their results leading up to a release,
and that list could be posted ("FreeBSD m.n has been verified to run
on such-and-so hardware with such-and-so configuration; deviate at
your own risk....").
A similar service could be provided (again by volunteers) for
given snapshots of -STABLE, and for testing of binary patches.
Tracking of the rates of bugs of varying severities is as simple
as a cron job firing off a script, but someone has to have the time to
write it. For commercial purposes, aging of bugs could also be
tracked, although, given that this is a volunteer effort, that might
put undue pressure on committers, and might make them quit if people
hound them ceaselessly to do more than they have time to do. (Heck, I
have bugs in my own for-pay job that are older than I want them to
be, yet no one there would remotely consider characterizing me as a
slacker.)
OK, no more half-baked ramblings (for now).
--
Chris BeHanna
Software Engineer (Remove "bogus" before responding.)
behanna@bogus.zbzoom.net
I was raised by a pack of wild corn dogs.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.32.0106242057320.18238-100000>