From owner-freebsd-stable Sun Jun 24 18:19:26 2001 Delivered-To: freebsd-stable@freebsd.org Received: from topperwein.dyndns.org (acs-24-154-28-172.zoominternet.net [24.154.28.172]) by hub.freebsd.org (Postfix) with ESMTP id B544A37B401 for ; Sun, 24 Jun 2001 18:19:09 -0700 (PDT) (envelope-from behanna@zbzoom.net) Received: from topperwein.dyndns.org (topperwein.dyndns.org [192.168.168.10]) by topperwein.dyndns.org (8.11.4/8.11.4) with ESMTP id f5P1KcM18421 for ; Sun, 24 Jun 2001 21:20:39 -0400 (EDT) (envelope-from behanna@zbzoom.net) Date: Sun, 24 Jun 2001 21:20:38 -0400 (EDT) From: Chris BeHanna Reply-To: Chris BeHanna To: FreeBSD-Stable Subject: RE: Staying *really stable* in FreeBSD In-Reply-To: <15157.25654.158066.311481@guru.mired.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 23 Jun 2001, Mike Meyer wrote: > FreeBSD Admin types: > > I haven't posting anything in some time, so I'm making up for it now with > > this tome. 8-) It says nothing important and means nothing, so skip as you > > like. > > You do have some very good points, and some of them are being > addressed already. > > > Don't think that the computer industry doesn't look at what goes on at the > > FreeBSD jamboree. FreeBSD could be the next Linux, (stop booing) in terms > > of gaining a much larger industry following since we all know it totally > > trashes Linux. But and it's a big BUT, the industry PERCEPTION of the > > FreeBSD community, it's response to security problems, it's ability to > > recognize problems with stability, and reliability, seems to matter more > > than what the reality is. > > This is all very true, but you have to consider what the goal of the > project - or rather, the people working on it - is. While I don't > speak for them, as far as I'm concerned getting lots of people to use > FreeBSD is less important than continuing to provide a quality > computing platform. One of the advantages of open source is that the > developers can ignore the popularity contest of the market, and > concentrate on quality. Not that I wouldn't like FreeBSD to be the > most popular platform around, but if it has to become Windows - or > even Linux - to do so, it clearly isn't worth it. I've seen enough > tools go sour in pursuit of market share that I'd rather not see a > single change that sacrifices quality for market share. > There's an answer for those who want FreeBSD and "professional" support, testing, etc.: BSD/OS. Short of folks stepping up and volunteering to certify -STABLE on a given configuration (and of course, that's all that could be certified, complicated hardware and software interactions being, well, complicated), that's the best you're going to do. I'm just thinking out loud here; none of what follows should be read as "Somebody *should* do this." By way of background, I worked on the SVR 4.2 MIPS port back in 1992, and was involved in some detail with what is done to verify the quality of a commercial UNIX operating system release. I've worked in the commercial software field ever since, including for the world's major source of telecomm software, and I've lived firsthand the CMM Level 5 lifestyle. I like FreeBSD, and if things continue as they have been, I'll continue using it, tracking the list and cvsup'ing at what appear to be opportune times, tar-ing off my last good build beforehand, and saving my last known good kernels. I am, to my great chagrin, NOT an experienced kernel hacker, and I haven't spent much time in the depths of libc (apart from the SYSV rtld code). Almost all of my experience is in user-level code. That said: I don't have the time to write an entire "BVS" ("BSD Verification Suite"), nor to keep it up-to-date; however, I'd be willing to contribute (and maintain) pieces here and there if a test framework and/or harness (read: API/coding convention) were put into place. There's DejaGNU, but it suffers from the GPL. eTET might be a possibility. I'll be happy to investigate this in my (limited) spare time if Jordan (or whomever else) is willing to help me find a place to put it (be kind! :-). Ideally, the authors of a given piece of FreeBSD would write and maintain the code that tested their piece, and would add tests to reproduce bugs as they attacked those bugs. Inevitably, some parts of the test suite would lag behind, and would emit spurious failure reports. The code freeze on FreeBSD proper prior to a release might be a good time to update those parts of the test suite. Once an (up-to-date) test suite is in place, regression will help to verify that -STABLE really is stable--but again, only on the platform and for the configuration on which the test suite is run. A list of volunteers could submit their results leading up to a release, and that list could be posted ("FreeBSD m.n has been verified to run on such-and-so hardware with such-and-so configuration; deviate at your own risk...."). A similar service could be provided (again by volunteers) for given snapshots of -STABLE, and for testing of binary patches. Tracking of the rates of bugs of varying severities is as simple as a cron job firing off a script, but someone has to have the time to write it. For commercial purposes, aging of bugs could also be tracked, although, given that this is a volunteer effort, that might put undue pressure on committers, and might make them quit if people hound them ceaselessly to do more than they have time to do. (Heck, I have bugs in my own for-pay job that are older than I want them to be, yet no one there would remotely consider characterizing me as a slacker.) OK, no more half-baked ramblings (for now). -- Chris BeHanna Software Engineer (Remove "bogus" before responding.) behanna@bogus.zbzoom.net I was raised by a pack of wild corn dogs. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message