From owner-freebsd-stable@FreeBSD.ORG Wed Sep 13 21:24:41 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A442716A415; Wed, 13 Sep 2006 21:24:41 +0000 (UTC) (envelope-from kline@sage.thought.org) Received: from sage.thought.org (dsl231-043-140.sea1.dsl.speakeasy.net [216.231.43.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 28B1043D49; Wed, 13 Sep 2006 21:24:39 +0000 (GMT) (envelope-from kline@sage.thought.org) Received: from sage.thought.org (kline@localhost [127.0.0.1]) by sage.thought.org (8.13.6/8.12.10) with ESMTP id k8DLOdH8091122; Wed, 13 Sep 2006 14:24:39 -0700 (PDT) (envelope-from kline@sage.thought.org) Received: (from kline@localhost) by sage.thought.org (8.13.6/8.13.1/Submit) id k8DLOceP091121; Wed, 13 Sep 2006 14:24:38 -0700 (PDT) (envelope-from kline) Date: Wed, 13 Sep 2006 14:24:38 -0700 From: Gary Kline To: Pawel Jakub Dawidek Message-ID: <20060913212438.GA90960@thought.org> References: <20060909173813.GA1388@FS.denninger.net> <20060913144605.GD70245@garage.freebsd.pl> <20060913181504.GB90378@thought.org> <20060913202603.GA72187@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060913202603.GA72187@garage.freebsd.pl> X-Organization: Thought Unlimited. Public service Unix since 1986. X-Of_Interest: Observing twenty years of service to the Unix community User-Agent: Mutt/1.5.11 Cc: Gary Kline , freebsd-stable@freebsd.org Subject: Re: ARRRRGH! Guys, who's breaking -STABLE's GMIRROR code?! X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Sep 2006 21:24:41 -0000 On Wed, Sep 13, 2006 at 10:26:03PM +0200, Pawel Jakub Dawidek wrote: > On Wed, Sep 13, 2006 at 11:15:04AM -0700, Gary Kline wrote: > > On Wed, Sep 13, 2006 at 04:46:05PM +0200, Pawel Jakub Dawidek wrote: > > > On Sat, Sep 09, 2006 at 12:38:13PM -0500, Karl Denninger wrote: > > > > This is not cool folks. > > > > > > I'm really sorry for the breakage. I'm trying to treat -STABLE very > > > gently, unfortunately this time I made a mistake. > > > > > > The change was committed to HEAD at 9 August. The change fixed one bug, > > > but introduced another, which I didn't expected. The change seemed to be > > > trivial and I only tested that it fixes the bug I was tracking down, I > > > haven't looked for regressions. > > > > > > > Well, after this lengthy discussion, I've switched to -RELEASE. > > -STABLE just ain't... We all realize that none of us would > > put out a buggy release--not even -CURRENT. But let me ask > > the next obvious question. How difficult would it be to > > build a regression test, or suite of tests? Obviously, this > > could be done over months -> years. (In my last lifetime > > as a hacker I was in the kernel test group [a BSD-4.4 based > > release on new architecture]. ) It's a bit hard to believe > > that with all the genius in this effort, that no regression > > testing is done. > > I'm trying to implement regression tests to the code I add. You can find > them in /usr/src/tools/regression/: > > geom_concat 2 files, 2 tests > geom_eli 15 files, 5818 tests > geom_gate 3 files, 6 tests > geom_mirror 7 files, 27 tests > geom_nop 2 files, 2 tests > geom_raid3 12 files, 13 tests > geom_shsec 2 files, 6 tests > geom_stripe 2 files, 2 tests > ipsec 1 file, 306 tests > redzone9 1 file, 6 tests > usr.bin/pkill 27 files, 49 tests > > As I said already, I mistakenly thought the change was trivial and the > only thing I tested was if it fixes a bug I was tracking down back then. > > We dicuss from time to time that we should have service simlar to > tinderbox, which will run regression tests regularly and report > regressions to the mailing lists - the more we automate the smaller > chance for a human mistake like mine. Unfortunately this is not yet > done. You're right in saying that the more automation, the more stability. Hats off for all this good work (from somebody who has been there before:).... This is the kind of thing tht needs to be done (i) to catch bugs before they are committed, and (ii) to make BSD all the more trustworthy and bullet-proof. HAving run FBSD since 2.0.5 and only *one* "fatal trap" is pretty hard to beat. gary > > -- > Pawel Jakub Dawidek http://www.wheel.pl > pjd@FreeBSD.org http://www.FreeBSD.org > FreeBSD committer Am I Evil? Yes, I Am! -- Gary Kline kline@thought.org www.thought.org Public service Unix