From owner-freebsd-qa Mon Dec 20 18:57:28 1999 Delivered-To: freebsd-qa@freebsd.org Received: from staff.accessus.net (staff.accessus.net [209.145.151.3]) by hub.freebsd.org (Postfix) with ESMTP id D020514DEC; Mon, 20 Dec 1999 18:57:21 -0800 (PST) (envelope-from doogie@staff.accessus.net) Received: by staff.accessus.net with Internet Mail Service (5.5.2650.21) id ; Mon, 20 Dec 1999 20:57:21 -0600 Message-ID: From: Jason Young To: "'mistwolf@mushhaven.net'" , freebsd-qa@FreeBSD.ORG Cc: "'jkh@freebsd.org'" Subject: RE: After 3.4 finally goes out the door Date: Mon, 20 Dec 1999 20:57:14 -0600 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01BF4B5F.18AB1FAC" Sender: owner-freebsd-qa@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_001_01BF4B5F.18AB1FAC Content-Type: text/plain; charset="iso-8859-1" Well, I was trying to avoid the 'someone oughta' and 'you really should' type of rambling. :) I simply propose that if we could just gain some more time to QA the "real" release, then we'd get some truly useful work done. As for a standardized testing methodology or procedure? IMHO, that would be counterproductive. Everybody's got a different usage pattern, and different hardware, and trying to force people into the same mold will be hard and ultimately bad and wrong. For instance, if you had some script to follow in sysinstall - "test this, and then this, and then this, and test this" - would the sysinstall bugs have been found? If end-users with different hardware and configurations and usage patterns hadn't tested moused, would we have shipped with roller wheel support? No, it's people doing what they normally do, which is quite likely to be outside of what the developer tested. This is a Good Thing, and we need people doing their own unexpected, possibly weird, sometimes nonobvious stuff. The developers aren't total morons and the obvious stuff obviously works fine. The developers already have and already run the various regression tests that are available. A technically savvy QA volunteer might want to run those, and certainly that's fine, but I don't think we're short of dedicated and technical savvy developers familiar with the OS and its internals. What we need, IMHO, is a bunch of people putting it in real world situations, testing it under their various application and user loads, and seeing how it does. We had this in the form of the 3.4-RELEASE QA volunteer team, and everybody did a great job with the time and resources available. I hope that jkh and friends can see this, and see that just a little bit of foresight we can make sure that the next CD that ships holds a truly solid and kickass release. > -----Original Message----- > From: Jamie Norwood [mailto:mistwolf@mushhaven.net] > Sent: Monday, December 20, 1999 8:00 PM > To: Jason Young > Cc: 'Colin'; freebsd-qa@FreeBSD.ORG; 'jkh@freebsd.org' > Subject: Re: After 3.4 finally goes out the door > > > I agree with this totally. I also feel someone should step forward > and coordinate; I didn't do more than install and see if it worked > as I am not familier with how things should be tested. Also, I feel > that some things should be made available such as CVSUP scripts > and such to make sure the people involved really are testing the > right things. Could we maybe get a 'branch' named 'qa' that would show > rc or qa in the uname output? That would also make things > easier as we > could then test that tree for the cd, and bug fixes that are essential > could then be made to that tree only, and normal developement could > continue on the -STABLE/-RELEASE tree. > > Jamie > > > On Mon, Dec 20, 1999 at 07:50:12PM -0600, Jason Young wrote: > > > > I think it's a good idea to keep the list around, but I > don't think that the > > QA volunteers were really used to their full potential. > > > > I tested when I could (early in the testing cycle), and I > feel that most of > > my efforts were for naught, because as usual, a large > percentage of the > > commits in the testing period were very last minute (after > most or all of > > the QA builds). > > > > Yeah, it's a free software project and nobody's paid, and > what gets out the > > door is still a fine piece of work (and major kudos to jkh > and the rest of > > the guys on that). But, I don't think much meaningful > testing of the real > > release was accomplished due to all the really really late > changes. The last > > minute "oh I really need such and such MFC'd" trips are > killers (sysinstall, > > moused, you won't have to rack your brain too hard for examples). > > > > I assume that the schedule for 3.5-RELEASE is pretty much > set already. Would > > it be possible to back most of that schedule up about a > week, and in the > > week before the CD is cut, put the "pretty-much-final, > we-really-mean-it, > > won't touch it unless the release is definitely fscked" up > for general FTP? > > If the schedules are set well in advance, nobody will be hurting or > > complaining of the loss of one week's time in a three or > four month -STABLE > > release cycle. The QA volunteers need a set release to > assure its quality, > > ya know? This is no extra work or major change for anyone. > > > > I think that if we could get the "frozen" release candidate > out there in > > front of people for at least a week or so, we could muchly > reduce the size > > of our ERRATA files. > > > > > -----Original Message----- > > > From: Colin [mailto:cwass99@home.com] > > > Sent: Monday, December 20, 1999 7:18 PM > > > To: freebsd-qa@FreeBSD.ORG > > > Subject: After 3.4 finally goes out the door > > > > > > > > > I realize we got off to a rather slow start, but all > > > things considered > > > better than many expected. My question is, is there any > > > intent for this list > > > to survive the release of 3.4, as basically a home for > people with the > > > time/energy/hardware to talk about testing? > > > I think it would be a marvelous idea, assuming > > > sufficient interest of > > > course, to continue this with 4.0-RC when that starts the > > > long arduous path > > > from -CURRENT ;) > > > > > > Cheers, > > > Colin > > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > > with "unsubscribe freebsd-qa" in the body of the message > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-qa" in the body of the message > ------_=_NextPart_001_01BF4B5F.18AB1FAC Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable RE: After 3.4 finally goes out the door

Well, I was trying to avoid the 'someone oughta' and = 'you really should' type of rambling. :) I simply propose that if we = could just gain some more time to QA the "real" release, then = we'd get some truly useful work done.

As for a standardized testing methodology or = procedure? IMHO, that would be counterproductive. Everybody's got a = different usage pattern, and different hardware, and trying to force = people into the same mold will be hard and ultimately bad and = wrong.

For instance, if you had some script to follow in = sysinstall - "test this, and then this, and then this, and test = this" - would the sysinstall bugs have been found? If end-users = with different hardware and configurations and usage patterns hadn't = tested moused, would we have shipped with roller wheel support? No, = it's people doing what they normally do, which is quite likely to be = outside of what the developer tested. This is a Good Thing, and we need = people doing their own unexpected, possibly weird, sometimes nonobvious = stuff. The developers aren't total morons and the obvious stuff = obviously works fine.

The developers already have and already run the = various regression tests that are available. A technically savvy QA = volunteer might want to run those, and certainly that's fine, but I = don't think we're short of dedicated and technical savvy developers = familiar with the OS and its internals. What we need, IMHO, is a bunch = of people putting it in real world situations, testing it under their = various application and user loads, and seeing how it does.

We had this in the form of the 3.4-RELEASE QA = volunteer team, and everybody did a great job with the time and = resources available. I hope that jkh and friends can see this, and see = that just a little bit of foresight we can make sure that the next CD = that ships holds a truly solid and kickass release.

> -----Original Message-----
> From: Jamie Norwood [mailto:mistwolf@mushhaven.net= ]
> Sent: Monday, December 20, 1999 8:00 PM
> To: Jason Young
> Cc: 'Colin'; freebsd-qa@FreeBSD.ORG; = 'jkh@freebsd.org'
> Subject: Re: After 3.4 finally goes out the = door
>
>
> I agree with this totally. I also feel someone = should step forward
> and coordinate; I didn't do more than install = and see if it worked
> as I am not familier with how things should be = tested. Also, I feel
> that some things should be made available such = as CVSUP scripts
> and such to make sure the people involved = really are testing the
> right things. Could we maybe get a 'branch' = named 'qa' that would show
> rc or qa in the uname output? That would also = make things
> easier as we
> could then test that tree for the cd, and bug = fixes that are essential
> could then be made to that tree only, and = normal developement could
> continue on the -STABLE/-RELEASE tree.
>
> Jamie
>
>
> On Mon, Dec 20, 1999 at 07:50:12PM -0600, Jason = Young wrote:
> >
> > I think it's a good idea to keep the list = around, but I
> don't think that the
> > QA volunteers were really used to their = full potential.
> >
> > I tested when I could (early in the = testing cycle), and I
> feel that most of
> > my efforts were for naught, because as = usual, a large
> percentage of the
> > commits in the testing period were very = last minute (after
> most or all of
> > the QA builds).
> >
> > Yeah, it's a free software project and = nobody's paid, and
> what gets out the
> > door is still a fine piece of work (and = major kudos to jkh
> and the rest of
> > the guys on that). But, I don't think much = meaningful
> testing of the real
> > release was accomplished due to all the = really really late
> changes. The last
> > minute "oh I really need such and = such MFC'd" trips are
> killers (sysinstall,
> > moused, you won't have to rack your brain = too hard for examples).
> >
> > I assume that the schedule for 3.5-RELEASE = is pretty much
> set already. Would
> > it be possible to back most of that = schedule up about a
> week, and in the
> > week before the CD is cut, put the = "pretty-much-final,
> we-really-mean-it,
> > won't touch it unless the release is = definitely fscked" up
> for general FTP?
> > If the schedules are set well in advance, = nobody will be hurting or
> > complaining of the loss of one week's time = in a three or
> four month -STABLE
> > release cycle. The QA volunteers need a = set release to
> assure its quality,
> > ya know? This is no extra work or major = change for anyone.
> >
> > I think that if we could get the = "frozen" release candidate
> out there in
> > front of people for at least a week or so, = we could muchly
> reduce the size
> > of our ERRATA files.
> >
> > > -----Original Message-----
> > > From: Colin [mailto:cwass99@home.com]
> > > Sent: Monday, December 20, 1999 7:18 = PM
> > > To: freebsd-qa@FreeBSD.ORG
> > > Subject: After 3.4 finally goes out = the door
> > >
> > >
> > >      I = realize we got off to a rather slow start, but all
> > > things considered
> > > better than many expected.  My = question is, is there any
> > > intent for this list
> > > to survive the release of 3.4, as = basically a home for
> people with the
> > > time/energy/hardware to talk about = testing?
> > >      I think = it would be a marvelous idea, assuming
> > > sufficient interest of
> > > course, to continue this with 4.0-RC = when that starts the
> > > long arduous path
> > > from -CURRENT ;)
> > >
> > > Cheers,
> > > Colin
> > >
> > >
> > > To Unsubscribe: send mail to = majordomo@FreeBSD.org
> > > with "unsubscribe = freebsd-qa" in the body of the message
> > >
>
>
> To Unsubscribe: send mail to = majordomo@FreeBSD.org
> with "unsubscribe freebsd-qa" in the = body of the message
>

------_=_NextPart_001_01BF4B5F.18AB1FAC-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-qa" in the body of the message