From owner-freebsd-current@FreeBSD.ORG Fri Jan 11 22:10:50 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DE3816A468 for ; Fri, 11 Jan 2008 22:10:50 +0000 (UTC) (envelope-from askbill@conducive.net) Received: from conducive.net (lindfield.ch [203.194.153.81]) by mx1.freebsd.org (Postfix) with ESMTP id E970B13C46B for ; Fri, 11 Jan 2008 22:10:49 +0000 (UTC) (envelope-from askbill@conducive.net) Received: from c-75-75-30-250.hsd1.va.comcast.net ([75.75.30.250]:64960 helo=pb.local) by conducive.net with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.63 (FreeBSD)) (envelope-from ) id 1JDS5P-000O59-5I for freebsd-current@freebsd.org; Fri, 11 Jan 2008 22:10:48 +0000 Message-ID: <4787E964.1080604@conducive.net> Date: Fri, 11 Jan 2008 22:10:44 +0000 From: =?UTF-8?B?6Z+T5a625qiZIEJpbGwgSGFja2Vy?= User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.2) Gecko/20070221 SeaMonkey/1.1.1 MIME-Version: 1.0 To: freebsd-current@freebsd.org References: <478556AD.6090400@bsdforen.de> <20080110003524.GB5188@soaustin.net> <200801111935.50821.peter.schuller@infidyne.com> <20080111204148.GA4787@soaustin.net> In-Reply-To: <20080111204148.GA4787@soaustin.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Improving the handling of PR:s X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Jan 2008 22:10:50 -0000 Mark Linimon wrote: > On Fri, Jan 11, 2008 at 07:35:41PM +0100, Peter Schuller wrote: >> But understandable or not, the problem becomes particularly frustrating when >> it affects PR:s that contain patches. Such PR:s constitute direct >> contributions to the project. In cases where such patches are correct/good >> enough, the non-application of those patches have what I believe to be >> significant effects. > > I think you make some excellent points -- especially with how > understandable it is for someone who's submitted a patch which got > ignored, to not do the work to submit the next one. > > With respect to patches, the two things that I've tried -- and they have > been insufficient -- are the following: > > - weekly email of "PRs containing patches" > > - weekly email of "PRs recommended by the bugbuster team" > > We can make the following observations: > > - the "push" paradigm doesn't work particularly well. Partly this is > due to fatigue if you try to read through all this stuff (see below). > > - the "recommended" list has been slightly effective, but it's going > to take some time for it to take off. For one thing, it has been > underpublicized; for another, we don't have the culture of committers > getting "warm fuzzies" from committing PRs. (Since no one working > on that kind of stuff gets paid AFAIK, that's the only positive > feedback they're going to get.) > > Last week someone made the observation that if these things were instead > updated daily and posted to a web page, that it would be far more useful. > I've taken that up as a task. Observation: PR or 'other' - it is presently rather difficult to find [older | most] patches- which reduces the opportunity for testing and feedback, and increases the likelihood of having the situation reported yet-again. And again.. so .. - Can there be a port category (real or 'virtuaized', added - along the lines of: 'patches and workarounds' With caveats, of course.... but hopefully also with provision for co-located feeback, both positive and negative. > > Your other point about "latest PRs" is also a good one. That should be > relatively easy to do. I'll take up that task. > Question: Can an improved 'data mining' project add value? One wherein the tools to look for commonality - sometimes of the less obvious nature - would be easier for all-hands? I include in this the ability to build 'reports' - not just do the present ad hoc searches. If such could be done 'better', it might also benefit from crossproject (i.e all the *BSD's) scope. >> * The committer may not be familiar with the code and, even though the patch >> looks good generally, it is felt that someone familiar with that part of the >> system needs to have a look. > > In particular, this hurts for areas of the system that are unmaintained. > >> What I suggest is a system where a group of non-committers systematically >> pre-process PR:s, to provide feedback and additional quality assurance to the >> committer that is to process the PR. >> >> This group of people could either be a self-proclaimed group of volunteers, or >> perhaps a group of people satisfying the criteria of "guy we kinda trust to >> do testing and provide a useful indication of sanity and correctness, but not >> with a commit bit". > > So far it hasn't happened. We've set up the freebsd-bugbusters@ mailing > list and the #freebsd-bugbusters IRC channel on EFNet (and please join > us!) and the latter is where our last 2 bugathons took place. > > It's clear that there are several people who want to help process the > PRs, and we don't have a good answer for them on "how can I contribute?". > The existing tool, and social conventions, don't allow for non-committers > to change PR states. As far as we've done in the past is to grant people > "GNATS access" rights but not "commit rights", on an experimental basis. > We've done this twice, and although it has worked well, just two people > isn't enough. (One has gone on to become a full committer -- which is > great!; the other current does not have as much time for FreeBSD work). > Several hundred PRs were dealt with by these two folks, so I consider the > experiment a success. > > What we used as a qualification was "track record of responding to PRs and > questions on mailing lists", fwiw. > >> One possible answer to this that I have gotten in the past with another >> project, is that noone is stopping me from just grabbing a bunch of PR:s and >> posting follow-ups saying I've tested something, or otherwise giving >> feedback. > > Yes, but that's open-loop as well. It's the same situation as with submitting > the patch in the first place: it's going to get frustrating if no committer > picks it up and commits it. > > There's not too many people so thick-skinned and persistent as to keep > going forward when no one's using their work. > >> For example, patches with a high confidence of correctness due to many people >> affirming that they have tested it could be automatically prioritized for >> committers to deal with, and there will be a clear and systematic record of >> what testing was done, and by who. > > Right. The "priority" field in GNATS has been so abused as to become > useless. (People assume that setting it higher will cause some kind of > magic to happen.) > > My current thinking is that what committers ought to see is a metric of > correctness, and a metric of priority, _as set by someone who's vetted > the PR_. The weekly mailings are too poor an approximation of either > to be useful. > > Adding the second metric would cure one problem that you don't mention -- > which is that few people have the interest and patience to plow through > N-thousand PRs. It's not humanly possible to look at them all -- even > the new ones as they come in. There's simply too many. So, you create > an expectation "why bother, there's so many anyways". We need to break > that chain of expectation. A good fix is a good fix. The PR count will > never get to zero; I (with bugmaster hat on) would be thrilled if we can > get to the point of just steady-state. DB Admins, SysAdmins, 'Management' need BSD, too and not all the work that needs to be done requires 'C' expertise. Some of us also have spare bandwidth and hardware as well as time, so... > >> When I talk about priority above, I do not mean to imply that any committer >> should work on things they do not want to work on. But I have to assume that >> a lot of patches get committed because a committer decided to go and pick a >> few PR:s to process, rather than the committer necessarily has a burning >> interest in that particular patch. > > I think most get committed because a committer sees a PR come in on the > mailing list and grabs it. Much less often do committers go through the > database looking for things to fix. Again, the lousy "search/browse" > capabilities of the existing tool let us down here. > >> As such, if said committer could spend those minutes closing five times as >> many PR:s because the up-front confidence in the correctness of the patch is >> so much higher, perhaps it would help the system to "scale", moving some of >> the burden off the committers. > > Right. What we need is to start small and create a _culture_ of where > it's fun (or at least intellectually interesting) to work on this stuff. > I think the IRC channel may still be the best bet for this. > > Once I finish fiddling around with the sparc64-6 and sparc64-7 release > builds (the first approximation is finished, but I believe we can still > add some more packages), I intend to task-switch over to looking towards > defining a PR workflow and floating some proposals. I hope to have > something concrete to present at BSDCan. Please join us on bugbusters@ > to discuss any more ideas, too. > > mcl Coders will always set their own priorities. Even when working for a wage. ;-) But all-hands benefit from greater stability and fewer 'critical' things breaking, so rational self-interest might be more effectively applied by each participant if the 'big picture' - in several flavors - was simply published in some 'better' manner. You've ID'ed some of the possibilities - there are bound to be more. Perhaps some of the committers - or at least the project in general - could benefit from more 'virtual office staff' - folks who specifically are NOT coders - to remove obstacles and help leverage their expertise? Background research, documentation, finding and enlisting testers who can help, building and maintaining databases - *doing* tests. Some of that is more amenable to being 'directed' than the coding efforts themselves. Different personality types... JMHK$2CW.. Bill