Date: Sat, 12 Jan 2008 00:41:55 +0100 From: Kris Kennaway <kris@FreeBSD.org> To: =?UTF-8?B?6Z+T5a625qiZIEJpbGwgSGFja2Vy?= <askbill@conducive.net> Cc: freebsd-chat@freebsd.org Subject: Re: Improving the handling of PR:s Message-ID: <4787FEC3.20803@FreeBSD.org> In-Reply-To: <4787E964.1080604@conducive.net> References: <478556AD.6090400@bsdforen.de> <20080110003524.GB5188@soaustin.net> <200801111935.50821.peter.schuller@infidyne.com> <20080111204148.GA4787@soaustin.net> <4787E964.1080604@conducive.net>
next in thread | previous in thread | raw e-mail | index | archive | help
韓家標 Bill Hacker wrote: > Mark Linimon wrote: >> On Fri, Jan 11, 2008 at 07:35:41PM +0100, Peter Schuller wrote: >>> But understandable or not, the problem becomes particularly >>> frustrating when it affects PR:s that contain patches. Such PR:s >>> constitute direct contributions to the project. In cases where such >>> patches are correct/good enough, the non-application of those patches >>> have what I believe to be significant effects. >> >> I think you make some excellent points -- especially with how >> understandable it is for someone who's submitted a patch which got >> ignored, to not do the work to submit the next one. >> >> With respect to patches, the two things that I've tried -- and they have >> been insufficient -- are the following: >> >> - weekly email of "PRs containing patches" >> >> - weekly email of "PRs recommended by the bugbuster team" >> >> We can make the following observations: >> >> - the "push" paradigm doesn't work particularly well. Partly this is >> due to fatigue if you try to read through all this stuff (see below). >> >> - the "recommended" list has been slightly effective, but it's going >> to take some time for it to take off. For one thing, it has been >> underpublicized; for another, we don't have the culture of committers >> getting "warm fuzzies" from committing PRs. (Since no one working >> on that kind of stuff gets paid AFAIK, that's the only positive >> feedback they're going to get.) >> >> Last week someone made the observation that if these things were instead >> updated daily and posted to a web page, that it would be far more useful. >> I've taken that up as a task. > > Observation: PR or 'other' - it is presently rather difficult to find > [older | most] patches- which reduces the opportunity for testing and > feedback, and increases the likelihood of having the situation reported > yet-again. And again.. > > so .. > > - Can there be a port category (real or 'virtuaized', added - along the > lines of: > > 'patches and workarounds' > > With caveats, of course.... but hopefully also with provision for > co-located feeback, both positive and negative. > >> >> Your other point about "latest PRs" is also a good one. That should be >> relatively easy to do. I'll take up that task. >> > > Question: Can an improved 'data mining' project add value? > > One wherein the tools to look for commonality - sometimes of the less > obvious nature - would be easier for all-hands? > > I include in this the ability to build 'reports' - not just do the > present ad hoc searches. > > If such could be done 'better', it might also benefit from crossproject > (i.e all the *BSD's) scope. > >>> * The committer may not be familiar with the code and, even though >>> the patch looks good generally, it is felt that someone familiar with >>> that part of the system needs to have a look. >> >> In particular, this hurts for areas of the system that are unmaintained. >> >>> What I suggest is a system where a group of non-committers >>> systematically pre-process PR:s, to provide feedback and additional >>> quality assurance to the committer that is to process the PR. >>> >>> This group of people could either be a self-proclaimed group of >>> volunteers, or perhaps a group of people satisfying the criteria of >>> "guy we kinda trust to do testing and provide a useful indication of >>> sanity and correctness, but not with a commit bit". >> >> So far it hasn't happened. We've set up the freebsd-bugbusters@ mailing >> list and the #freebsd-bugbusters IRC channel on EFNet (and please join >> us!) and the latter is where our last 2 bugathons took place. >> >> It's clear that there are several people who want to help process the >> PRs, and we don't have a good answer for them on "how can I contribute?". >> The existing tool, and social conventions, don't allow for non-committers >> to change PR states. As far as we've done in the past is to grant people >> "GNATS access" rights but not "commit rights", on an experimental basis. >> We've done this twice, and although it has worked well, just two people >> isn't enough. (One has gone on to become a full committer -- which is >> great!; the other current does not have as much time for FreeBSD work). >> Several hundred PRs were dealt with by these two folks, so I consider the >> experiment a success. >> >> What we used as a qualification was "track record of responding to PRs >> and >> questions on mailing lists", fwiw. >> >>> One possible answer to this that I have gotten in the past with >>> another project, is that noone is stopping me from just grabbing a >>> bunch of PR:s and posting follow-ups saying I've tested something, or >>> otherwise giving feedback. >> >> Yes, but that's open-loop as well. It's the same situation as with >> submitting >> the patch in the first place: it's going to get frustrating if no >> committer >> picks it up and commits it. >> >> There's not too many people so thick-skinned and persistent as to keep >> going forward when no one's using their work. >> >>> For example, patches with a high confidence of correctness due to >>> many people affirming that they have tested it could be automatically >>> prioritized for committers to deal with, and there will be a clear >>> and systematic record of what testing was done, and by who. >> >> Right. The "priority" field in GNATS has been so abused as to become >> useless. (People assume that setting it higher will cause some kind of >> magic to happen.) >> >> My current thinking is that what committers ought to see is a metric of >> correctness, and a metric of priority, _as set by someone who's vetted >> the PR_. The weekly mailings are too poor an approximation of either >> to be useful. >> >> Adding the second metric would cure one problem that you don't mention -- >> which is that few people have the interest and patience to plow through >> N-thousand PRs. It's not humanly possible to look at them all -- even >> the new ones as they come in. There's simply too many. So, you create >> an expectation "why bother, there's so many anyways". We need to break >> that chain of expectation. A good fix is a good fix. The PR count will >> never get to zero; I (with bugmaster hat on) would be thrilled if we can >> get to the point of just steady-state. > > DB Admins, SysAdmins, 'Management' need BSD, too and not all the work > that needs to be done requires 'C' expertise. > > Some of us also have spare bandwidth and hardware as well as time, so... > >> >>> When I talk about priority above, I do not mean to imply that any >>> committer should work on things they do not want to work on. But I >>> have to assume that a lot of patches get committed because a >>> committer decided to go and pick a few PR:s to process, rather than >>> the committer necessarily has a burning interest in that particular >>> patch. >> >> I think most get committed because a committer sees a PR come in on the >> mailing list and grabs it. Much less often do committers go through the >> database looking for things to fix. Again, the lousy "search/browse" >> capabilities of the existing tool let us down here. >> >>> As such, if said committer could spend those <n> minutes closing five >>> times as many PR:s because the up-front confidence in the correctness >>> of the patch is so much higher, perhaps it would help the system to >>> "scale", moving some of the burden off the committers. >> >> Right. What we need is to start small and create a _culture_ of where >> it's fun (or at least intellectually interesting) to work on this stuff. >> I think the IRC channel may still be the best bet for this. >> >> Once I finish fiddling around with the sparc64-6 and sparc64-7 release >> builds (the first approximation is finished, but I believe we can still >> add some more packages), I intend to task-switch over to looking towards >> defining a PR workflow and floating some proposals. I hope to have >> something concrete to present at BSDCan. Please join us on bugbusters@ >> to discuss any more ideas, too. >> >> mcl > > Coders will always set their own priorities. Even when working for a wage. > > ;-) > > But all-hands benefit from greater stability and fewer 'critical' things > breaking, so rational self-interest might be more effectively applied by > each participant if the 'big picture' - in several flavors - was simply > published in some 'better' manner. > > You've ID'ed some of the possibilities - there are bound to be more. > > Perhaps some of the committers - or at least the project in general - > could benefit from more 'virtual office staff' - folks who specifically > are NOT coders - to remove obstacles and help leverage their expertise? > > Background research, documentation, finding and enlisting testers who > can help, building and maintaining databases - *doing* tests. Some of > that is more amenable to being 'directed' than the coding efforts > themselves. Different personality types... > > JMHK$2CW.. Hi Bill, After your last anti-FreeBSD rant didn't you promise to unsubscribe from the mailing lists and not come back? What went wrong? :( Kris
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4787FEC3.20803>