From owner-freebsd-bugbusters@FreeBSD.ORG Thu Feb 4 10:27:05 2010 Return-Path: Delivered-To: freebsd-bugbusters@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9CD121065679 for ; Thu, 4 Feb 2010 10:27:05 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (lefty.soaustin.net [66.135.55.46]) by mx1.freebsd.org (Postfix) with ESMTP id 7DBF88FC13 for ; Thu, 4 Feb 2010 10:27:05 +0000 (UTC) Received: by mail.soaustin.net (Postfix, from userid 502) id 86F008C060; Thu, 4 Feb 2010 04:09:48 -0600 (CST) Date: Thu, 4 Feb 2010 04:09:48 -0600 From: Mark Linimon To: tyler@monkeypox.org Message-ID: <20100204100948.GA11167@lonesome.com> References: <20100117213049.GA2259@kiwi.sharlinx.com> <877hr9ltym.fsf@kobe.laptop> <20100201072150.GD30608@kiwi.sharlinx.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100201072150.GD30608@kiwi.sharlinx.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hackers@freebsd.org, Giorgos Keramidas , freebsd-bugbusters@FreeBSD.org Subject: Re: Weekend PR smashing X-BeenThere: freebsd-bugbusters@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Coordination of the Problem Report handling effort." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Feb 2010 10:27:05 -0000 [adding freebsd-bugbusters@ to the Cc:] I'm sorry I didn't respond to your earlier message. I am currently way behind on tasks that I've already promised people. The KDE howto that you cited (http://quality.kde.org/develop/howto/howtobugs.php) is quite a good one. Our pr-guidelines document isn't as thorough. Part of the problem is that we don't have enough metadata in GNATS to capture some of the things that we would like bugbusters to do, e.g., as they suggest: - attempting to reproduce an 'unconfirmed' bug and change it to 'new' The closest that we have is the 'analyzed' state, which we have used in the past to indicate 'confirmed'. If you can indeed reproduce a bug, it's fair enough to let bugmeister@ know and we can change the state. Alternatively, you can submit a followup and suggest that. Several of the folks on the bugbusting team (e.g. people who have GNATS access) monitor the mailing lists and try to track followups. (I personally track bugs@, ports-bugs@, and a few of the others, but not some of the specialty ones like net@.) - check if a report is a duplicate Reports that are duplicates indicate that various users are being affected by one underlying problem. At one point I was trying to gather them all into a page. I was hoping more people would do the analysis and send me additions for it. However, it looks as though the script that generates that page has rotted. I'll re-add it to my list of things-to-do ... http://people.freebsd.org/~linimon/studies/prs/well_known_prs.html At this point it may be easier to look at the template that I grind up to produce that page: http://people.freebsd.org/~linimon/wellknown.prs It's clearly over a year stale, and could thus use a new set of eyes. - add '(regression)' to the title if indicated I think we've done a fairly good job of getting '[regression]' (our styling of the same idea) into the existing ones, and keeping up with new ones as they come in. Similarly, we've done pretty well with '[patch]' AFAIK. That's my reaction to some specifics on the the KDE page. Now for some more general comments. We have more kern/ PRs than any other category. This category is overloaded to mean both kernel, libraries, networking, and device drivers. http://people.freebsd.org/~linimon/studies/prs/pr_tag_index.html makes this much more tractable. (I have other ideas about how to do much better, but see below.) Whether or not existing PRs are still relevant seems to vary a bit by sub-category. There would also a slightly different way of looking at things, the ones with '[panic]' in the Synopsis. Hmm, I thought there was such a page, but it doesn't seem so. I'll put it on my list to create one. As for the bin/ PRs, you'd be surprised at how many of them are still relevant, even after several years. I've resisted the suggestion that others have made in the past just to close them for this reason. (In the past I have made an effort to contact submitters and ask them if they are still experiencing the problem, but portmgr duties have kept me away from that for a long time. This is something else where we need help.) The i386 and amd64 PRs are more likely to be "can't install/run on a particular piece of hardware", and generally require the most interaction with users to isolate the problem (e.g. to ACPI interacting badly with a buggy BIOS; irq problems; devices not being supported by our current drivers; and so forth.) To work with these people need to have a bit of experience with low-level debuggers and how to examine stack traces. > one of the things that concerns me is the sheer number of PRs with > patches that either have been committed without the PRs being updated Well, that's a task that needs more people working on. You can start by looking at: http://people.freebsd.org/~linimon/studies/prs/prs_possibly_committed.html which is PRs that have had followups appended to their Audit-Trail by the checkin scripts. If these changes have already been merged to all the relevant -stable branches, they should be closed; otherwise, if they are not already set to 'patched', then they should be, and assigned to the committer who made the change. > or the patches are simply sitting idly in PRs. There are certainly a large number of these. IWBNI we could get more of them into the 'analyzed' state to claim 'we think this patch might solve the problem'. > The list by the bugbusters waiting for committers to check them out > is pretty huge as well: Yes, I'm well aware of that, since I set it up :-) This goes to the more general problem that it's more fun to add new things than it is to fix existing things. FreeBSD has traditionally done much better at adding new features than in the more mundane maintenance work. This is a problem not just with FreeBSD or even open source projects in general, but all software. What's interesting is that this situation is slowly changing (again IMHO). Over the past year we have seen several people 'graduate' from being on the bugbusting team to having src commit bits. Slowly some of the PRs that fit the classical "user-annoyance" classification are being fixed. As more people start to do this, the more tractable the problem starts to seem, even for long-time committers who may have despaired over the state of the database over the years. (It is also worth mentioning that with the help of Remko, Gavin, and others, the Synopsis lines of the existing PRs are now much more likely to be useful and correct than even, say, two years ago; and thus, the data-harvesting pages I put up became possible.) Building up people's belief that the overall problem is tractable is a chicken-and-egg problem. Again, compared to where we were, a great deal of progress has been made, albeit slowly. > It's a little difficult to muster up too much motivation to fix issues when > the fixes will then sit waiting for committer review for months on end. My suggestion is to avoid looking at the overall number of PRs, and instead find things that interest you to work on. Are you interested in helping to triage existing PRs; or perhaps, in responding to new PRs that come in and work with users to try to narrow down the problem (especially useful for install/boot issues and regressions); or perhaps in one particular part of the system such as networking? Or perhaps the "overall user experience" things such as sysinstall? > Without annoying committers, is there any way I can help get patches > "through" and into the tree in less than a lunar cycle? ;) The only way that I have found so far is to create the specialized reports (both HTML and email) and try to publicize them. New ideas are welcome. Lastly, my (currently stalled) project to create a prototype for a new PR system tries to deal with some of the limitations of, and frustrations with, GNATS. For instance, see my presentation at BSDCan 2008: http://people.freebsd.org/~linimon/presentations/prs2008/ But I'll reiterate, the way to go (IMHO) is to pick one thing out of the "my suggestion" paragraph above that you think might be interested in, and start with that. I'll be happy to listen to any feedback you might have. mcl From owner-freebsd-bugbusters@FreeBSD.ORG Fri Feb 5 00:32:25 2010 Return-Path: Delivered-To: freebsd-bugbusters@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C623106566B; Fri, 5 Feb 2010 00:32:25 +0000 (UTC) (envelope-from pgollucci@p6m7g8.com) Received: from EXHUB015-4.exch015.msoutlookonline.net (exhub015-4.exch015.msoutlookonline.net [207.5.72.96]) by mx1.freebsd.org (Postfix) with ESMTP id 86B578FC13; Fri, 5 Feb 2010 00:32:25 +0000 (UTC) Received: from philip.hq.rws (174.79.184.239) by smtpx15.msoutlookonline.net (207.5.72.103) with Microsoft SMTP Server (TLS) id 8.2.176.0; Thu, 4 Feb 2010 16:22:19 -0800 Message-ID: <4B6B64BA.9020803@p6m7g8.com> Date: Fri, 5 Feb 2010 00:22:18 +0000 From: "Philip M. Gollucci" Organization: P6M7G8 Inc. User-Agent: Thunderbird 2.0.0.23 (X11/20091208) MIME-Version: 1.0 To: Mark Linimon References: <20100117213049.GA2259@kiwi.sharlinx.com> <877hr9ltym.fsf@kobe.laptop> <20100201072150.GD30608@kiwi.sharlinx.com> <20100204100948.GA11167@lonesome.com> In-Reply-To: <20100204100948.GA11167@lonesome.com> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit Cc: tyler@monkeypox.org, freebsd-bugbusters@freebsd.org, Giorgos Keramidas , freebsd-hackers@freebsd.org Subject: Re: Weekend PR smashing X-BeenThere: freebsd-bugbusters@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Coordination of the Problem Report handling effort." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2010 00:32:25 -0000 Mark Linimon wrote: > [adding freebsd-bugbusters@ to the Cc:] > http://people.freebsd.org/~linimon/studies/prs/well_known_prs.html You also might find the following script that wraps all of Mark's work a tiny bit a little better to get a lot of information on one page. I use it for ports but should translate over to src quite well. http://people.freebsd.org/~pgollucci/myprs.pl.txt which in its current state makes this http://people.freebsd.org/~pgollucci/myprs.html -- ------------------------------------------------------------------------ 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C Philip M. Gollucci (pgollucci@p6m7g8.com) c: 703.336.9354 VP Apache Infrastructure; Member, Apache Software Foundation Committer, FreeBSD Foundation Consultant, P6M7G8 Inc. Sr. System Admin, Ridecharge Inc. Work like you don't need the money, love like you'll never get hurt, and dance like nobody's watching. From owner-freebsd-bugbusters@FreeBSD.ORG Fri Feb 5 08:37:08 2010 Return-Path: Delivered-To: freebsd-bugbusters@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0958E106568B; Fri, 5 Feb 2010 08:37:08 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (lefty.soaustin.net [66.135.55.46]) by mx1.freebsd.org (Postfix) with ESMTP id DDCEF8FC12; Fri, 5 Feb 2010 08:37:07 +0000 (UTC) Received: by mail.soaustin.net (Postfix, from userid 502) id 4A58D8C060; Fri, 5 Feb 2010 02:37:07 -0600 (CST) Date: Fri, 5 Feb 2010 02:37:07 -0600 From: Mark Linimon To: tyler@monkeypox.org Message-ID: <20100205083707.GA30463@lonesome.com> References: <20100117213049.GA2259@kiwi.sharlinx.com> <877hr9ltym.fsf@kobe.laptop> <20100201072150.GD30608@kiwi.sharlinx.com> <20100204100948.GA11167@lonesome.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100204100948.GA11167@lonesome.com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hackers@freebsd.org, Giorgos Keramidas , freebsd-bugbusters@FreeBSD.org Subject: Re: Weekend PR smashing X-BeenThere: freebsd-bugbusters@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Coordination of the Problem Report handling effort." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2010 08:37:08 -0000 On Thu, Feb 04, 2010 at 04:09:48AM -0600, Mark Linimon wrote: > Reports that are duplicates indicate that various users are being affected > by one underlying problem. At one point I was trying to gather them all > into a page. I was hoping more people would do the analysis and send me > additions for it. However, it looks as though the script that generates > that page has rotted. I'll re-add it to my list of things-to-do ... > > http://people.freebsd.org/~linimon/studies/prs/well_known_prs.html This report is now fixed. > We have more kern/ PRs than any other category. This category is > overloaded to mean both kernel, libraries, networking, and device > drivers. http://people.freebsd.org/~linimon/studies/prs/pr_tag_index.html > makes this much more tractable. I forgot to mention the 2-level hierarchy that I have set up, where you can look at PRs starting with e.g. "disk/driver" and then drill down to a page that references all the related PRs by manpage. It may have been just as well, since the report had also gotten stale. However, it is once again up-to-date: http://people.freebsd.org/~linimon/studies/prs/prs_for_all_groups.html > There would also a slightly different way of looking at things, the ones > with '[panic]' in the Synopsis. Hmm, I thought there was such a page, > but it doesn't seem so. I'll put it on my list to create one. Now created: http://people.freebsd.org/~linimon/studies/prs/prs_for_tag_panic.html Finally, I have fixed other problems, such as broken links, in other various pages under http://people.freebsd.org/~linimon/studies/prs/ mcl