From owner-freebsd-current@FreeBSD.ORG Wed Aug 18 15:25:55 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 18DA116A4CE; Wed, 18 Aug 2004 15:25:55 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9E5F243D53; Wed, 18 Aug 2004 15:25:54 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i7IFNvaB056588; Wed, 18 Aug 2004 11:23:57 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i7IFNvJ9056585; Wed, 18 Aug 2004 11:23:57 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 18 Aug 2004 11:23:57 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: drhodus@machdep.com In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: gordon@freebsd.org cc: chris@behanna.org cc: current@freebsd.org Subject: Re: Public Access to Perforce? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Aug 2004 15:25:55 -0000 On Wed, 18 Aug 2004, David Rhodus wrote: > > That comment seems to be in stark contrast to reality. One of the most > > important goals in using Perforce to supplement CVS has been to get > > developers to stop keeping weeks or months of "in progress" changes solely > > on their notebook or workstation, and to increase collaboration > > opportunities among developers. Previously, developers would maintain > > Right, but there are many more developers outside of the people with > access to the perforce server. I don't think you'll find disagreement here. I agree with the premise, but not the conclusion: openness is good, but that doesn't mean FreeBSD is no longer an open source project. > > As already pointed out, most if not all of the interesting "in progress" > > work in Perforce is regularly and mechanically exported as patch sets or > > via cvsup by the developers, something they can now do much more easily > > now than they could before. Most of the major group work going on in > > Perforce is exported via cvsup10 (TrustedBSD, etc). In addition, many > > The trustedbsd trees are the only thing exported there, well for other > than some mostly dead trees. None of the TLS, AMD64, NETSMP, and who > knows what else since its not redly a public forum. The AMD64 work in general appears to be merged within days of being put in Perforce. The netperf work is exported as regular patch sets on the netperf web page, including careful annotation of all changes, and even an RSS feed of the change log. My understanding is that the TLS work is now merged. > > I think you'd have to work fairly hard to find open source projects that > > have no ouststanding local patch sets of as-yet uncommitted and > > So with perforce development software, fbsd will become extra stable > hence removing the need for the -current tree ? I think that does not accurately describe our intent. There will remain a -CURRENT, but by using additional branching, we can reduce the thrashing in -CURRENT by allowing work in progress development to be more parallel. When you have many dozens or hundreds of active developers working on a source tree, managing that parallelism is necessary. In previous years, this has been done using separate work areas, often based on separate CVS repositories or private developer work areas. Now, we're attempting to focus that work more centrally. For example, if you follow the netperf work, you'll see that some changes go into the netperf work area weeks or months in advance of hitting the main tree. Other changes are merged within a couple of days. This allows experimental approaches in the netperf branch that would be too unstable to put on every desktop, but that are acceptable in a more constrained and directed source tree. Each area of highly experimental work introduces substantial instability: by constraining that instability for initial testing and fielding in a specific work area, we bound the degree of universal instability better. That work will need more broad exposure in -current, but if something is known to crash 60% of the time under high load, there's no reason to defer merging it for a bit. Sometimes, this will take the form of making things conditionally compiled but using different defaults (i.e., I merged ADAPTIVE_GIANT quickly to CVS, but didn't enable it by default for a bit longer, whereas it was the default in the netperf branch). Other times, it can't be managed by conditional compilation: while roto-tilling the socket buffer locking code in steps in the netperf branch, I broken compiling and/or running for extended periods (12+ hours for compiling, and a week for running). I was then able to merge these changes in relatively clean change sets to CVS once I'd finished the grunt work. One of our hopes was to find a model that has some of the benefits of a "contrib" model, wherein only more stable/discrete change sets go into CVS, but without the more painful aspects of CVS vendor branches or the notion of the "primary" copy being maintained elsewhere. With the netperf work, for example, the CVS repository will be the master copy, and I include full revision change information when I merge from Perforce to CVS. Perforce is a branch off of CVS, not vice versa. > > experimental changes; by providing a central infrastructure to manage > > this, we're helping developers to avoid loss and collaborate better/more. > > Local and experimental changes are a necessary part of the development > > process for any reasonably large project, as the software mainline > > requires greater stability than the stability of every work in progress. > > But all of these is why we have had the -stable and -current trees. > Yes, its not the best method but at least its an method which provides > open access to the development work that is ongoing. I think you're arguing that parallelism and branching are OK as long as there's openness; I think that an argument against parallelism and branching does not serve our needs. Parallelism and branching has always happened with large open source projects, including FreeBSD, Linux, NetBSD, OpenBSD, KDE, etc; the challenge has been increasing the openness without increasing the workload. Given the scope of work that takes place, I think putting all intermediate steps in CVS in -current doesn't serve the broader community: effective large-scale changes take time to perfect, and branching gives that opportunity without exposing all the intermediate steps. Using Perforce for netperf allows me to make mis-steps or experiment without hurting other development efforts, for example. I can try a locking strategy and discover it's a bad idea without enforcing that approach on everyone. > > The unavailability of the perforce.freebsd.org web site is due to bugs in > > the older version of the Perforce web server, and that the software has > > The perforce.freebsd.org web site is a marsh pit to navigate. I think > more people would be content If the site could be cleaned up and the > method of offering a .tar.gz file of every tree on the hour for download > via the website was added. I agree. Up until now, accessibility to individual projects in Perforce has been managed by the developers working on the project. I.e., I maintain a netperf web page, netperf patch sets, a checkout of the netperf work on fxr.watson.org with diffing against various FreeBSD and *BSD revisions, etc. Centralizing that infrastructure (or at least, providing similar centralized infrastructure) takes this notion to the logical next step, increasing access to those outside the immediate committer community while maintaining the benefits we've gained by using Perforce. I think the primary disagreement I have with your comments is the assertion that Perforce has made the FreeBSD project less open: I believe it hasn't, because it's brought what were previously private personal repositories and work more into the open, whereas previously that work was still inaccessible to the broader community. It has improved parallel development capabilities through developer collaboration, multi-way branching, light-weight branching, and provided a better tool for development and communication. Open source projects are no different from close source ones in the desire to use better tools, improve developer efficiency, etc. Perforce has made a dramatic difference in our ability to do work like TrustedBSD, netperf, etc. Avoiding throwing out babies with bath water is important. I agree with your comment that this can be improved further, but that's true of most tools in the open source (and closed source) worlds. Getting the Perforce web page back online is a priority (I've CC'd Gordon to see if we can harass him into getting it online now that he's a Perforce admin). Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research