From owner-freebsd-hackers Fri Jun 4 5:29: 2 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from iquest3.iquest.net (iquest3.iquest.net [209.43.20.203]) by hub.freebsd.org (Postfix) with SMTP id D81191507E for ; Fri, 4 Jun 1999 05:28:59 -0700 (PDT) (envelope-from toor@dyson.iquest.net) Received: (qmail 4858 invoked from network); 4 Jun 1999 12:28:55 -0000 Received: from dyson.iquest.net (198.70.144.127) by iquest3.iquest.net with SMTP; 4 Jun 1999 12:28:55 -0000 Received: (from root@localhost) by dyson.iquest.net (8.9.1/8.9.1) id HAA06346; Fri, 4 Jun 1999 07:28:49 -0500 (EST) From: "John S. Dyson" Message-Id: <199906041228.HAA06346@dyson.iquest.net> Subject: Re: 3.2-stable, panic #12 In-Reply-To: <59278.928486484@zippy.cdrom.com> from "Jordan K. Hubbard" at "Jun 4, 99 01:54:44 am" To: jkh@zippy.cdrom.com (Jordan K. Hubbard) Date: Fri, 4 Jun 1999 07:28:49 -0500 (EST) Cc: brian@Awfulhak.org, dillon@apollo.backplane.com, dyson@iquest.net, ahasty@mindspring.com, crossd@cs.rpi.edu, freebsd-hackers@FreeBSD.ORG, schimken@cs.rpi.edu X-Mailer: ELM [version 2.4ME+ PL32 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > I'm sure that the fact that -release ended up with such obvious > > instabilities was out of your control (IMHO RELENG_3 shouldn't have > > been dubbed -stable 'till 3.2 was tagged), but I'd bet that this did > > Just a side comment on this - there was tremendous flammage that the > RELENG_3 branch wasn't created with 3.0 and when 3.1 came out, the > most frequently asked question I got was "when are you going to > fucking branch already?" > > My point is that you very definitely cannot please everyone all of the > time. Do it quick and some will scream. Do it slow and others will > scream. Either way, you're getting screamed at. > There are a couple of kinds of errors that can happen with hasty (not Amancio :-)) code changes... The most insidious kind of problem is when various newly created problems cascade without being detected or controlled. As the new problems or regressions accumlate, those problems become progressively more difficult to unwind and recover from. This can become extremely difficult when features (or desirable behaviors) are lost and new, orthogonal features are added either simultaneously or in quick succession. The straight forward and obvious "it doesn't work", or "it has gotten terribly slow all of a sudden" are both really easy to deal with compared with other manifestions of code damage. It is the "the system worked better in FreeBSD VX.Y and we cannot figure out which of the changes make the system act differently in odd ways" that is the real intellectual challenge. Minor "it's broken" problems aren't what I have been most worried about -- those can be fixed, often without a detailed understanding of the code and how the various pieces interoperate. What needs to be carefully reviewed and watched are the cumulative changes and the negative effects due to a cascade of unintended side effects and removed features. Alot of the individual "features" in the code aren't dot items that can be placed on marketing literature. Some of the more interesting features in the code are a result of multiple individual behaviors designed into the code with purpose that work together to produce the set of behaviors that have been historically known as FreeBSD's generally superior performance. (Not that FreeBSD's behavior is perfect, but there is some historical work that I have always been willing to help people working on the code understand and take advantage of.) There is absolutely no reason why fixing the straight forward bugs, not so straight forward bugs, or cleaning up the way that the source looks in any way need to normally have signficant (or stealthy) negative side effects. It is clear that it is very difficult to make substantial modifications to the complex code in the system without unintended side-effects, and it is very unwise to avoid consulting with individuals who might have a different (or longer term and/or historical) view as to how the various sub-systems work. When DG and I were creating the original behaviors that have historically been associated with the FreeBSD type performance, it would have been wonderful to have had lots of support offered and committed from those people who had significant experience with the original code and understanding of other superior algorithms and techniques. Recently, a year or so before I left, I had started getting help from Alan Cox , and his help was greatly appreciated by me, and I always tried to credit his work as well as resonable (he is indeed an individual who I very seriously respect.) Frankly, his understanding of major sections of the code is often beyond my own (it is difficult for me to precisely evaluate it, but his observations have been quite impressive, and his work has shown understanding much beyond the surface.) I suggest that embracing other "experts" input is a desirable behavior, because no-one that I have seen on the team (either core or contributors) is so expert as to never need input and help. Frankly, utilizing me for input on certain parts of the code only after waiting until the problem is out of control (either feature having been removed without knowing or a solution has been created that would have been done better or easier with input from others) is misuse or nonuse of a potentially valuable resource. No-one is infallable, but being able to admit fallability and use input from others to mitigate effects of ones own limitations is critical when working on complex packages such as embodied in the FreeBSD memory and I/O mgmt code. None of the stuff is perfect, but just because there are problems here and there doesn't mean that those who created the works didn't understand them. In fact, some of the "suboptimal" behavior might have been as a result of a tradeoff that isn't obvious from a superficial code review. Alot of the work done by DG and me was original, and often created relatively from scratch, with requirements and goals very different from equivalent works of the time. If we had a blueprint to work from (or if we wanted to create another system that acted like all of the others), then the work would have been much easier, and would have been much less complex. I have indeed found that ALC (who is at least in some areas as competent or more so that me) is sometimes willing to use me for review or ideas, because it is apparent that he recognizes that even extremely competent people can gain from others (even less experienced) insight. I suggest that a FreeBSD culture be nurtured that encourages collaboration and cooperation, and encourage healthy egos by recognizing that even experts don't stand alone and aren't as effective in creative endeavours if they try to do it all themselves. John To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message