Date: Tue, 07 Aug 2001 10:26:49 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: j mckitrick <jcm@FreeBSD-uk.eu.org> Cc: Wes Peters <wes@softweyr.com>, Ted Mittelstaedt <tedm@toybox.placo.com>, freebsd-advocacy@FreeBSD.ORG Subject: Re: time to step up to the SMP plate? Message-ID: <3B7024D9.55EAFBDD@mindspring.com> References: <003f01c11cd7$563d7860$1401a8c0@tedm.placo.com> <3B6EAD30.8161F4E5@softweyr.com> <3B6F9145.54945750@mindspring.com> <20010807121133.A73889@dogma.freebsd-uk.eu.org>
next in thread | previous in thread | raw e-mail | index | archive | help
j mckitrick wrote: > > | FreeBSD is setting itself up to be similarly limited; Linux > | already is hitting its head on the same issue. > > Is it possible the real limit here is just the practical limit of open > source development? When almost everyone is a volunteer working in their > spare time, not only is management and design difficult, development can > slow to a crawl for a myriad reasons. > > I've seen more than one comment lately where this is becoming a concern. I remain convinced that the major limitations in Open Source developement models are emergent properties of the tools that are being used. It's true that there are a number of obstacles to sustained developement inherent in the model, but its also true that commercial interests and academic research can more than make up for any inhernet shortcomings, if the projects can be convinced to accept the fruits of that labor. When I first wrote loadable kernel modules and released them shortly before the Novell purchase of USL back in 1994, they were a small project that had been carried out in a (mostly) academic setting: my first prototypes were in 1993 on an SVR3 system in a University lab, since the GNU toolchain didn't support PIC compilation, until Jeffrey Hsu added support to it. Even then, it wasn't until several months after NetBSD adopted the code that FreeBSD grudgingly adopted the code as well. It's also ironic, I think, that the KLD stuff there today is the original design I wanted: a small linker in the kernel, capable of directly loading modules without outside intervention, but that design was shouted down as causing "too much kernel bloat". The funny thing about that is that it has resulted in exactly the opposite effect, as most drivers are now available as kernel modules, and aren't loaded unless the hardware is present, or the code is in the boot path. I could make similar claims about my SYSINIT() code, which Julian integrated over a number of strong objections, but is now taken for granted as something that can't be lived without, and has grown beyond the original scope of the work, in order to support tunables, both at boot time and run time, the list of which is aggregated out of the available code. So isolated small projects can result in big returns, though there have been isolated mishaps, like the failure to integrate Jack Vogel's work at Sun Microsystems into a functioning SMP for FreeBSD, until much later in the game: FreeBSD had a working SMP version, for some definitions of working, as of October 27th, 1995; the code provided much of the basis for the work which was eventually integrated, although it passed from Jack's hands to mine to Peter and Steve, and the rest of the people who ended up integrating it into the then much-changed FreeBSD tree. Things like Luigi's SACK, FACK, and TSACK implementations have pretty much languished; TSACK isn't really useful any more, now that Microsoft clients support SACK directly. Similarly, the reimplementations at Pittsburgh have failed to be integrated, even though they were developed on FreeBSD, and QLinux has LRP and Fair Share shecduling, even though the Rice University work was all done on FreeBSD boxes, once it was broken out of the Scala Server Project. There are literally tens of dozens of similar failures to execute in the past of most large Open Source projects, and, it seems to me, more in the future. In FreeBSD's case, it's mostly an issue of the tools being unable to handle multiple lines of developement, which has resulted in the "One True HEAD Branch" phenomenon, where work not occurring on -current is often discarded, due to the emotional investment people have in -current. Linux avoided this serialization problem, only to fall victim to its own similar issue of serializing through Linus (or Alan Cox), for lack of a source code control system. Linus has really never articulated well his gut reaction to source code control, instead engaging in much hand waving, but it's clear that he sees the danger of being locked into particular emergent properties by the tools, even if he doesn't give very rational justification for his gut feelings. We had a nice example of the "gated community" effect when Matt Dillon became able to dedicate a significant amount of his time to working on FreeBSD, after Best Internet, the company he helped found, was acquired by by Verio. The gatekeepers were simply unable to keep up with his rate of contribution of code, made over 8 and 12 hour days, and they squelched him, taking away his commit bits until he agreed to slow down to a much slower pace, where they could keep up with reviewing code. I think that FreeBSD has suffered from incidents like this in the past (e.g. John Dyson leaving the community was a major loss, IMO), and it comes down to the "One True HEAD Branch" phenomenon, once again, where the only way for ideas to compete openly is for them to be implemented in different distributions... since all of the BSD projects have the same form-factor: core team of gate keepers, committers, and the threat of removing commit bits temporarily, until someone complies. The bottom line is that real developement on anything but the HEAD branch rarely occurs (except for the commercially funded developement, which can't risk -current in a shipping product), and so their work occurs on -stable, which ends up being "the dead branch", where MFC is the rule, and MTC rarely happens (MFC = Merge From Current; MTC = Merge To Current). For example, I have several changes that I've done to FreeBSD 4.3; one up'ed the open network connection limitation from 32k, and has been tested up to 1.6M simultaneous connections. With slightly more time, I could easily push this to 2M or further. I also have changes to the TCP/IP stack, as well as several other changes to network drivers and such, which I believe could yield a 40% or better performance improvement on the top end, and minimally a 30% improvement in overall throughput capacity under extreme load (as opposed to falling off to near zero). One of the changes effectively eliminates NETISR: Van Jacobson's "holy grail". Although the company I work for seems willing to let the code back into the main line FreeBSD tree after two product release cycles, if only to offload the maintenance burden, and so our implementation "wins", it will probably not make it back into FreeBSD, since the changes are against 4.3-stable, not -current, and -current is moving at what commercially can only be considered a reckless speed, in directions not entirely justified by research or the Computer Science literature (certainly, no one is citing papers or research which justify the design decisions being made). So getting back to the inital point, I think it is a tools issue, probably more than anything else, which has the emergent property of selecting one "blessed" branch, and sending all the rest off to Coventry. I've only been surpriased in this opinion once so far, and I have been involved behind the scenes or out front in the genesis of no less than six Open Source projects. The surprise came from the OpenLDAP project, which took the University of Michigan code, and applied "The FreeBSD Patches" -- 120k of diffs from me, collected in "patchkit style" from a number of sources, particularly Critical Angle's site, which archived, but didn't organize or review the patches -- and then imported the UMich code, the patches, and then went on from there. My surprise arose from what I contend will be the next battlefield in Open Source vs. Commercial developement: complexity. I felt that it would not be possible, in the context of an Open Source project, to implement the LDAPv3 protocol. Kurt Zeilenga, happily, proved me wrong, mostly because of the commercial support of the Net Boolean... and his position, relative to the project, as an "alive and kicking" project founder. I believe had it not been for that, the OpenLDAP project would have failed to achive LDAPv3 interoperability. It seems like several commercial companies have latched onto "the complexity defense" to throw up barriers to entry, in an attempt to keep Open Source out of their mainstream commercial markets. I watch the IETF very closely, and participate when I feel it's needed, and I have to say I've recently seen some of the most arcane and unnecessarily complex RFCs shoved through the IETF standardization process by large companies working in concert, in what can only be described as protectionist tactics. Hopefully, Open Source will be able to overcome its inherit handicaps; it may be that FreeBSD switches to using Perforce, or some other tool that gets rid of the "One True Head Branch" phenomenon, despite the commercial use pain that such a switch would entail (P4 is free for Open Source, but not for commercial companies tracking an Open Source project, whose repository is only available in P4). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-advocacy" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3B7024D9.55EAFBDD>