Date: Thu, 21 Jul 2005 20:00:40 +0100 (BST) From: Robert Watson <rwatson@FreeBSD.org> To: Alexey Yakimovich <aiy@ferens.net> Cc: 'Marc Olzheim' <marcolz@stack.nl>, freebsd-stable@FreeBSD.org Subject: RE: Quality of FreeBSD Message-ID: <20050721194500.W9208@fledge.watson.org> In-Reply-To: <200507211803.j6LI34dV005050@ferens.net> References: <200507211803.j6LI34dV005050@ferens.net>
next in thread | previous in thread | raw e-mail | index | archive | help
This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-1481328753-1121972440=:9208 Content-Type: TEXT/PLAIN; charset=iso-8859-1; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Thu, 21 Jul 2005, Alexey Yakimovich wrote: > First of all thank you very much all for your replies. > I just want to add some comments based on previous mails. > > - I completely agree with MikeM - any kind of complex software could be= =20 > tested with right prepared test cases, specially if they are going to be= =20 > reused in the next release; The trick is balancing the investment of time in different areas, and=20 motivating people to do the things that aren't enjoyable, don't receive=20 much appreciation, etc. Testing is both difficult and time-consuming. It= =20 works best when people are willing to dedicate all or more of their time=20 to the task, since it requires the building of frameworks, the regular=20 application of those tests, etc. People who step forward to work=20 consistently on testing and bug reporting, like Peter Holm, do the project= =20 an invaluable service. And people like Marc Olzheim who take the time to= =20 evaluate the system thoroughly, work through the bug report and fix cycle,= =20 and have the patience to deal with situations where there aren't enough=20 hours in the day to fix a problem make it all worthwhile. It's easy to=20 say that more testing should be done, but testing requires as much=20 expertise in the internals of a piece of software as writing it, and far=20 more time. > - if those problems happened to 5 branch, probably it would happened=20 > again for 6 or 7, so why I have to switch to 6 right now? Is it because= =20 > 5 will never be fixed? Does word "production" mean something to FreeBSD= =20 > project now? As has been discussed extensively in this thread and other threads, the=20 FreeBSD development model typically addresses change at the tree HEAD,=20 where the changes are tested and evaluated, and then they are back-ported.= =20 Some changes are low-risk, and are backported quickly (minor locking=20 fixes, error handling, etc). Others are higher risk, and are backported=20 only when they are felt to have received sufficient testing (driver=20 re-writes, structural changes). Other changes are considered too large to= =20 ever back backported, as you might as well move the users forward as it=20 will be less work and come to much the same thing (major architectural=20 changes, such as SMPng, new hardware platforms, new kernel subsystems).=20 I can't promise that every fix in HEAD (7.x) or the upcoming 6-STABLE=20 branch will make it to 5-STABLE, because many of the changes there won't=20 be appropriate for a backport, or would take so much work to backport that= =20 the time is better spent on other tasks. However, the hope is to bring as= =20 many changes as is sensible back. As we've already discussed, there are several important improvements=20 germinating in 6.x, and many of them will be things that can and will be=20 backported. If you look at the network stack differences between 5.x and= =20 6.x, you'll find very few, because I and others have worked to agressively= =20 merge fixes, usually on a time lag of between one week and one month. I=20 know this is also true in other areas of the system. If you're aware of=20 changes that fix something in 6.x or 7.x that haven't been backported, and= =20 it's been over a month, please contact the developer to ask about a=20 backport. > - I remember some time ago you can stay on current all the time not=20 > worrying that your box is crashed and didn't auto rebooted; Certainly. I also remember long periods of time where you didn't want to= =20 be running current unless you were a VM kernel hacker, such as leading up= =20 to the 3.x release cycle, or just after the introduction of background=20 fsck in 5.x. The 6.x/7.x HEAD branches have been quite on the stable side= =20 compared to the 3.x and 5.x development cycle, and my hope is they will=20 remain that way. > - chip hardware was always in use by FreeBSD, as far as I remember, or=20 > something is changed recently, specially to US, and people buying only=20 > expensive hardware. Probably it is no longer important to support chip=20 > hardware because of more important FreeBSD clients like Yahoo or Apple=20 > use real hardware, not the stupid one like ATA and they have these=20 > "aggressive" project schedules. Believe me I know what "aggressive"=20 > project schedule means, with long, long list of new features. It is=20 > important for such companies like Yahoo only and I know why, because=20 > it's easy to sell useless product with lots of new features than stable= =20 > product with few ones. For regular guy better to have some stable system= =20 > running all the time and doing real work (development or providing some= =20 > service) than rebooting the box, because of some new fancy feature. It's= =20 > getting close to Windows right now. All software development involves the balancing of risks and benefits.=20 That's one of the reasons why the FreeBSD Project offers several=20 development branches, which allow users to balance new features and long=20 running "stale" source code. Notice that we'll be supporting the 4.x=20 branch for several years to come. Of course, if you run 4.x, you won't be= =20 getting many new features, but it's a quite valid option. And likewise,=20 you won't be able to run properly on the newest hardware, because running= =20 on new hardware requires significant architectural changes, such as the=20 introduction of ACPI, rewrites of device driver frameworks, new file=20 systems, and so on. > - IBM, Yahoo, Intel, Apple ..., those guys are smart, having millions of= =20 > unpaid open source developers working on them. The problem is that some= =20 > day those projects will have theirs "aggressive" project schedules, then= =20 > will disappeared or changed to .com. So make sure you are still doing=20 > what you like to do and you are having a fun of it. I think you'll find many FreeBSD developers enjoy working on FreeBSD best= =20 when they receive constructive feedback on the work they do, consisting of= =20 thanks when it works, and helpful bug reports when it doesn't. Some=20 FreeBSD developers live to write new features; others live to get things=20 working "just right", answer questions on mailing lists, or give talks at= =20 conferences. If the balance doesn't seem right, that means there's room=20 for new developers who want to work on the areas that don't get enough=20 attention. :-) Robert N M Watson > > Thanks, > Alexey > >> -----Original Message----- >> From: Robert Watson [mailto:rwatson@FreeBSD.org] >> Sent: Thursday, July 21, 2005 5:21 AM >> To: Marc Olzheim >> Cc: Alexey Yakimovich; freebsd-stable@FreeBSD.org >> Subject: Re: Quality of FreeBSD >> >> >> On Thu, 21 Jul 2005, Marc Olzheim wrote: >> >>> Indeed. That's why my company started taking FreeBSD 5.3 in use for >>> production servers when it was out. Since then numerous >> bugs were fixed, >>> some of which reported by us. Now that we're X bug fixes >> later in time >>> and started to get a good feeling about the number of open >> problems, it >>> is extremely annoying to hear the "This will (probably) not >> be fixed in >>> 5.x" statements. That conflicts with 'gradually get >> resolved'. What do >>> you recommend larger consumers to do ? Keep using FreeBSD 4 >> and start >>> testing FreeBSD 6.x, dropping 5.x all together ? >>> >>> I know FreeBSD 5 was a strange exception in the relase >> scheduling and >>> that a lot has been learned from it for the future and I'm >> certainly not >>> unthankful for all the work that's done, but I'd like a >> clear answer on >>> what to do now in regard to taking FreeBSD 5 into 'real' >> production... >> >> Marc, >> >> I should start out by saying I appreciate your clear and concise bug >> reports, and the list of your company's show-stopper 5.x bugs >> has made the >> rounds among FreeBSD developers. I'm happy that at least one of the >> issues on the list was fixed by me. :-) As you probably saw >> yesterday, >> I've started bugging Poul-Henning to look at the pty problem you're >> experiencing, and will get that on our 6.0 release >> show-stopper list. I >> haven't yet had a chance to reproduce it locally, but it >> sounds like that >> should be straight forward. >> >> FreeBSD 5 has been an exception -- "normally", in as much as major >> releases have a "normal", the set of new features is a lot >> less agressive, >> and it has been our goal with 6.x to restore the expectation >> of a more >> rapid release cycle with a less agressive feature set. This >> should reduce >> the number of problems by virtue of reducing the level of change. It >> should also make it easier for users to pick what version to >> run on, as >> the amount of adaptation they have to do to slide forward a >> version will >> be greatly reduced. I.e., right now it's relatively easy to >> move back and >> forward between 5.x and 6.x. >> >> With respect to 5.x vs 6.x upgrades: I've seen companies take two >> different strategies. Most of them have been at least >> experimenting with >> deploying 5.x, and are very interested in its feature set. >> Support for >> large file systems, 64-bit support on newer AMD and Intel hardware, >> improved PAM support, etc. Some of my customers are specifically >> interested in the support for mandatory access control, but that's >> obviously a less common feature request :-). The biggest determining >> factor for companies today comes from their own product >> schedule, since >> most big consumers of FreeBSD treat it as a component in a >> "product" they >> deliver for others. >> >> For example, my understanding is that Yahoo is now deploying >> 6.0 betas >> across their server environment with great success, but was actually >> unable to seriously deploy 5.x because their goal was to support full >> 32-bit compatibility on 64-bit amd/intel hardware, which has >> only recently >> reached the level of maturity they require. In fact, you'll >> notice if you >> follow FreeBSD commit logs that much of that support has come >> from Yahoo!. >> Since 6.x is maturing in pretty good synch with their >> deployment timeline >> for 5.x, they are actually deploying 6.x. Of course, Yahoo! >> has a team of >> in-house OS developers who adapt FreeBSD for their needs, and >> is quite >> capable of debugging a kernel or two if they run into problems. >> >> The ATA driver issue is a sticky one for many users -- we >> hope to get the >> 6.x ATA code back into 5.x in the next 5.x release. However, >> hard-earned >> experience tells us that ATA driver code is notoriously >> difficult to get >> right across the broad range of available hardware. Soren has been >> lobbying to get it merged to 5.x, but given the level of >> testing performed >> so far, we can't yet justify the merge. My hope is that with >> 6.0 out the >> door and a lot of testing of that code, we can get it merged >> back to 5.x >> before 5.5. Many other fixes have gone into 5.x, correcting >> many of the >> most significant issues. If you compare 5.4 with 5.3, you'll >> find that in >> most cases, it's both faster and more stable. >> >> The tty issue is a sticky one also. The tty code in 6.x has been >> substantially rewritten to better support the SMPng >> environment. Because >> the tty code "plugs in" to a number of device drivers, T1 >> adapter drivers, >> etc, changing the tty interfaces is a fairly big event, and >> will affect >> third party vendors like Cronyx. This code has also not yet >> seen as wide >> deployment as I'd like, so it's also something that really isn't >> appropriate for an MFC immediately. However, once it has >> seen significant >> 6.0 deployment, it may well be. A question then will be whether it's >> better to simply say "you're better off making the jump to >> 6.x, which is >> minor" than backporting, and it's something we can't really >> answer until >> we're comfortable that it's seen sufficient deployment. My >> hope is that >> we can identify a workaround for 5.x that will avoid the code >> upheaval a >> full backport would require. It's not as ideal as having the >> "right" fix, >> but it would stop the panics. I need to ping phk and some of >> the other >> tty-centric folk to look at this some more. >> >> In terms of advice: >> >> If you have a "product" due out more than 3 months from now, >> I think 6.x >> is the obvious way to go: you want to be ahead of the curve >> so that you >> can have the foundation for your product in sync with the FreeBSD >> production release cycle, and avoid jumping major releases >> early in the >> product life cycle. 6.x has significant performance and stability >> improvements -- performance especially in the area of file system >> performance on SMP, preemption, network stack, and memory >> management, and >> stability especially in the area of tty support. By >> "product", I mean a >> range of things: the OS foundation of an embedded product such as a >> firewall or storage appliance, or deployment of an internal >> product, such >> as a virtual server product at an ISP. >> >> On the other hand, if you're deploying today, I think that >> unless you're >> prepared to deal with the 6.0 bug fix cycle (both the BETA/RC >> cycle, and >> the inevitable post-release fixes for a .0 release), 5.4+patches or >> 5-STABLE is the right place to sit. At least two of the >> critical bugs on >> your list were fixed in 5-STABLE after the release of 5.4, so >> for some, >> 5-STABLE is the best place to be. We've opted not to do a >> patch/errata >> update for 5.4 for the socket error you were receiving on the >> basis that >> it doesn't affect a wide audience and doesn't correct a >> "Critical" failure >> -- i.e., a crash or the like, unlike some of the NFS server >> fixes, for >> which we did do an errata fix. >> >> From the perspective of the FreeBSD developers, if you can >> tolerate the >> 6.x release process, we encourage you to jump on that >> bandwagon. It will >> help us release a better 6.0, and that's where the future >> lies. Our goal >> is to make 6.x a pretty seemless upgrade from 5.x, as it has a less >> agressive feature set, and far fewer user-visible changes (i.e., no >> conversion to OpenPAM, devfs,=A0UFS2, large compiler version >> upgrade, ... as >> in 5.x). When I upgraded my personal web/shell server to 6.x >> from 5.x >> last week, I didn't have to change any configuration in /etc >> at all, other >> than a painless pass through mergemaster to merge the _dhcp user and >> group. As always, we look to the freebsd-stable users to >> help us test new >> features ahead of the release. >> >> Thanks, >> >> Robert N M Watson >> > > --0-1481328753-1121972440=:9208--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050721194500.W9208>