Date: Mon, 29 Aug 2011 04:43:18 +0200 From: Michal Varga <varga.michal@gmail.com> To: Doug Barton <dougb@FreeBSD.org> Cc: freebsd-ports@FreeBSD.org Subject: Re: Ports system quality Message-ID: <1314585798.82067.338.camel@xenon> In-Reply-To: <4E5AB672.4020607@FreeBSD.org> References: <4E5A48AC.6050201@eskk.nu> <20058.20743.791783.342355@jerusalem.litteratus.org> <BLU0-SMTP182102B9C96837517ECB6BB93150@phx.gbl> <20110828172651.GB277@magic.hamla.org> <20110828173059.GT17489@deviant.kiev.zoral.com.ua> <20110828181356.GD277@magic.hamla.org> <20110828183300.GX17489@deviant.kiev.zoral.com.ua> <20110828184542.GE277@magic.hamla.org> <20110828152234.54cc9fac@seibercom.net> <20110828193046.GA668@magic.hamla.org> <1314564889.82067.89.camel@xenon> <4E5AB672.4020607@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 2011-08-28 at 14:43 -0700, Doug Barton wrote: > If you're talking about the recent ruby update, an enormous amount of > work went into that prior to the trigger being pulled in an effort to > make it as smooth as possible. It's unfortunate that in spite of that > effort there were still some "issues" that were only discovered after > users rushed to perform the upgrade. In this case backing out the change > was the responsible course of action. I didn't want to mention the Ruby update specifically, because then the debate would turn into this or that specific issue, and honestly, maybe a year ago, I'd still think that it's really just about a specific issue in every single case. But over a course of time, I'm now inclined to see it more as a full picture, and that picture is, sadly, "something just constantly being broken" (yes, in the big picture). No matter how much some individuals invest into FreeBSD ports, the overall loss of quality control is pretty visible, and again - no single individual will fix that all. No two of them, no dozen of them. This is something that has to be fixed on a higher level, and a nice step in that direction would be a complete revamp of (currently like, close to none?) quality control procedures, and then require port maintainers (and commiters) to actually follow them, every time. Even if they are volunteers, and everyone contributes on their own time and resources, sure, but I would, personally, rather have 10000 (or 1000, if there's no other choice) fully working ports again, instead of those 22000 there all the time untested, broken, or missing features or introducing regressions every minor update. > Now, how do we fix this? It has been suggested numerous times that one > solution to this problem would be a "stable" ports tree. The idea being > that after changes have had a chance to shake out for a while in the > head of the ports tree they get merged back to a stable branch. This > needs to happen, yesterday. Stable ports tree *alone* as a solution is a waste of time (yes, I know how ridiculous that statement sounds in this thread, but-). There are only two possible outcomes of this, one worse than another: 1. Nobody will be using the "testing" tree, because it will be constantly superbroken (like, much more than ports are now). In theory, at least for some people, such tree would sound like a great idea to "get your latest Firefox / Xorg / Gimp / whatever you like", but the dependency tree you will need to pull will basically overwrite half of your stable ports tree anyway, so you can as well keep using "testing" on full time. Which in turn means having everything broken all the time. Which in turn means, rarely anybody will be using it outside of port maintainer/commiter circles, which means you won't get any additional testing anyway. And / Or: 2. Stable tree will become terribly outdated, as every just a little bit complicated app comes with 300 dependencies (think of GTK/QT and all the way down from there). This means - do I want, as a maintainer, promote this new (and properly tested) Firefox to the stable tree? Well, but I will need you guys to promote me that latest GTK and right, recent gstreamer, and, uhmmm, some recent smb stuff and just a few more, and oh, they won't work without their latest dependencies too so in turn, we either end up with a ~100 port promote to the stable tree, or we will wait a few months until all the needed components get there on their own. So -stable tree will rot and nobody will want to use it because "man, it has like already three releases old Pidgin, what good is that for?" (which in fact is a security threat by itself, so probably not the very best example, but you get the basic idea). What is needed much more than two different port trees, is a cooldown period combined with *required* testing by port maintainers (and by port commiters), with some actual consequences if they fail to do so. Now this again sounds ridiculous (to some), shooting volunteers against the wall for breaking rules, so that alone is why this will never happen. But the fact is, that in the current state of ports, we advertise something that doesn't really work as a whole. As the current rules go now, port gets a go as long as it compiles, and that's why we have now those beautiful 22000 ports to advertise. Except that there is nothing that requires a port maintainer to know how their port is being used: Again, I'm not going to point fingers, but there was this little dependency port once that after one upgrade totally broke about every single GTK application. How so? Did the maintainer even briefly test it before submitting the new version? As it eventually turned out in a private conversation, he actually tested it pretty thoroughly, with one exception... Port maintainer was a server user and never even ran FreeBSD as a desktop system. And there is this minor issue, that his port is also one of the dependencies that gets pulled into every single GTK application, right? So during his testing, nothing wrong showed up with the new version, as with the limited (non-GTK) scope that he was ever using (and testing) his port for, everything was fine. On the other hand, for desktop users, the new port broke practically everything. Yay. So how would one prevent situations like these without *forcing* volunteers to do job out of the scope they volunteered for? What is the solution for that? Requiring them to test for something they don't even use, or even know? What's the better option then? Having the guy drop maintainership if he is not interested in fully maintaining it as a dependency? Or advertising a working Gnome (KDE, XFCE, whatever) Desktop on FreeBSD, and having down there someone maintaining a low level dependency that can completely break the *desktop OS* every single update, while obviously he is not even testing for it? This is just one of those situations that won't get magically solved by just another ports tree. Obviously, one additional "testing" tree will catch some of the breakages (depending on number of people willing to deal with unstable ports, and in my humble expectations, that would be like: - almost nobody for server configurations (because why) - almost nobody for desktop configurations, because that would mean desktop that's not even starting 90% of the time, so why even bother ), but being the visionary I am, I think it won't change as much as it now shows on a paper. There already are testing repos for various ports (Gnome, Xorg, VirtualBox, etc.), and barely anyone uses them. What is needed more are new, much stricter maintainership and committing policies than there are in effect now, and seriously - who would dare to even suggest such thing and still live through the mass burning and exodus following it? > The other thing that will help between now and then is to manage your > change windows a little more conservatively. Except for security-related > updates there is almost always zero reason to upgrade to new versions of > things immediately after they hit the ports tree. With all due respect > to those involved, one of the reasons the ruby thing was such a mess was > that users jumped in and started upgrading stuff without knowing what > they were doing, or why. Personally as soon as the notice about the > upcoming change went out I put the knob in make.conf to keep my systems > at 1.8 to make sure I wasn't affected. I'd have to strongly disagree with this point of view. Thing to keep in mind is, that at any point in time, there is always *something* that had just hit the tree only moments ago. What if I had those 30 ports to upgrade (or make it 300 if one upgrades once over a month), all of them being cross/dependencies along the path requiring some of each other, and now I have to cherry pick them and blacklist/exclude some based on their arrival time... That's crazy. Many people haven't even heard about half of those dependencies down the tree (and why should they), and how many of them even don't know that Ruby is a programming language (why should they, it's crap anyway) and that 1.9 is a major upgrade. Why would someone upgrading a web browser, music player (or more specifically, *bringing all his applications up to date*) know about some Ruby and what it is and that 1.8 vs 1.9 can also be a major deal-breaker, when for him, it's just some random dependency 30 levels below? Thinking about upgrading ports in this way would require every single port user to become a maintainer of... everything. What you propose is the opposite of cooldown period - you'd require every ports user keeping track on every single port installed and know 'how important' in the overall scheme it is and when it's safe to update it and when you have to schedule a week-long alarm to let it cool down a bit until it's (probably) safe to use it (as some other people got probably burned and hopefully reported it). This is really a bad idea. When ports hit the tree, they should either be stable and ready to use, or not be there at all. m. -- Michal Varga, Stonehenge (Gmail account)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1314585798.82067.338.camel>