Date: Fri, 16 Dec 2016 08:24:21 +0100 From: David Demelier <demelier.david@gmail.com> To: Matthew Seaman <matthew@freebsd.org> Cc: freebsd-ports@freebsd.org Subject: Re: (In)Stability of the Quarterly Branch Message-ID: <CAO%2BPfDd0BWuUnY62EeOVzv21pZ3estu=Vgz4chvbqWcaD6adaQ@mail.gmail.com> In-Reply-To: <d806b1f6-9f9a-6546-d44f-5b04f3c422a9@FreeBSD.org> References: <3e7f94efc6428181a289742d7dd627df@acheronmedia.com> <CAO%2BPfDcYDy=w9Xaf02zWiYNO38Yex0ioX6z4a-5KL8k7e9qgQA@mail.gmail.com> <20161215170154.0ca2017914c0bb032516b413@gmail.com> <d806b1f6-9f9a-6546-d44f-5b04f3c422a9@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
2016-12-15 17:25 GMT+01:00 Matthew Seaman <matthew@freebsd.org>: > On 2016/12/15 16:01, Olivier Duchateau wrote: >>> The problem is that there are no tests in FreeBSD ports. All source >>> based systems I've tested: pkgsrc, FreeBSD ports, OpenBSD, Gentoo; >>> FreeBSD is the one that have the most instability. Not to mention >>> committers that commit without testing the port, just look at >>> www/redmine to get your point of view on that issue. > >> Are your serious when you said, there're no tests on FreeBSD ports. I >> can tell you Xfce ports are tested with FreeBSD i386 9.3 and amd64 >> 11.0 machines (on real hardware, no virtualization), and on poudriere >> with Gtk+ 3.20 (port version is not not in ports tree, it's defaut >> toolkits for the next stable release 4.14). >> >> For the LXQt desktop is the same thing (tested with official ports >> tree Qt5 and which one in plasma5 branch (on KDE repository). >> >> I'm also working on the Pantheon desktop (desktop environment of >> Elementary OS, I use Vala 0.30.2 and Vala 0.34.4, in order to test >> stability of applications. >> >> I use also OpenBSD macppc, it's piece of shit. WebKit browers are >> broken, Xfce components crash often, stable branch is outdated, fix >> are not propagated in stable branch. Personally I prefer the FreeBSD >> scheme, because I'm sure it's quite stable. > > Most port committers will run compile tests any time they update a port: > the better ones will test compilation on all supported FreeBSD versions > and all hardware architectures they have access to (ie. generally i386 > and amd64). > I'm not talking about being sure that the port builds, but that the software works. This is a next step that is too often forgotten. For example I remember several years ago having a problem with audio/mumble. The port was building fine, the window opened fine but it was impossible to speak because there was a problem regarding the CELT libraries IIRC. That a port build is nice, that it works is better. And it's the same thing for www/redmine, each time I install it on a fresh machine, `service redmine start` won't start (after configuring of course) because the Gemfile is broken again. These are not the only ones. Glade also suffers a bug that makes it almost unusable. > Additionally the package build cluster will rebuild any modified ports > within a few days for all of the OS versions and architectures the > project tries to provide ports for: that's yet another level of > validating the coding of the port itself. > > However, I believe the OP's point is that *we do not routinely run the > software's own built-in regression tests for the packages we succeed in > building*. This is something that is slowly coming. For instance, you > can run 'make test' for many python, ruby or perl packages and see those > tests being run. TEST_DEPENDS is pretty much standardized as the way to > install dependencies required for testing nowadays. > Yes, I fully understand the requirements of such tests. I just would like that maintainers test the port by building it and *by running* them. This is time consuming for sure, but if the maintainers have no time, then just keep an old but fully working version :-) Regards, -- Demelier David
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAO%2BPfDd0BWuUnY62EeOVzv21pZ3estu=Vgz4chvbqWcaD6adaQ>