From owner-freebsd-ports@freebsd.org Fri Dec 16 07:24:23 2016 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B974BC8250B for ; Fri, 16 Dec 2016 07:24:23 +0000 (UTC) (envelope-from demelier.david@gmail.com) Received: from mail-ua0-x236.google.com (mail-ua0-x236.google.com [IPv6:2607:f8b0:400c:c08::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7407C7A3; Fri, 16 Dec 2016 07:24:23 +0000 (UTC) (envelope-from demelier.david@gmail.com) Received: by mail-ua0-x236.google.com with SMTP id 51so30493135uai.1; Thu, 15 Dec 2016 23:24:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=fzCv1Mtx9hg4g2GHMwRYpEFNtDZ/94SrjV9vj9et2NI=; b=OFsBlcrj6l8aro5fzUmBwKpQdSGf1+HMUPbfKe1xZxy+aKVDxRNVHnu8Q1Uf/Qg3Z1 eVmN47sndjaINVpdHQZA1FSXaHj+HDRUPzyZJ+pwuBMXb1xYHe6vCgGZt/MhVKM1YDR6 sEfjsAqLZR5ZmpVyHeK2fgr0PTEHNXmKcM4MSQAZTZBkY3iaLH/CAmWnL4fBcz3XlCMv P6Ud9fD/ZitAS5/tIvEjiryf+aie1TGoNNSqCiqNXPzWI7oyrmgUwz3nqZypHeWXwsSb 9IkKAH4heuhC0AtcJr77BzzhaeURFHoKqXxsy0YNQYdnXTch/rgTrlZpcHaLLADreLBO 2J6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=fzCv1Mtx9hg4g2GHMwRYpEFNtDZ/94SrjV9vj9et2NI=; b=QxZF+4nuh0KcRjXcYEuv1A1OBSR0iV2IjVrDNG5bv+ckXIKdRNbe+ebJv7Lpji/5Jj drNgog3TC3wtbmI0287iPIlQOtwLNhQqEEgprPG9M3c4OPmW67SnGuXM/T1TK8Rjm5Pn is2kE+BmI17SYIWd51AgnE9+KTCgmbA09KDmdmXaTNfLNJR6X/Wu+qQfD0TnkCWE6bEt voHMr0vlrH8w8gUBsrly/7DNoEsJSMM066dMEgaU/AUrQRMIKnmXf/kbqVdYv/lzozg6 WrvwCc4A6XG5AbBam/jB2dEYCzl4UB126hiQSNs/0pha7EOj+I6Dj6dpBYyZ6Tv2PznQ opLA== X-Gm-Message-State: AKaTC02Zdw4iHKo+fApa0G0GUyEVKinC1SEPf6mojdLiGXa0m+OVFgt/16z4DvDnHgBROymHor195WJtpgwkdw== X-Received: by 10.176.82.48 with SMTP id i45mr1145424uaa.126.1481873062340; Thu, 15 Dec 2016 23:24:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.103.47.83 with HTTP; Thu, 15 Dec 2016 23:24:21 -0800 (PST) In-Reply-To: References: <3e7f94efc6428181a289742d7dd627df@acheronmedia.com> <20161215170154.0ca2017914c0bb032516b413@gmail.com> From: David Demelier Date: Fri, 16 Dec 2016 08:24:21 +0100 Message-ID: Subject: Re: (In)Stability of the Quarterly Branch To: Matthew Seaman Cc: freebsd-ports@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 07:24:23 -0000 2016-12-15 17:25 GMT+01:00 Matthew Seaman : > On 2016/12/15 16:01, Olivier Duchateau wrote: >>> The problem is that there are no tests in FreeBSD ports. All source >>> based systems I've tested: pkgsrc, FreeBSD ports, OpenBSD, Gentoo; >>> FreeBSD is the one that have the most instability. Not to mention >>> committers that commit without testing the port, just look at >>> www/redmine to get your point of view on that issue. > >> Are your serious when you said, there're no tests on FreeBSD ports. I >> can tell you Xfce ports are tested with FreeBSD i386 9.3 and amd64 >> 11.0 machines (on real hardware, no virtualization), and on poudriere >> with Gtk+ 3.20 (port version is not not in ports tree, it's defaut >> toolkits for the next stable release 4.14). >> >> For the LXQt desktop is the same thing (tested with official ports >> tree Qt5 and which one in plasma5 branch (on KDE repository). >> >> I'm also working on the Pantheon desktop (desktop environment of >> Elementary OS, I use Vala 0.30.2 and Vala 0.34.4, in order to test >> stability of applications. >> >> I use also OpenBSD macppc, it's piece of shit. WebKit browers are >> broken, Xfce components crash often, stable branch is outdated, fix >> are not propagated in stable branch. Personally I prefer the FreeBSD >> scheme, because I'm sure it's quite stable. > > Most port committers will run compile tests any time they update a port: > the better ones will test compilation on all supported FreeBSD versions > and all hardware architectures they have access to (ie. generally i386 > and amd64). > I'm not talking about being sure that the port builds, but that the software works. This is a next step that is too often forgotten. For example I remember several years ago having a problem with audio/mumble. The port was building fine, the window opened fine but it was impossible to speak because there was a problem regarding the CELT libraries IIRC. That a port build is nice, that it works is better. And it's the same thing for www/redmine, each time I install it on a fresh machine, `service redmine start` won't start (after configuring of course) because the Gemfile is broken again. These are not the only ones. Glade also suffers a bug that makes it almost unusable. > Additionally the package build cluster will rebuild any modified ports > within a few days for all of the OS versions and architectures the > project tries to provide ports for: that's yet another level of > validating the coding of the port itself. > > However, I believe the OP's point is that *we do not routinely run the > software's own built-in regression tests for the packages we succeed in > building*. This is something that is slowly coming. For instance, you > can run 'make test' for many python, ruby or perl packages and see those > tests being run. TEST_DEPENDS is pretty much standardized as the way to > install dependencies required for testing nowadays. > Yes, I fully understand the requirements of such tests. I just would like that maintainers test the port by building it and *by running* them. This is time consuming for sure, but if the maintainers have no time, then just keep an old but fully working version :-) Regards, -- Demelier David