From owner-freebsd-ports@freebsd.org Thu Apr 28 06:48:55 2016 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4631EB1F035 for ; Thu, 28 Apr 2016 06:48:55 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3657E1EB1 for ; Thu, 28 Apr 2016 06:48:55 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 31B3BB1F034; Thu, 28 Apr 2016 06:48:55 +0000 (UTC) Delivered-To: ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31524B1F033 for ; Thu, 28 Apr 2016 06:48:55 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (unknown [IPv6:2602:304:b010:ef20::f2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gw.catspoiler.org", Issuer "gw.catspoiler.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0E8891EB0 for ; Thu, 28 Apr 2016 06:48:55 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.15.2/8.15.2) with ESMTP id u3S6mhDX006562; Wed, 27 Apr 2016 23:48:47 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201604280648.u3S6mhDX006562@gw.catspoiler.org> Date: Wed, 27 Apr 2016 23:48:43 -0700 (PDT) From: Don Lewis Subject: Re: Ports tree gone unstable? To: michelle@sorbs.net cc: ports@freebsd.org, vmiller@hostileadmin.com, rkoberman@gmail.com In-Reply-To: <572140C0.1030903@sorbs.net> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Apr 2016 06:48:55 -0000 On 28 Apr, Michelle Sullivan wrote: > Don Lewis wrote: >> I'd thought that poudriere was using the host copy of pkg to do the >> final part of the respository build, but since poudriere doesn't list >> pkg as a dependency, that appears not to be the case. It looks like >> poudriere is running pkg (from the repository being constructed) in the >> jail for that. > > Yeah.. except something around the time went about "upgrading" the OS to > use pkg as well... which screwed the OS... fortunately I caught the > first VM it tried to do it to and was able to limit the damage just to > that VM so the rebuild was minimum. How were you upgrading ports when this happened. I ask because the old pkg_* tools didn't have the equivalent of "pkg upgrade". I'm guessing that you probably used "portupgrade -P" or "portmaster -P" as a wrapper around the pkg_* tools, and I think that requires a copy of the ports tree on the client machines even though binary packages are being used. If that's the situation, I think what probably happened is that when up updated to a version of the ports tree after support for old-style packages was turned off, poudriere rebuilt the repository with new-style packages. Then when you ran portupgrade or portmaster on the client with the same version of the ports tree, the framework then told portmaster/portupgrade to not look for WITH_PKGNG=yes in /etc/make.conf and just to go ahead and use pkg to upgrade the packages. Since the database of installed packages hadn't been upgraded with pkg2ng, chaos ensued. At the time of this transition, I was still using portupgrade to build everything from source. Before I switched to pkgNG, I was having problems with database corruption because the old package tools didn't really handle running out of disk space during an upgrade. I only upgraded packages infrequently because the process was so painful. It would take two or three days to do an upgrade on my desktop and I'd have to monitor it around the clock. If portupgrade ran into an error or stopped to ask a question just after I went to sleep, then I'd lose many hours of potential build time. Because of the infrequent upgrades I would have to deal with all of the intervening special cases in UPDATING that accumulated between upgrades, and the portupgrade -fr and -a options didn't interoperate well, so I ended up having to build some ports multiple times. If things crashed, then I'd have to run portupgrade -rf again, rebuilding a lot of things unnecessarily since there was no way of doing a restart. A classic cause of that would happen if portupgrade decided to rebuild gdm, in which case it would stop gdm before removing the the old version and restart it after installing the new version. Unfortunately, stopping gdm would kill Xorg and thus the terminal window where portupgrade was running. Eventually things got to the point that I could no longer tolerate the extended downtime of my primary desktop machine, so I started building binary packages using portupgrade -p on a faster headless machine. The builds still took a long time, but the final upgrade on my desktop using "pkg upgrade" was *much* faster. Building the packages with portupgrade was still flakey and eventually broke when the ports tree was converted to staging. At that point I bit the bullet and converted to poudriere and life was so much better. Not only did that eliminate a lot of manual intervention to build the packages, but the paralled builds sped things up a lot. Even though poudriere isn't especially efficient about deciding what needs to be rebuilt, the build times for my package set went from several days to under 12 hours, and the latter includes a number of huge ports that I never used to build.