From owner-freebsd-ports@freebsd.org Wed Apr 27 18:54:40 2016 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E4E3B1F146 for ; Wed, 27 Apr 2016 18:54:40 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4E8A31304 for ; Wed, 27 Apr 2016 18:54:40 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4A207B1F145; Wed, 27 Apr 2016 18:54:40 +0000 (UTC) Delivered-To: ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49C22B1F144 for ; Wed, 27 Apr 2016 18:54:40 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from gw.catspoiler.org (unknown [IPv6:2602:304:b010:ef20::f2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gw.catspoiler.org", Issuer "gw.catspoiler.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A0701303 for ; Wed, 27 Apr 2016 18:54:40 +0000 (UTC) (envelope-from truckman@FreeBSD.org) Received: from FreeBSD.org (mousie.catspoiler.org [192.168.101.2]) by gw.catspoiler.org (8.15.2/8.15.2) with ESMTP id u3RIsTj7004243; Wed, 27 Apr 2016 11:54:33 -0700 (PDT) (envelope-from truckman@FreeBSD.org) Message-Id: <201604271854.u3RIsTj7004243@gw.catspoiler.org> Date: Wed, 27 Apr 2016 11:54:29 -0700 (PDT) From: Don Lewis Subject: Re: Ports tree gone unstable? To: michelle@sorbs.net cc: ports@freebsd.org, vmiller@hostileadmin.com, rkoberman@gmail.com In-Reply-To: <5720FE37.2040309@sorbs.net> MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Apr 2016 18:54:40 -0000 On 27 Apr, Michelle Sullivan wrote: > Don Lewis wrote: >> On 27 Apr, Michelle Sullivan wrote: >>> Don Lewis wrote: >>>> On 27 Apr, Rick Miller wrote: >>>>> On Wed, Apr 27, 2016 at 12:53 PM, Michelle Sullivan >>>>> wrote: >>>>> >>>>>> Kevin Oberman wrote: >>>>>> >>>>>>> On Wed, Apr 27, 2016 at 8:06 AM, Michelle Sullivan >>>>>>> wrote: >>>>>>> >>>>>>> After a portsnap update it seems all my jails won't build the current tree >>>>>>>> returning the following error: >>>>>>>> >>>>>>>> ====>> MOVED: sysutils/puppet renamed to sysutils/puppet38 >>>>>>>> ====>> MOVED: textproc/rubygem-augeas renamed to >>>>>>>> textproc/rubygem-ruby-augeas >>>>>>>> >>>>>>>> ====>> Computing deps for converters/libiconv >>>>>>>> ====>> Computing deps for archivers/unzip >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> >>>>>>>> ====>> Computing deps for converters/p5-Encode >>>>>>>> ====>> Computing deps for converters/p5-Convert-BinHex >>>>>>>> ====>> Computing deps for converters/p5-Encode-Locale >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> ====>> Computing deps for converters/p5-JSON-PP >>>>>>>> ====>> Computing deps for converters/p5-JSON >>>>>>>> ====>> Computing deps for converters/p5-JSON-XS >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> >>>>>>>> ====>> Computing deps for converters/p5-Text-Iconv >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> ====>> Computing deps for databases/ip4r >>>>>>>> ====>> Computing deps for databases/gdbm >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> ====>> Computing deps for databases/p5-Bucardo >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> >>>>>>>> Terminated >>>>>>>> Terminated >>>>>>>> Terminated >>>>>>>> Terminated >>>>>>>> ====>> Cleaning up >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> ====>> Computing deps for databases/p5-DBD-Pg >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/ccache' not found. >>>>>>>> ====>> Computing deps for databases/memcached >>>>>>>> ====>> Error: Invalid port origin '/usr/local/bin/automake-1.15' not >>>>>>>> found. >>>>>>>> ====>> Umounting file systems >>>>>>>> >>>>>>>> Checked updating but don't see anything to suggest that port origins of >>>>>>>> '/usr/local/bin/ccache' are normal.. >>>>> It looks like you're building with Poudriere. I observed similar behavior, >>>>> but not the exact message the other day. I don't remember what origin it >>>>> was complaining about, but located a post (either on a mailing list or >>>>> forums) recommending a `pkg install poudriere`. It did resolve the problem >>>>> in this particular scenario. >>>> This is probably caused by the recent change to globally drop >>>> ${PORTSDIR} from *_DEPENDS. The framework changes initially were done >>>> in bsd.port.mk r399278, but the the actual removal of ${PORTSDIR} didn't >>>> happen until r411970, r412342, ... >>>> >>> Ok that sorta makes a bit more sense... however as this is a jail and >>> the tree is updated why did it break? (I have no local mods in the 'ng' >>> build tree - except an additional (local only) couple of ports which are >>> copied in manually after the portsnap update)... >>> >>> Of course the nice thing is my non-ng tree is still working 100% - but >>> that would be because it didn't get the change... but again that's a >>> completely separate tree and the 2 are not associated with each other in >>> any way... >> I was assuming that this was your non-ng tree where you have local >> framework changes ... > > No, completely separate repo as the new trees are constantly breaking my > tree so I keep them entirely separate. >> >> Did you upgrade ports from something older than r411970 (Sun Mar 27 >> 01:23:25 2016 UTC) > > It would have been around march I did the last build so yes probably > prior to Mar 27. > >> to something more recent? > To the latest. >> If poudriere on your host >> is seriously old, it might not cope with the framework change. It looks >> like you need at least 3.1.9, which was released on Wed Oct 14 21:06:00 >> 2015 UTC. > Yeah, 3.1.x changes the base OS without authority and breaks the entire > build system (can't build anything but the official tree in it) so it's > been deemed a security issue (because it "upgrades" the existing > repositories) and therefore cannot be installed or used on any of the > existing build servers. Sorry, this is all from memory ... my poudriere machine will be offline for several more hours so I can't use it as a reference. Poudriere shouldn't be changing anything in the base OS. It probably creates some temp files under /tmp and puts the package repositories that I builds and the log files under its own directories under /var/tmp. You should be able to build as many different ports trees as you want and they can be downloaded via portsnap or svn, or created by hand. I think I've got 4-6 ports trees that I use with poudriere. The repository that gets updated by a poudriere run is named with a combination of the jail name, the ports tree name, and the set name (-z option). The latter can be use to select an alternate make.conf to set different port options. > Question is why would it be needed? Surely the tree is the tree in the > jail and has nothing to do with the host? or is it not a case of > everything is done in the jail, just the actual building is and > therefore I need new build servers for the NG tree.. Which basically > means I should just decide to fork or erase the whole system because I > can't "NG" right now and I can't actually continue to build in parallel > because of this breakage? Only the actual building is done in jails. When poudriere first starts up, it looks at the list of ports that you want to build and then uses the Makefiles for each of those ports to determine the dependencies of each and the proper build order. It then deletes all of the outdated packages (I'm not sure if they actually get deleted from the repository at this port or just flagged), starts up all of the jails for the build (the ports tree is read-only mounted in each jail and each jail has its own private copy of the base jail prototype), and then starts building. When the building is done, then the package repository is updated. The ports tree used by each build jail is not related to any ports tree on the host (unless you do something like "poudriere ports -c -F -f none -M /usr/ports -p systemports", see ). It's possible to run poudriere without having /usr/ports installed on the host. If you think this is a security risk, you could run poudriere in a VM.