From owner-freebsd-ports@freebsd.org Thu Dec 7 11:00:10 2017 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CB55E8321C for ; Thu, 7 Dec 2017 11:00:10 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from mail.rlwinm.de (mail.rlwinm.de [138.201.35.217]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 53894813D7 for ; Thu, 7 Dec 2017 11:00:09 +0000 (UTC) (envelope-from crest@rlwinm.de) Received: from crest.bultmann.eu (unknown [IPv6:2a00:c380:c0d5:1:1da3:2972:cee:bda6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.rlwinm.de (Postfix) with ESMTPSA id 3842E4C3A for ; Thu, 7 Dec 2017 11:00:02 +0000 (UTC) Subject: Re: Flavors *COMPLETELY* break the port system (synth and poudriere are useless) To: freebsd-ports@freebsd.org References: <865f71f7-5d2d-90fd-8b41-c00d2317d083@rlwinm.de> From: Jan Bramkamp Message-ID: <04c3f524-b594-078b-f28c-597c4da595e8@rlwinm.de> Date: Thu, 7 Dec 2017 12:00:01 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Dec 2017 11:00:10 -0000 On 06.12.17 23:23, Dave Horsfall wrote: > On Wed, 6 Dec 2017, Jan Bramkamp wrote: > >> Synth and poudriere are parallel build tools and as such are very >> taxing on the system. I suspect your system is unstable under such >> load, because of a configuration error or unreliable hardware. One >> such configuration error that bit me is tmpfs mounted without size >> limitation. Without size limits it can exhaust RAM + swap and crash >> the system. Limit the sum of all your tmpfs mounts to significantly >> less than RAM + swap. > > What happens then?  Does the build process merely crash instead?  I ask > because my minimal system (all I can afford on my "income") has 512MB > memory (all it will take) ad 1GB swap; building Ruby etc kills it, so I > use packages in that case i.e. no customisation if I wanted it. In that case I wouldn't use tmpfs at all. One conceptual difference between the old portmaster/portupgrade tools and the newer tools like poudriere and synth is that the old tools try to minimize rebuilds by modifying the live system. This can break the system if the upgrade fails and will leave it inconsistent while the upgrade runs. Also some ports misbehave and interact with installed software they didn't list as dependency if it is installed. For these reasons the new tools create jail/chroot environments to build ports in a clean environment. Poudriere is designed around ZFS features (snapshots and cloning). Those can be emulated with UFS and overlay file systems, but Poudriere works best on a big ZFS based system with enough RAM to keep the build dirs in tmpfs (e.g. 8GB RAM per parallel builder and one builder per CPU core/thread). You can configure poudriere to use a single builder and UFS, but it will be a slow process. Synth reduces that overhead somewhat by reusing the host system. Its focus is more on keeping a single system up to date instead of compiling sets of ports to custom repos for other systems and the curses UI is a nice touch. Also synth can try to avoid building ports by prefetching packages from an upstream repo. To be honest few FreeBSD devs still care about self hosting FreeBSD on such tiny systems (0.5GB RAM, 1-2 cores). There are devs interested on optimizing FreeBSD for small embedded systems but you aren't expected to rebuild FreeBSD from source on a wireless access point or dedicated firewall appliance. If you have to compile and value your time get adequate hardware. Old 2U dual sockets servers may burn a lot of power but are quite cheap and you don't have to run a compile server 24/7.