From owner-freebsd-pkg@freebsd.org Fri Apr 28 20:10:25 2017 Return-Path: Delivered-To: freebsd-pkg@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A940CD54855 for ; Fri, 28 Apr 2017 20:10:25 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: from h2.pinyon.org (h2.pinyon.org [65.101.20.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 66E9DF1C for ; Fri, 28 Apr 2017 20:10:24 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: by h2.pinyon.org (Postfix, from userid 58) id 7419417945; Fri, 28 Apr 2017 13:10:23 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=pinyon.org; s=DKIM; t=1493410223; bh=Jz8KF7vEU6c0GVtSYfIBZAW9fQ6QRilOp9foOubjV2Y=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=T7AXB7GCgIYcerrE8l9MojU4FqeJXVSAzjPJNMNzlQ45GHMX/D+06FK3SNc2GguCC eJ6EFQM0VVlTJzjEhCKDKVx61KOXIs8uIdvi0mjOX3/sSWyYu4DxPvWaLqgbqElVPN ude9xH3I2/eoiNaNLDjUB5bTTTJaovjQhKL2xy30= X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on h2.n1.pinyon.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=unavailable autolearn_force=no version=3.4.1 Received: from [10.0.10.15] (h1.pinyon.org [65.101.20.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by h2.pinyon.org (Postfix) with ESMTPSA id 2DB2417918; Fri, 28 Apr 2017 13:10:22 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=pinyon.org; s=DKIM; t=1493410222; bh=Jz8KF7vEU6c0GVtSYfIBZAW9fQ6QRilOp9foOubjV2Y=; h=Subject:To:Cc:References:From:Date:In-Reply-To; b=fW8hVFwQ/nllbl7rbw6Em8UUb82zQbSBs6SYxBhbOFwbKTN9h5nRgfwK9fCEu25w5 26ydwkvHRrH7qKLO5vaEYjLbsPJNRlUxNzFPLwPJihFuE2S7ezzOSjNCZ/pfvgLi1w /eo3l890NcooM/45wrwjrkdXm1DOWkFPlEeDR1HE= Subject: Re: poudriere suggestions To: Rainer Duffner Cc: freebsd-pkg@freebsd.org References: <20170428144150.4db646cc@rsbsd> <4E32A700-BD92-4BF1-A0C4-A7269123A827@ultra-secure.de> From: "Russell L. Carter" Message-ID: <71a8cb99-6c1e-d552-43e2-34b2f1dcd3b3@pinyon.org> Date: Fri, 28 Apr 2017 13:10:21 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0.1 MIME-Version: 1.0 In-Reply-To: <4E32A700-BD92-4BF1-A0C4-A7269123A827@ultra-secure.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-pkg@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Binary package management and package tools discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Apr 2017 20:10:25 -0000 On 04/28/17 12:09, Rainer Duffner wrote: > >> Am 28.04.2017 um 16:29 schrieb Russell L. Carter > >: >> >> There is a tension between the (mostly) single-threaded ports and >> the monsters like libreoffice, chromium, webkit-*, firefox, and >> llvm*. > > > So, it’s not possible to speed up the build of libreoffice by > parallelizing it? I must have not been clear. The "monsters" (so to speak), are fully parallelizable, but if you set PARALLEL_JOBS to a high number (e.g. the number of cores/threads) and also set MAKE_JOBS_NUMBER in poudriere.d/make.conf to a high number (e.g. the number of cores/threads), then you might end up with a portion of the build scheduling MAKE_JOBS_NUMBER*PARALLEL_JOBS cores/threads. That will bring many systems to their knees. So for instance, via a lot of observation, I've found that PARALLEL_JOBS=6, MAKE_JOBS_NUMBER=7 works best for my 32 thread Xeons. This has the disadvantage that when running a lot of single threaded jobs through only PARALLEL_JOBS number of threads are active, but I keep the peak load down far enough that the big multithreaded jobs minimize thrashing and maximize efficiency. Similarly, if at the end of the poudriere bulk build there are 2 long running jobs, it does not matter that PARALLEL_JOBS is set higher; you will only get 2*MAKE_JOBS_NUMBER available threads working on the those two jobs (and nothing else will run, everything else is complete). So I see a lot of either 7 or 2*7 load average for up to a couple of hours on my 32 thread box. Suboptimal. I would like to get hard numbers on this but that would be a really big effort given the computational effort in a 4-8 hr poudriere run. What eventually needs to happen is that MAKE_JOBS_NUMBER changes according to the current number of buildable JOBS and the available cores/threads (which is a function of the current working set of jobs at the time of scheduling this job). This will require reservations and priorities and tracking the current thread demand, but I suppose that's a long way off yet. HPC schedulers do this. It turns out this is a supercomputer problem, kinda cool. > I only build a 2000-ish subset of ports - and libreoffice (together with > webkit-gtk2) takes more time than anything else. > Or rather, these take more time than the rest, combined. I've got a similar number subset, but it seems to be chromium that's the real time sink. > > I’ve got a 2x6 core Xeon and 64GB RAM. hth, Russell