From owner-freebsd-pkg@freebsd.org Fri Apr 28 14:34:59 2017 Return-Path: Delivered-To: freebsd-pkg@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E80DCD536D6 for ; Fri, 28 Apr 2017 14:34:59 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: from h2.pinyon.org (h2.pinyon.org [65.101.20.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BFCD07A0 for ; Fri, 28 Apr 2017 14:34:59 +0000 (UTC) (envelope-from rcarter@pinyon.org) Received: by h2.pinyon.org (Postfix, from userid 58) id 088E21708B; Fri, 28 Apr 2017 07:29:13 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=pinyon.org; s=DKIM; t=1493389753; bh=FGM9QVfYrBnaDdFUhiH9Yev3tSsVHi39w2wjLeuWxPM=; h=Subject:To:References:From:Date:In-Reply-To; b=nT7cWUbwbuhkXDNk1x9Qrr8kEVguBH6uJ5LNssgoHAK7RZ60JVNFcYs07opZ/ZhMH 2t6z8oHEuvLNN32gaGiY3JvCk+gTqrDwBBNfptNqC+VaSzS8RY/Jy0eT/U6yhs1mVQ y6+mtOo0aDQq8yzwzvxKIiinaoNQBzfvfE7MLWEY= X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on h2.n1.pinyon.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from [10.0.10.15] (h1.pinyon.org [65.101.20.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by h2.pinyon.org (Postfix) with ESMTPSA id 20C701706B for ; Fri, 28 Apr 2017 07:29:12 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=pinyon.org; s=DKIM; t=1493389752; bh=FGM9QVfYrBnaDdFUhiH9Yev3tSsVHi39w2wjLeuWxPM=; h=Subject:To:References:From:Date:In-Reply-To; b=fvkfTrNiyLj5p1eSVOm69Im6WlaifXWbxJctjRWQNPdFeMpWAUt9LC692hYXa1d+B ggRL4OVkiKDtYbBYNSJaw+LzD8SO6qBtGs1bcTJ/UBC/FHlGVevD40jzWg82KQ3B/w svl3o49JwTXjt5k+MmQrgMTl50Y+6tCnabFt52UQ= Subject: Re: poudriere suggestions To: freebsd-pkg@freebsd.org References: <20170428144150.4db646cc@rsbsd> From: "Russell L. Carter" Message-ID: Date: Fri, 28 Apr 2017 07:29:11 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.0.1 MIME-Version: 1.0 In-Reply-To: <20170428144150.4db646cc@rsbsd> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-pkg@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Binary package management and package tools discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Apr 2017 14:35:00 -0000 On 04/28/17 04:41, Beeblebrox via freebsd-pkg wrote: > Hello. [...] > * The build que optimization in poudriere might consider port build > length as a parameter, so that all build jails get full usage. I > sometimes see 2 or 3 jails idle because poudriere is waiting on an > ongoing build, whereas the port in question could have been built > much sooner in the process but smaller ports had been queued. There is a tension between the (mostly) single-threaded ports and the monsters like libreoffice, chromium, webkit-*, firefox, and llvm*. I recently built up a (used) dual E5-2650 (32 threads) w/ 128GB of RAM to study the problem because my FX-8300 (8 threads) w/ 32GB of RAM was starting to take regularly more than 8 hrs w/ e.g. PARALLEL_JOBS=4 and MAKE_JOBS_NUMBER=3. Any higher parallelism and the build runs out of swap (standard 2GB) and takes forever. However, that configuration leaves lots and lots of space waiting, similar to what you have observed. With a few overnights in on the new box I am now running PARALLEL_JOBS=6, MAKE_JOBS_NUMBER=7, w/ build times (not repeatable workloads, nature of the beast) around 4 hours. Still, I see lots of idle in the beginning stages (mostly single-threaded) and chromium trudges along with 7 cores for a long time at the end. For throughput to improve, MAKE_JOBS_NUMBER and PARALLEL_JOBS need to be a bit smarter. I think a worthy experiment might be to dynamically generate MAKE_JOBS_NUMBER like so: MAKE_JOBS_NUMBER(current scheduling step) = NCORES - CURRENT_ACTIVE_NJOBS(single threaded) - CURRENT_ACTIVE_NJOBS(multi threaded) * MAKE_JOBS_NUMBER(previous scheduling step) Even this has problems though; chromium could get launched with just a thread or two, and then the waiting would begin. But it's a start. It's a fun, and not easy to solve problem. Optimal dynamic bin packing. I suspect google has spent a lot of time working on it... The HPC crowd have been working on it for 40 years. Russell > Thanks >