From owner-freebsd-ports@freebsd.org Sat Mar 4 11:09:09 2017 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C715CF92B6 for ; Sat, 4 Mar 2017 11:09:09 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C78E11E6D for ; Sat, 4 Mar 2017 11:09:08 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from [88.217.109.95] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.86_2) (envelope-from ) id 1ck7YY-0002g6-Rl for freebsd-ports@freebsd.org; Sat, 04 Mar 2017 12:08:58 +0100 Received: from localhost.my.domain (c720-r292778-amd64 [127.0.0.1]) by localhost.unixarea.de (8.15.2/8.14.9) with ESMTPS id v24B8vfa002872 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sat, 4 Mar 2017 12:08:58 +0100 (CET) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.15.2/8.14.9/Submit) id v24B8vhY002871 for freebsd-ports@freebsd.org; Sat, 4 Mar 2017 12:08:57 +0100 (CET) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Sat, 4 Mar 2017 12:08:57 +0100 From: Matthias Apitz To: freebsd-ports@freebsd.org Subject: running poudriere with 8 builders Message-ID: <20170304110857.GA2793@c720-r292778-amd64> Reply-To: Matthias Apitz Mail-Followup-To: Matthias Apitz , freebsd-ports@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Operating-System: FreeBSD 11.0-CURRENT r292778 (amd64) User-Agent: Mutt/1.5.24 (2015-08-30) X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 88.217.109.95 X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Mar 2017 11:09:09 -0000 Hello, I acquired a server which my company sent to dust (called it "decommissioning"), a Dell PowerEdge r210 with 8 CPU 3.3 GHz and 16 GByte RAM. I have had to insert 2 SAS disks, each of 300 GByte. The beast compiled CURRENT's world in an hour, and the kernel in 6 minutes, wow! Now I setup poudriere and a ports tree in it, all from CURRENT too. I started with 8 builders and poudriere needs halv an hour to start/clone the builders: root@jet:/usr/home/guru # poudriere bulk -f /usr/local/etc/poudriere-list -J 8 -j freebsd-r314251 -p ports-20170303 [00:00:00] ====>> Creating the reference jail... done [00:03:46] ====>> Mounting system devices for freebsd-r314251-ports-20170303 [00:03:46] ====>> Mounting ports/packages/distfiles [00:03:46] ====>> Using packages from previously failed build [00:03:46] ====>> Mounting packages from: /usr/local/poudriere/data/packages/freebsd-r314251-ports-20170303 [00:03:46] ====>> Copying /var/db/ports from: /usr/local/etc/poudriere.d/freebsd-r314251-options [00:03:47] ====>> Appending to make.conf: /usr/local/etc/poudriere.d/freebsd-r314251-make.conf /etc/resolv.conf -> /usr/local/poudriere/data/.m/freebsd-r314251-ports-20170303/ref/etc/resolv.conf [00:03:47] ====>> Starting jail freebsd-r314251-ports-20170303 [00:03:48] ====>> Logs: /usr/local/poudriere/data/logs/bulk/freebsd-r314251-ports-20170303/2017-03-04_09h50m18s [00:03:48] ====>> Loading MOVED [00:04:06] ====>> Calculating ports order and dependencies [00:09:17] ====>> pkg package missing, skipping sanity [00:09:17] ====>> Skipping incremental rebuild and repository sanity checks [00:09:17] ====>> Cleaning the build queue [00:09:20] ====>> Recording filesystem state for prepkg... done [00:09:20] ====>> Building 1760 packages using 8 builders [00:09:20] ====>> Starting/Cloning builders [00:33:47] ====>> Hit CTRL+t at any time to see build progress and stats [00:33:47] ====>> [01][00:00:00] Starting build of ports-mgmt/pkg [00:35:01] ====>> [01][00:01:14] Finished build of ports-mgmt/pkg: Success ... During this time (00:09:20 - 00:33:47) the CPUs are idling and top shows 8x 'cpdup' (I do not see any man page for this, what proc is this?). The overall throughput now is around 70 ports per hour. And top shows little CPU usage and a lot of pkg-static processes, perhaps for unpacking dependency packages. Any hints about how to improve the setup? Thanks matthias -- Matthias Apitz, ✉ guru@unixarea.de, ⌂ http://www.unixarea.de/ ☎ +49-176-38902045