From owner-freebsd-stable@freebsd.org Mon Jul 29 17:44:05 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D9B01B0690 for ; Mon, 29 Jul 2019 17:44:05 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene.sentex.ca (unknown [IPv6:2607:f3e0:0:3::18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 491668C8E0 for ; Mon, 29 Jul 2019 17:44:04 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:346a:6987:6201:ec77] ([IPv6:2607:f3e0:0:4:346a:6987:6201:ec77]) by pyroxene.sentex.ca (8.15.2/8.15.2) with ESMTPS id x6THi1aW060270 (version=TLSv1.2 cipher=AES128-SHA bits=128 verify=NO); Mon, 29 Jul 2019 13:44:02 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: Buildworld times (was Re: svn commit: r350256 - in stable/12: . contrib/compiler-rt/lib/sanitizer_common contrib/libunwind/src contrib/llvm/lib/DebugInfo/DWARF contrib/llvm/lib/MC contrib/llvm/lib/Object contrib/llvm/lib/Ta...) To: Jeremy Chadwick , freebsd-stable@freebsd.org References: <20190727023830.GA53438@koitsu.org> From: mike tancsa Message-ID: <2929237d-220a-2efc-d658-ea0ed867a408@sentex.net> Date: Mon, 29 Jul 2019 13:44:01 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <20190727023830.GA53438@koitsu.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 491668C8E0 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::18 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [0.73 / 15.00]; ARC_NA(0.00)[]; RDNS_NONE(1.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; NEURAL_HAM_LONG(-0.55)[-0.546,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sentex.net]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; LONG_SUBJ(1.70)[227]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: smtp.sentex.ca]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.90)[-0.901,0]; IP_SCORE(-1.72)[ipnet: 2607:f3e0::/32(-4.94), asn: 11647(-3.57), country: CA(-0.09)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; HFILTER_HOSTNAME_UNKNOWN(2.50)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2019 17:44:05 -0000 On 7/26/2019 10:38 PM, Jeremy Chadwick via freebsd-stable wrote: > (Please retain CCs, I am not subscribed to the list) > > Below is hard evidence of 3 things on stable/11 (not 12) after r350259: > > 1. r350259 adds *substantial* time to buildworld. Are you sure this is not the same as the issue in RELENG12 ? ie. the new version of clang is built as part of world since it differs from whats installed.  I had a RELENG11 box sitting around from July 4th (nanobsd2)# time make -j8 buildworld > /var/log/build.out 13421.587u 906.968s 1:01:36.13 387.6%   75193+900k 285907+3240194io 108879pf+0w 0(nanobsd2)# Updated to revision 350414. 0(nanobsd2)# time make -j8 buildworld > /var/log/build.out 18656.513u 1117.246s 1:23:53.24 392.8%  70122+916k 255703+3772612io 104343pf+0w 0(nanobsd2)# Did installworld/kernel. buildworld after that 0(nanobsd2)# time make -j8 buildworld > /var/log/build.out 12998.964u 649.697s 58:48.61 386.7%     75748+907k 358314+3247016io 108534pf+0w 0(nanobsd2)# Which is probably about the same as before the update build time wise > 2. WITHOUT_CLANG_EXTRAS+WITHOUT_CLANG_FULL+WITHOUT_LLDB can help improve > the situation after r350259, but it is still no where near as fast as > pre-r350259. > > 3. Kernel build times are fine; issue is with world. > > TL;DR for lazy folks: > > stable/11 r350330 world + minimal clang = 1:29:34 > stable/11 r350330 world + full clang = 1:46:31 > stable/11 r350252 world + minimal clang = 56:52 > stable/11 r350252 world + full clang = 1:14:30 > > I cannot even begin to tell you how big of an impact this has on my > low-end dual-core VPS box (world takes hours upon hours). > > We've been down this road before, many many times, since the > introduction of clang/LLVM. Here's just a few that went no where. I > couldn't find the more-useful one that had some concrete numbers in it, > dating back to pre-2016 (maybe sometime in 2014 or 2015?): > > https://lists.freebsd.org/pipermail/freebsd-current/2017-January/thread.html#64431 > https://lists.freebsd.org/pipermail/freebsd-stable/2017-January/thread.html#86646 > https://lists.freebsd.org/pipermail/freebsd-questions/2016-November/thread.html#274684 > > Does anyone have a good/recent write-up on how to switch to gcc? :-) > > > System > ====== > * Intel Core 2 Quad Q9550 @ 2.83GHz > * 8GB ECC RAM > * Samsung SSD 840 EVO 250GB filesystem (UFS2 + SU (not SUJ) + TRIM) + 32GB swap > * Running stable/11 r349226 > * Misc notes > - r350330 happened to be what was "master" at the time of my test > - r350252 was the commit on stable/11 immediately before r350259 > - Switching to r350252 accomplished via: cd /usr/src && svnlite up -r350252 > - System uses kern.maxvnodes=856944, last tuned 2018/06/07 > > > Test #1, building r350330 minimal clang > ======================================= > # cat /etc/src.conf > WITHOUT_ATM=true > WITHOUT_BLUETOOTH=true > WITHOUT_DEBUG_FILES=true > WITHOUT_FLOPPY=true > WITHOUT_FREEBSD_UPDATE=true > WITHOUT_IPFILTER=true > WITHOUT_IPX=true > WITHOUT_LIB32=true > WITHOUT_NDIS=true > WITHOUT_NETGRAPH=true > WITHOUT_PPP=true > WITHOUT_SENDMAIL=true > WITHOUT_TESTS=true > WITHOUT_WIRELESS=true > WITH_OPENSSH_NONE_CIPHER=true > WITHOUT_CLANG_EXTRAS=true > WITHOUT_CLANG_FULL=true > WITHOUT_LLDB=true > WITHOUT_LLVM_TARGET_AARCH64=true > WITHOUT_LLVM_TARGET_ARM=true > WITHOUT_LLVM_TARGET_MIPS=true > WITHOUT_LLVM_TARGET_POWERPC=true > WITHOUT_LLVM_TARGET_SPARC=true > WITHOUT_REPRODUCIBLE_BUILD=true > # cat /etc/make.conf > KERNCONF=X7SBA_RELENG_11_amd64 > CPUTYPE?=core2 > SVN_UPDATE=yes > STRIP= > CFLAGS+= -fno-omit-frame-pointer > > Result: > # rm -fr /usr/obj/* > # cd /usr/src > # time make -j4 buildworld > 19906.874u 1280.928s 1:29:33.51 394.3% 57966+778k 23504+14200io 13867pf+0w > # time make -j4 buildkernel > 1592.460u 196.047s 7:36.61 391.6% 48704+614k 6627+18158io 7361pf+0w > > > Test #2, building r350330 full clang > ==================================== > "full clang" means same as Test #1 but with these 3 src.conf lines > commented out, i.e. CLANG_EXTRAS, CLANG_FULL, and LLDB are ENABLED: > > WITHOUT_CLANG_EXTRAS=true > WITHOUT_CLANG_FULL=true > WITHOUT_LLDB=true > > Result: > # rm -fr /usr/obj/* > # cd /usr/src > # time make -j4 buildworld > 23779.674u 1463.156s 1:46:30.75 394.9% 57621+783k 20093+15423io 7283pf+0w > # time make -j4 buildkernel > 1594.079u 194.345s 7:36.48 391.7% 48707+614k 5301+18013io 5342pf+0w > > > Test #3, building r350252 minimal clang > ======================================= > Same configs as Test #1 > > Result: > # rm -fr /usr/obj/* > # cd /usr/src > # time make -j4 buildworld > 12582.693u 882.543s 56:52.35 394.6% 62698+760k 21432+9694io 6923pf+0w > # time make -j4 buildkernel > 1649.559u 184.934s 7:48.01 391.9% 57053+622k 7566+18291io 5402pf+0w > > > Test #4, building r350252 full clang > ==================================== > Same configs as Test #2 > > # rm -fr /usr/obj/* > # cd /usr/src > # time make -j4 buildworld > 16600.975u 1068.754s 1:14:29.53 395.3% 63271+774k 8683+10876io 4707pf+0w > # time make -j4 buildkernel > 1650.654u 183.966s 7:47.47 392.4% 57117+623k 2829+17951io 1926pf+0w >