From owner-freebsd-toolchain@FreeBSD.ORG Tue Sep 2 23:11:08 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6CB61847 for ; Tue, 2 Sep 2014 23:11:08 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 527F412DD for ; Tue, 2 Sep 2014 23:11:08 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s82NB8Tv017870 for ; Tue, 2 Sep 2014 23:11:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Tue, 02 Sep 2014 23:11:08 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Sep 2014 23:11:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 Garrett Cooper changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |imp@FreeBSD.org, | |rodrigc@FreeBSD.org --- Comment #2 from Garrett Cooper --- This may have already been addressed by imp@ in https://svnweb.freebsd.org/base?view=revision&revision=268376 . I can't verify what version of FreeBSD the jenkins server or kyua servers are running though, so I don't know if this bug happened before or after the enhancement was made to rm to ignore fts_read errors with rm -f. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Tue Sep 2 23:16:19 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCBEFC93 for ; Tue, 2 Sep 2014 23:16:19 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C2AE1131B for ; Tue, 2 Sep 2014 23:16:19 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s82NGJ49046426 for ; Tue, 2 Sep 2014 23:16:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Tue, 02 Sep 2014 23:16:19 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: imp@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Sep 2014 23:16:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 Warner Losh changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Triage |Issue Resolved Resolution|--- |FIXED --- Comment #3 from Warner Losh --- Already fixed, as Garrett states. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Tue Sep 2 23:21:27 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6F50DAF for ; Tue, 2 Sep 2014 23:21:27 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 55757144A for ; Tue, 2 Sep 2014 23:21:27 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s82NLRCj051986 for ; Tue, 2 Sep 2014 23:21:27 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Tue, 02 Sep 2014 23:21:27 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Sep 2014 23:21:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 --- Comment #4 from Garrett Cooper --- The fix hasn't been MFCed to stable/10 or stable/9 though... should it? -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Tue Sep 2 23:30:13 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6F964F5 for ; Tue, 2 Sep 2014 23:30:13 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8CCFA14AD for ; Tue, 2 Sep 2014 23:30:13 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s82NUDop055431 for ; Tue, 2 Sep 2014 23:30:13 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Tue, 02 Sep 2014 23:30:13 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: imp@FreeBSD.org X-Bugzilla-Status: Needs MFC X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Sep 2014 23:30:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 Warner Losh changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Issue Resolved |Needs MFC Resolution|FIXED |--- --- Comment #5 from Warner Losh --- Yes. It does. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Tue Sep 2 23:41:19 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D215D60B for ; Tue, 2 Sep 2014 23:41:19 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B88F61773 for ; Tue, 2 Sep 2014 23:41:19 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s82NfJIQ093416 for ; Tue, 2 Sep 2014 23:41:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Tue, 02 Sep 2014 23:41:19 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: Needs MFC X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: imp@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Sep 2014 23:41:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 Garrett Cooper changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |imp@FreeBSD.org -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Thu Sep 4 13:01:40 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9B9115BE for ; Thu, 4 Sep 2014 13:01:40 +0000 (UTC) Received: from m.saper.info (m.saper.info [IPv6:2a01:4f8:a0:7383::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "m.saper.info", Issuer "Marcin Cieslak 2011" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 360F613B3 for ; Thu, 4 Sep 2014 13:01:36 +0000 (UTC) Received: from localhost (saper@localhost [127.0.0.1]) by m.saper.info (8.14.9/8.14.9) with ESMTP id s84D1Ycr015922 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 4 Sep 2014 13:01:35 GMT (envelope-from saper@saper.info) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=saper.info; s=Sep2014; t=1409835695; bh=0OHW5T+Xq49iKjGKy7B8yRNNW7HdidujwdIQ5QcBHSU=; h=Date:From:To:Subject; b=Qb7BqpEbCgU+ta315KQ+H65Xvr+zCiuwLZN0dncj2yDTtb9zowpc/O4udbCDE46lH NC3RGERVWosASQww+ceLcQ97jl94xg8Urj/boxRV+llmgLJKiGVIJAaSiLxHOLRA/A iJLdhTqIhvTfgmEV2CRjnL0z8Lqrift3F7ijEn+s= Date: Thu, 4 Sep 2014 13:01:34 +0000 (UTC) From: Marcin Cieslak To: freebsd-toolchain@FreeBSD.org Subject: error: empty struct has size 0 in C, size 1 in C++ [-Werror,-Wc++-compat] Message-ID: User-Agent: Alpine 2.11 (BSF 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Sep 2014 13:01:40 -0000 In my clang 3.3 FreeBSD -CURRENT (r259613 as of Dec 2013) I used to have CFLAGS+=-Wc++-compat in my /etc/make.conf probably from some older guide to enable CTF and DTrace. During an upgrade to today's r271089, with this flag (and -Werror) the "make toolchain" command fails on the empty structures, for example struct pthread{} in lib/libc/gen/_pthread_stubs.c Not sure this is a real problem, rather a heads-up to those who did not clean up their make.conf regularly. I don't think I've had this problem with clang 3.3 Old (r271089) clang is: $ clang --version FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 Target: x86_64-unknown-freebsd11.0 Thread model: posix New (being built): $ /usr/obj/usr/src/tmp/usr/bin/clang --version FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 Target: x86_64-unknown-freebsd11.0 Thread model: posix //Marcin From owner-freebsd-toolchain@FreeBSD.ORG Sat Sep 6 02:21:51 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CBC66BD6 for ; Sat, 6 Sep 2014 02:21:51 +0000 (UTC) Received: from mail-ig0-x231.google.com (mail-ig0-x231.google.com [IPv6:2607:f8b0:4001:c05::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9E47E10D7 for ; Sat, 6 Sep 2014 02:21:51 +0000 (UTC) Received: by mail-ig0-f177.google.com with SMTP id h18so94029igc.10 for ; Fri, 05 Sep 2014 19:21:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=24M5sf8ZEDPQnSsL6zy1XNdPpXzC7eiNXg6h4ILHYz4=; b=TFf1z+5woEY8qe9nSB6XSGoYm+cBpMwIRsQ4GA6DZqNAx8O6hVPMBf+jiv/vIVZQu2 Or+7JGrXEVljMDrE/t0gl388IN38V4zP2wrBBLY8Do7N9mCi6RWbDfQr66J5U0+xxUFA 5nLHhmd2szxGbbhyJ+7pDn7dzb+7tB17lzwRkwMThghUi/7DbIKE43C3ZYd+skyyZ8MI XVkobL4ntU9wK4QgYHOpsp73mJNdE/yHAFFgxnm0TAMuTvjVXi5RP5ijSsQOeUUP0KGl Oqo2DRAn7mtYwHKlggIei5VUGe02OZC6otyyiQYAzhViciOLRaxT4wUI5F+iMpkrs8rj Aheg== MIME-Version: 1.0 X-Received: by 10.43.96.65 with SMTP id cf1mr18770968icc.26.1409970110665; Fri, 05 Sep 2014 19:21:50 -0700 (PDT) Received: by 10.50.72.69 with HTTP; Fri, 5 Sep 2014 19:21:50 -0700 (PDT) Date: Fri, 5 Sep 2014 19:21:50 -0700 Message-ID: Subject: Building clang in buildworld as part of the bootstrap process -- is it really necessary? From: Garrett Cooper To: freebsd-toolchain@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: conrad.meyer@emc.com X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Sep 2014 02:21:51 -0000 Hi all, One of the questions that came up from a co-worker is "why do I need to build clang in buildworld if I already installed it from ports"? I could see some valid reasons for doing this (one needs a cross-compiler, one might need specific options that might not be set in the ports version), but for native builds I would tend to agree with this logic. With gcc it was wasteful building the compiler each buildworld, but with clang it seems annoying continuing on this path because the compile takes a long time to complete. Alternatively, would anyone be opposed to adding some logic to automatically bypass the bootstrap compiler, i.e. add some logic to Makefile.inc1 that would skip compiling clang/gcc if and only if the target triplet and version met some required values? Thanks! -Garrett From owner-freebsd-toolchain@FreeBSD.ORG Sat Sep 6 03:16:12 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9B7E5564 for ; Sat, 6 Sep 2014 03:16:12 +0000 (UTC) Received: from mail-ie0-f171.google.com (mail-ie0-f171.google.com [209.85.223.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 60A061798 for ; Sat, 6 Sep 2014 03:16:11 +0000 (UTC) Received: by mail-ie0-f171.google.com with SMTP id rp18so15263118iec.30 for ; Fri, 05 Sep 2014 20:16:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=iTU87tcXZjnBFR20sdCoVkdoTB+hRsYbQv+FkrV9Sb8=; b=QFtZcFKWAeejJDuYP6ku2NuG6vf0KYePJ0TbuNrToYwXcOBF3BBoDTim5wF1rD2nj+ jOjoaqm5ez/WNVcPnB45D7bquVei6muUOl2v9ZPk/TyjjtbENIrfT9p/A2PvnbpKfw8+ 5Zr/CFV+YENGkOiCrvG412wGSZ2Q1+kEVgBv2lI3Wk3Lhqjz2eeIskgI0pvfrKgH7eZb qMYs6zGEyxpZgYYMv142Hg+19zkdzXVrXmIl13K9kNbI13W0vVGWPqRWC6v0tgQVKlGF LXwejYtD5NP0XEbFjLUly+VSd/YaOrY6KyiK0gPrB28Dk9tQKeTSbUJKDUbxeojxIR0i 1bxw== X-Gm-Message-State: ALoCoQljtkh8lQs9Mk/y30BJRJi0pVdvlGk86UHhft4dhEuHlYexdwU0lx6O3kSb3y4mHTUkmV2o X-Received: by 10.50.1.37 with SMTP id 5mr9356624igj.47.1409973370962; Fri, 05 Sep 2014 20:16:10 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id ig9sm5220597igb.13.2014.09.05.20.16.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 05 Sep 2014 20:16:10 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_D60743D6-5EA6-4248-9081-016DC9036B2D"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Building clang in buildworld as part of the bootstrap process -- is it really necessary? From: Warner Losh In-Reply-To: Date: Fri, 5 Sep 2014 21:16:07 -0600 Message-Id: <01C283B7-C9AF-4AE8-A192-FBC7C04D207E@bsdimp.com> References: To: Garrett Cooper X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-toolchain@freebsd.org, conrad.meyer@emc.com X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Sep 2014 03:16:12 -0000 --Apple-Mail=_D60743D6-5EA6-4248-9081-016DC9036B2D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Sep 5, 2014, at 8:21 PM, Garrett Cooper = wrote: > Hi all, > One of the questions that came up from a co-worker is "why do I > need to build clang in buildworld if I already installed it from > ports"? I could see some valid reasons for doing this (one needs a > cross-compiler, one might need specific options that might not be set > in the ports version), but for native builds I would tend to agree > with this logic. With gcc it was wasteful building the compiler each > buildworld, but with clang it seems annoying continuing on this path > because the compile takes a long time to complete. The clang built during buildworld is used to bootstrap. So it is = required sometimes. Usually when there=92s a binary compatibility issue, = which is rare but does happen. It is also installed as cc. The ports clang may or may not build the world. However, if you want to = say =93I know what I=92m doing=94 you can set WITHOUT_CLANG_BOOTSTRAP = and WITHOUT_GCC_BOOTSTRAP to tell the build to do no building of = bootstrap tools and to use the host=92s instead. This usually works, but = may fail from time to time with port-built compilers. If you don=92t want buildworld to build any compiler, you can add = WITHOUT_CLANG=3Dt and WITHOUT_GCC=3Dt. This will create the system = without any compilers. You=92ll need to set CC to /usr/local/bin/clang35 = or whatever as well. There may be other settings you need as well. > Alternatively, would anyone be opposed to adding some logic to > automatically bypass the bootstrap compiler, i.e. add some logic to > Makefile.inc1 that would skip compiling clang/gcc if and only if the > target triplet and version met some required values? There=92s enough violent opposition to this that it will never happen. = buildworld is supposed to always be safe. Don=92t mess with that. It = isn=92t designed to be fast. If you want fast, you can tell it to be = fast with different options, but it will never be fast by default = because we guessed that we think it might be safe. don=92t mess with = buildworld. There might be support for a go-fast catch-all flag that = sets a bunch of other flags to make things go fast, but once you get = down that rabbit hole you=92ll find one man=92s optimization is another = woman=92s intolerable omission. tl;dr: NO. Warner --Apple-Mail=_D60743D6-5EA6-4248-9081-016DC9036B2D Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJUCnx3AAoJEGwc0Sh9sBEAz3AP/2voqkTdU21h0vk3HAux0oVZ K9oSDo2UMn5geiREW8zh2NQHscTHdr5VVHIxWe6J2XCM7ZRgzlJlTzr9ZqJc3hAb jHogF43jxcueySdhiE+G9WLtbi9FUnT8cpyBxwNcC9VDfk2V+xaqRLq+OLT4Ar1F Ss7YJfH8FyG6Q8fhjItAOya8F4dV9tBR6iFtZ1tNmOjE5gmVM2RDKIfCcKv2BSig fiPDi1eGZFGuqzh/GDkBZ/zx/VobvJ6Itmz9JE5qx/YSzZ8WAu71AOCxmWe3zr+7 vR5h7FGq5Zhc1sEMBxpcERcO3TdT3GRbojH4MpXtFYSo7VbeD/KkwbHZjcoq2RDW TorDF1dch05MMlmrWVXYyHgQpBuGFufG5Mba0YL9J+gYzyCw5LrgI1+f1VL7Vu+s HV0tn9k9H6u0iomNLChVbz7e4Po5rS6OasBGpvOWW5+UFH2/oJ/q0KosHvTzCY8M Ok63iTdbcQkbEN61oYTzo1WSruz2ZkpyVVf46/zmuATu4ITuBfO3voM14Q5VyI6G 52r5IyUSiOvowOX6L6kjUYYxKtJubq9ocP/pVczj99Cxi5CiqW30Kqe5Ibg8FwxV OFP2/z8jVKCJvD7fD5jcf4asfD96kSkYY23479KNaqItm9u3WS5q/6XmzVvwxwNT jDdIxDK7xqY7Xjk/oLXt =cdt/ -----END PGP SIGNATURE----- --Apple-Mail=_D60743D6-5EA6-4248-9081-016DC9036B2D-- From owner-freebsd-toolchain@FreeBSD.ORG Sat Sep 6 05:47:27 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 507AA417 for ; Sat, 6 Sep 2014 05:47:27 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23C2317A3 for ; Sat, 6 Sep 2014 05:47:27 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id fa1so23663139pad.40 for ; Fri, 05 Sep 2014 22:47:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:mime-version:in-reply-to:content-type :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=knFzYMc2FPE/Gh+Nh982P7cgRd1K42BtdzdOnWopQ5Y=; b=YZccfBlivpApg0ywCaNtXAUJCz+9ZCTJRbc8FjsvwfSWc7RuipYBR8VLWPqavxrIYR A6S4kCVeJs/buR+sxCqGtsvxVSnwqaStvcgxhXFHq5uE7lYYoLHsGdC3OxEeM7mmHZQM 7hpkFORQEdk3lI9DCujMVGxOcBuJ3fkGPhoKHXBye401tQZXI8Dh5atjki1Lj3eno+UP Wt5AOcMIs38tdHhfl/nRPIIy4GoiU+WIokLdK7k/ruXCRBABuA5EVvc0VHPIwFdrL+9B MnOOAbTMXnRTTJNkVxxNCgBpdxoJULo1/Uf6U5ti2F57lK9EWMq+MvAapEz8mZYX1Z9u 3gfQ== X-Received: by 10.70.94.228 with SMTP id df4mr19013393pdb.64.1409982446239; Fri, 05 Sep 2014 22:47:26 -0700 (PDT) Received: from [10.88.239.179] (mobile-166-137-215-074.mycingular.net. [166.137.215.74]) by mx.google.com with ESMTPSA id cm8sm3215668pdb.95.2014.09.05.22.47.25 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 05 Sep 2014 22:47:25 -0700 (PDT) References: <01C283B7-C9AF-4AE8-A192-FBC7C04D207E@bsdimp.com> Mime-Version: 1.0 (1.0) In-Reply-To: <01C283B7-C9AF-4AE8-A192-FBC7C04D207E@bsdimp.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Message-Id: <215CB3BB-BA4B-41D9-9CBF-FC0FDEE9492C@gmail.com> X-Mailer: iPhone Mail (11D257) From: Garrett Cooper Subject: Re: Building clang in buildworld as part of the bootstrap process -- is it really necessary? Date: Fri, 5 Sep 2014 22:47:22 -0700 To: Warner Losh Cc: "freebsd-toolchain@freebsd.org" , "conrad.meyer@emc.com" X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Sep 2014 05:47:27 -0000 > tl;dr: NO. Makes sense. I'll do some poking around and see if things could potentially b= e optimized with the clang build. On beefy machines it's not a big deal, but= as we know on machines without a ton of memory or SSDs, it can become painf= ul, as expected. Thanks :)! -Garrett= From owner-freebsd-toolchain@FreeBSD.ORG Sat Sep 6 11:32:54 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ECE7725F for ; Sat, 6 Sep 2014 11:32:54 +0000 (UTC) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "tensor.andric.com", Issuer "CAcert Class 3 Root" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 73A861AF2 for ; Sat, 6 Sep 2014 11:32:54 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::410d:d7df:2b8e:d4bf] (unknown [IPv6:2001:7b8:3a7:0:410d:d7df:2b8e:d4bf]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id EB252B803; Sat, 6 Sep 2014 13:32:44 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_407B3BD6-F01E-46F4-B9CB-F020C3C5D3E0"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Building clang in buildworld as part of the bootstrap process -- is it really necessary? From: Dimitry Andric In-Reply-To: <01C283B7-C9AF-4AE8-A192-FBC7C04D207E@bsdimp.com> Date: Sat, 6 Sep 2014 13:32:31 +0200 Message-Id: <333A99CA-014B-47F8-A146-A439611A8A80@FreeBSD.org> References: <01C283B7-C9AF-4AE8-A192-FBC7C04D207E@bsdimp.com> To: Warner Losh X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-toolchain@freebsd.org, conrad.meyer@emc.com, Garrett Cooper X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Sep 2014 11:32:55 -0000 --Apple-Mail=_407B3BD6-F01E-46F4-B9CB-F020C3C5D3E0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On 06 Sep 2014, at 05:16, Warner Losh wrote: >=20 > On Sep 5, 2014, at 8:21 PM, Garrett Cooper = wrote: >>=20 >> One of the questions that came up from a co-worker is "why do I >> need to build clang in buildworld if I already installed it from >> ports"? I could see some valid reasons for doing this (one needs a >> cross-compiler, one might need specific options that might not be set >> in the ports version), but for native builds I would tend to agree >> with this logic. For one, the base version of clang has a number of patches which haven't yet been sent upstream (and might never be). I see the port already has a few of them, but certainly not all. That said, the ultimate goal is obviously to be able to build world with a stock version of an external compiler, be it clang or gcc. There is still quite a lot to be done to make that possible. >> With gcc it was wasteful building the compiler each >> buildworld, but with clang it seems annoying continuing on this path >> because the compile takes a long time to complete. >=20 > The clang built during buildworld is used to bootstrap. So it is = required sometimes. Usually when there=92s a binary compatibility issue, = which is rare but does happen. We already do something similar with BOOTSTRAPPING, where we attempt to detect which build tools on the host system are too old, and build them accordingly. I think something like this might also be possible for the toolchain components, for example by using the __FreeBSD_cc_version built-in define (which is also in our gcc, but it doesn't seem to be used very often). Or some other system entirely. > It is also installed as cc. That's actually another copy, the one built during the later stages. It might not even run on the host system. :) > The ports clang may or may not build the world. However, if you want = to say =93I know what I=92m doing=94 you can set WITHOUT_CLANG_BOOTSTRAP = and WITHOUT_GCC_BOOTSTRAP to tell the build to do no building of = bootstrap tools and to use the host=92s instead. This usually works, but = may fail from time to time with port-built compilers. >=20 > If you don=92t want buildworld to build any compiler, you can add = WITHOUT_CLANG=3Dt and WITHOUT_GCC=3Dt. This will create the system = without any compilers. You=92ll need to set CC to /usr/local/bin/clang35 = or whatever as well. There may be other settings you need as well. >=20 >> Alternatively, would anyone be opposed to adding some logic to >> automatically bypass the bootstrap compiler, i.e. add some logic to >> Makefile.inc1 that would skip compiling clang/gcc if and only if the >> target triplet and version met some required values? >=20 > There=92s enough violent opposition to this that it will never happen. = buildworld is supposed to always be safe. Don=92t mess with that. It = isn=92t designed to be fast. Yup, though in most cases, it should be sufficient to do an incremental buildworld instead of a "full" one. This would also prevent rebuilding any part of llvm and/or clang, as long as no changes occur in them. For example, the only recent change to clang was in one of the ARM target's .td files (to fix a problem with movw being sometimes emitted on ARMv6). In this case, only a bunch of .inc files will be regenerated and some of the files under lib/clang/libllvmarm* will be rebuilt, but certainly not the entire llvm and clang codebase. I would really like for incremental buildworld to become more robust. :) -Dimitry --Apple-Mail=_407B3BD6-F01E-46F4-B9CB-F020C3C5D3E0 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlQK8NMACgkQsF6jCi4glqP0ywCeJw9fy+0vzz8dg8CeMFQdHBln MogAn0edRn9qB0FDJdWH1pwFwEkx42/T =QSaW -----END PGP SIGNATURE----- --Apple-Mail=_407B3BD6-F01E-46F4-B9CB-F020C3C5D3E0-- From owner-freebsd-toolchain@FreeBSD.ORG Sat Sep 6 11:34:47 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D9CE3C9 for ; Sat, 6 Sep 2014 11:34:47 +0000 (UTC) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cloud.theravensnest.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CED961B13 for ; Sat, 6 Sep 2014 11:34:46 +0000 (UTC) Received: from [192.168.0.7] (cpc14-cmbg15-2-0-cust307.5-4.cable.virginm.net [82.26.1.52]) (authenticated bits=0) by theravensnest.org (8.14.9/8.14.9) with ESMTP id s86BSJko085057 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sat, 6 Sep 2014 11:28:31 GMT (envelope-from theraven@FreeBSD.org) X-Authentication-Warning: theravensnest.org: Host cpc14-cmbg15-2-0-cust307.5-4.cable.virginm.net [82.26.1.52] claimed to be [192.168.0.7] Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Building clang in buildworld as part of the bootstrap process -- is it really necessary? From: David Chisnall In-Reply-To: <215CB3BB-BA4B-41D9-9CBF-FC0FDEE9492C@gmail.com> Date: Sat, 6 Sep 2014 12:28:14 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <4AF47B56-109E-46A3-8EC4-75B95E063FCF@FreeBSD.org> References: <01C283B7-C9AF-4AE8-A192-FBC7C04D207E@bsdimp.com> <215CB3BB-BA4B-41D9-9CBF-FC0FDEE9492C@gmail.com> To: Garrett Cooper X-Mailer: Apple Mail (2.1878.6) Cc: "freebsd-toolchain@freebsd.org" , "conrad.meyer@emc.com" X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Sep 2014 11:34:47 -0000 On 6 Sep 2014, at 06:47, Garrett Cooper wrote: > Makes sense. I'll do some poking around and see if things could = potentially be optimized with the clang build. On beefy machines it's = not a big deal, but as we know on machines without a ton of memory or = SSDs, it can become painful, as expected. The build system for clang-in-base has improved a bit so, as you say, it = is now reasonably fast on beefy machines (release clang build with the = upstream build system takes 2-3 minutes on a fast machine, about 10 on = my laptop, the one in buildworld isn't quite as good at extracting = parallelism). On slow machines, it can be quite painful. The correct solution to this problem is likely to be to start creating = bootstrap-toolchain packages. This is also likely to be necessary for = architectures like MIPS and PowerPC before 11 anyway, because the host = compiler doesn't have the C++11 support required for bootstrapping a = newer LLVM and clang. We can work around that quite easily if we have = package for the bootstrap toolchain (possibly cross-compiled from an x86 = machine). David From owner-freebsd-toolchain@FreeBSD.ORG Sat Sep 6 14:41:35 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 838676CA for ; Sat, 6 Sep 2014 14:41:35 +0000 (UTC) Received: from mail-ig0-f181.google.com (mail-ig0-f181.google.com [209.85.213.181]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44CA91E67 for ; Sat, 6 Sep 2014 14:41:34 +0000 (UTC) Received: by mail-ig0-f181.google.com with SMTP id h15so767116igd.2 for ; Sat, 06 Sep 2014 07:41:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=XAV/ZpANo1e4KWDQdnuIN9jX06U+yT+ZPjW0L1BTCsk=; b=U6MJkFN/v2hPG7XyVYy8OlpdOeo+54LuvkJHuW0GipIhmPFpczdwdNFGRI0XXaCoMT o0AP3MtOioaK7oz9xHLWr9NWjR3MjlfwsaL3JYadJ5AjC1Gtv6GtXXpulwKBZKrJyR4h 7CnmEHAqk1bMN/RRITqWRV0vYr0zetRdyHvLLpPx+nWKX0ePbd2ErR/Q6a/1OJPtua6C /nHYT0a2F+jUhKwvFU6e/DQjs3I8ehbaL954ETHLFo4fMJXD67dAsUyVmfwrXJk9dLdW biDchgcYcae+DWw6dvcW+CdhH/niSrVuXPMzpa/X6+RhQe5n2fCQDwZDOFo2IVcrWrl/ r7Ag== X-Gm-Message-State: ALoCoQnv+PyFAXOtuKdiAaqQ1MnS5R42Cq2vB0DFcGJvChx4K28Kw7ASyDUeomtq3t2zXgYeEf6j X-Received: by 10.42.114.130 with SMTP id g2mr21612065icq.46.1410014487997; Sat, 06 Sep 2014 07:41:27 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id z2sm6233887igl.16.2014.09.06.07.41.26 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 06 Sep 2014 07:41:27 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_F19F6F4A-91D2-445C-A458-066C5594ECA2"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Building clang in buildworld as part of the bootstrap process -- is it really necessary? From: Warner Losh In-Reply-To: <4AF47B56-109E-46A3-8EC4-75B95E063FCF@FreeBSD.org> Date: Sat, 6 Sep 2014 08:41:23 -0600 Message-Id: <95AE2942-2EB8-4431-8B87-2304610DB262@bsdimp.com> References: <01C283B7-C9AF-4AE8-A192-FBC7C04D207E@bsdimp.com> <215CB3BB-BA4B-41D9-9CBF-FC0FDEE9492C@gmail.com> <4AF47B56-109E-46A3-8EC4-75B95E063FCF@FreeBSD.org> To: David Chisnall X-Mailer: Apple Mail (2.1878.6) Cc: "freebsd-toolchain@freebsd.org" , "conrad.meyer@emc.com" , Garrett Cooper X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Sep 2014 14:41:35 -0000 --Apple-Mail=_F19F6F4A-91D2-445C-A458-066C5594ECA2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Sep 6, 2014, at 5:28 AM, David Chisnall wrote: > On 6 Sep 2014, at 06:47, Garrett Cooper wrote: >=20 >> Makes sense. I'll do some poking around and see if things could = potentially be optimized with the clang build. On beefy machines it's = not a big deal, but as we know on machines without a ton of memory or = SSDs, it can become painful, as expected. >=20 > The build system for clang-in-base has improved a bit so, as you say, = it is now reasonably fast on beefy machines (release clang build with = the upstream build system takes 2-3 minutes on a fast machine, about 10 = on my laptop, the one in buildworld isn't quite as good at extracting = parallelism). On slow machines, it can be quite painful. Yes. Hence the ask for a faster way to rebuild the system. > The correct solution to this problem is likely to be to start creating = bootstrap-toolchain packages. This is also likely to be necessary for = architectures like MIPS and PowerPC before 11 anyway, because the host = compiler doesn't have the C++11 support required for bootstrapping a = newer LLVM and clang. We can work around that quite easily if we have = package for the bootstrap toolchain (possibly cross-compiled from an x86 = machine). I=92m not sure which problem this solves. If you are cross building, then you already are likely running on amd64 = which has a good compiler. If you are doing a native build, then = everybody but mips and armeb already have clang support in the base. For = those what you do with clang is irrelevant, so long as gcc is around. When gcc is kicked out of the base, we=92ll need to have an external = toolchain story for those platforms anyway. That story might include = generation of binary packages, but it will also need to include building = of those packages easily. Likely without buildworld, or a modified = variant that groks how to bootstrap a sysroot. Cross building for ports is getting close, we may be able to use it to = generate packages for nonx86 faster than native (recent successful full = builds are taking only 5x as long). I guess I=92m saying that while packages may be necessary, they aren=92t = sufficient unless they are easily reproduced. All this also needs to be well documented, or we=92re going to go off = the rails trying to support it. Warner --Apple-Mail=_F19F6F4A-91D2-445C-A458-066C5594ECA2 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJUCx0TAAoJEGwc0Sh9sBEAcPUP/0egJ1JBuCvxj95YlIalVnSk gvI/LXGyyFxIIaOpG52vyVZV8jMDaPrivYO0riff6vFM0lqmMhKuQvvKLEczNy6T w+riZ4cIBcDV7irS2plq5JNgoekKA3W/9r3VVEQhhcEc/yNdzFIJvPZcvRjA+h40 tUeY+X0tM0019x1FZbqqfYWx0VGKn9cpMdzxmvQX0iogAsTlO+LzCRNxZjqf61Ky l3/Bn9uMVQ1zg+B6h0bC/hJ/E7WG7AtLlO0tebhx+BG1P8tersvRkdFP4/Tc9GGY 1aiOSrxcczxR3ahxrNo2V0FtqlvpKJDi3VJG9nmFA/IWqbls3sEkVhmeVC8GIx10 kw60VjjZg5ng8SdZShKlte9x1Q0CU8PjCMkJFIaj2No1O1GEhwNE6aYud91asNPb RlOurfR/nPB4naht9QZ5FbOR+e8z1Ulpqc5uQo2LqXwxzIoZyM+naNSie6kmjNqt xv7GhOdV6K70Aqd6azi7qsrZMK8Fy22MJuF2d+FDwf5oQi+uIzJKZlHff5NPlWTX OyUAbL1qFOw8FPh67Ps8tQXJNVHRVWlTyyCsnV62tZKnubFxK+RBnGp/uV4T6dt2 RTZGXSQDY99ht8kCfvqT/vinSyai2Yq2yhl/zLXy0KNDzG5SIuIaQLGxU4Z1DI6b KUiNlA/3B9gP3jech1YE =bwVQ -----END PGP SIGNATURE----- --Apple-Mail=_F19F6F4A-91D2-445C-A458-066C5594ECA2-- From owner-freebsd-toolchain@FreeBSD.ORG Sat Sep 6 14:46:18 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0B74570E for ; Sat, 6 Sep 2014 14:46:18 +0000 (UTC) Received: from mail-ig0-f180.google.com (mail-ig0-f180.google.com [209.85.213.180]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C0FD01E7B for ; Sat, 6 Sep 2014 14:46:17 +0000 (UTC) Received: by mail-ig0-f180.google.com with SMTP id hn18so764567igb.13 for ; Sat, 06 Sep 2014 07:46:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=92BszKoF8uipq1K/ZJmK0RjW4hTFrO9VdJz1NqQExo8=; b=T5H2VtWMjkkQEHJ+Q9GJP3Z8MVUIb2Z4X0jGbwEiqXXy6ohlc1liWems96qnuJesLH l8cWSwqg/yuw3ayK1w/tB0AJgqUl7lRt/qB+UMHJavrKOJvhqipPE9/+1gTYPyxnZi33 Jim+XOwMk1O7BsMJrYxDaWvA5y74vKhHmdj+0YxLsinir/m/MPWYtPFqHzu2I2/YMMep qOYKsxVZSOdR1I0BZDKZRORLq65UQH3Qg+xarOSf1rxXpwQ/U7ue3tn2LLQByN03dOuJ Dr09ctjpAI6M7hi3b8Y1vZgcMbKk9c5tna+No9qMd+4QSkrX3Pt+ei4Hjz0KwU5+1nbv NmfQ== X-Gm-Message-State: ALoCoQndsUq5Lu+vSSuNG0BNCNlrbIeZDssoaSl+ud+NVxqOgbO+Kdndjv21+IXuxMkh46ii8pyk X-Received: by 10.51.17.66 with SMTP id gc2mr12507484igd.40.1410014771426; Sat, 06 Sep 2014 07:46:11 -0700 (PDT) Received: from bsdimp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id m4sm6234997igr.20.2014.09.06.07.46.10 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 06 Sep 2014 07:46:10 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_9BE5E8FF-B8DF-4BEE-972E-F6A58703A481"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: Building clang in buildworld as part of the bootstrap process -- is it really necessary? From: Warner Losh In-Reply-To: <333A99CA-014B-47F8-A146-A439611A8A80@FreeBSD.org> Date: Sat, 6 Sep 2014 08:46:07 -0600 Message-Id: <91F5D06E-CF51-4659-A52F-4512061BF6F0@bsdimp.com> References: <01C283B7-C9AF-4AE8-A192-FBC7C04D207E@bsdimp.com> <333A99CA-014B-47F8-A146-A439611A8A80@FreeBSD.org> To: Dimitry Andric X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-toolchain@freebsd.org, conrad.meyer@emc.com, Garrett Cooper X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Sep 2014 14:46:18 -0000 --Apple-Mail=_9BE5E8FF-B8DF-4BEE-972E-F6A58703A481 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Sep 6, 2014, at 5:32 AM, Dimitry Andric wrote: > On 06 Sep 2014, at 05:16, Warner Losh wrote: >>=20 >> On Sep 5, 2014, at 8:21 PM, Garrett Cooper = wrote: >>>=20 >>> One of the questions that came up from a co-worker is "why do I >>> need to build clang in buildworld if I already installed it from >>> ports"? I could see some valid reasons for doing this (one needs a >>> cross-compiler, one might need specific options that might not be = set >>> in the ports version), but for native builds I would tend to agree >>> with this logic. >=20 > For one, the base version of clang has a number of patches which = haven't > yet been sent upstream (and might never be). I see the port already = has > a few of them, but certainly not all. >=20 > That said, the ultimate goal is obviously to be able to build world = with > a stock version of an external compiler, be it clang or gcc. There is > still quite a lot to be done to make that possible. Yes. Some use cases work, others don=92t. >=20 >>> With gcc it was wasteful building the compiler each >>> buildworld, but with clang it seems annoying continuing on this path >>> because the compile takes a long time to complete. >>=20 >> The clang built during buildworld is used to bootstrap. So it is = required sometimes. Usually when there=92s a binary compatibility issue, = which is rare but does happen. >=20 > We already do something similar with BOOTSTRAPPING, where we attempt = to > detect which build tools on the host system are too old, and build = them > accordingly. I think something like this might also be possible for = the > toolchain components, for example by using the __FreeBSD_cc_version > built-in define (which is also in our gcc, but it doesn't seem to be > used very often). Or some other system entirely. We=92ve avoided playing the =93optimize the build=94 game for = buildworld. while we do make some exceptions for minor, ancillary tools, = we always assume a full build will always work, and that=92s the only = way the project supports people. >> It is also installed as cc. >=20 > That's actually another copy, the one built during the later stages. = It > might not even run on the host system. :) True. My point in mentioning this is that you don=92t get cc unless you = build clang or gcc in base. >=20 >> The ports clang may or may not build the world. However, if you want = to say =93I know what I=92m doing=94 you can set WITHOUT_CLANG_BOOTSTRAP = and WITHOUT_GCC_BOOTSTRAP to tell the build to do no building of = bootstrap tools and to use the host=92s instead. This usually works, but = may fail from time to time with port-built compilers. >>=20 >> If you don=92t want buildworld to build any compiler, you can add = WITHOUT_CLANG=3Dt and WITHOUT_GCC=3Dt. This will create the system = without any compilers. You=92ll need to set CC to /usr/local/bin/clang35 = or whatever as well. There may be other settings you need as well. >>=20 >>> Alternatively, would anyone be opposed to adding some logic to >>> automatically bypass the bootstrap compiler, i.e. add some logic to >>> Makefile.inc1 that would skip compiling clang/gcc if and only if the >>> target triplet and version met some required values? >>=20 >> There=92s enough violent opposition to this that it will never = happen. buildworld is supposed to always be safe. Don=92t mess with = that. It isn=92t designed to be fast. >=20 > Yup, though in most cases, it should be sufficient to do an = incremental > buildworld instead of a "full" one. This would also prevent = rebuilding > any part of llvm and/or clang, as long as no changes occur in them. buildwolrd -DNO_CLEAN already does that... > For example, the only recent change to clang was in one of the ARM > target's .td files (to fix a problem with movw being sometimes emitted > on ARMv6). In this case, only a bunch of .inc files will be = regenerated > and some of the files under lib/clang/libllvmarm* will be rebuilt, but > certainly not the entire llvm and clang codebase. >=20 > I would really like for incremental buildworld to become more robust. = :) I would too. However, every time clang gets upgraded, it clang seems to = break the no_clean case :( There are also other offenders, but we=92ve = not had smooth sailing in the upgrade with NO_CLEAN stuff for a long = time, hence my caution. jbuild shoed a lot of promise. We need to get that done if we want to = get away from all this crazy. Warner --Apple-Mail=_9BE5E8FF-B8DF-4BEE-972E-F6A58703A481 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJUCx4vAAoJEGwc0Sh9sBEAoYYQAIDRbxO4MnU/nYsXA9IOTs8K 3LJUfOlOJjId9kw/Py+IHYhOac7WyXbzGlVG4TqFoWWwZJBY8c9YfZbO32+S0pTZ /12Fi4xduYFxsDMzbO70RVdmePZY2GJS8DC9rQm2018a4oUHkY8H4gyD5VuqOuV5 soAJqRv0D5k14nnjo2Ff/NuxiIvwQVKGJV8gSBbAKK9rLSBkFB6BAU9CNQ9c1Z86 2rG0gz4irBJyiFDHGvdY8zoMXGP5OUMN1RRMYT2Q6HmU4QRQhu2wwGG14zrbWxBr gNK4BVaPuuYTzrlqSPHW8BD6CZpOCLNHY39A+cz5DaxB2c+iTkdcrPBtaQz6qxQo zqWxwLnXlad3CZn99gjlakWB7hPfM5w5ZDyQjieD8K1NAhYVvXBkuqR9hx1285C5 YzJgtMiboUkxwUbDzgcDs9RJyLfze8H7es0uBAgh58Zm/R+ple3tKIUDEH3vN56Q w0X/bC73MLpW7Xss0a87YnKJhCH2OK8SSrPtsekDY9Qfigk37/dRuWiMGh2YJcRW z7x5w5JjQM5raxyadtEcvPEw4JazQjVjvfhniGLK3MzsKl2cn2chtdu2GC4fJdps nCmIcGMtRpgpMfjByinmpxjgeTHeZ6LJth9DorWZAq5W9XrOn//1/AZ19nP20NJj LE0nbsbmd7+NQBcyHTEw =D9eb -----END PGP SIGNATURE----- --Apple-Mail=_9BE5E8FF-B8DF-4BEE-972E-F6A58703A481--