From owner-svn-src-all@freebsd.org Fri Sep 18 21:23:36 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0E4D53F04A7; Fri, 18 Sep 2020 21:23:36 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com [IPv6:2607:f8b0:4864:20::835]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BtRdz6WKTz488d; Fri, 18 Sep 2020 21:23:35 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qt1-x835.google.com with SMTP id n18so6392404qtw.0; Fri, 18 Sep 2020 14:23:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=ET8GrRfRE+NFYL1TiQ2Hx55/C+2d2lVJHYXdWpyop1c=; b=N/BMHhIOYJ4itINkz6zWR2kxxGUL/q7AdSEivhSD+JREKij8Zfl/Nu2VhnIeV0L4IW Kp5/LmkY11oD2Fy0qO/dudkL5uoRHnPOhm5b4Wl2yW+l2qPeVteqI++XfaNNKppILXMx +X2rr8SWDvcJFOZtJ+EhT6XxsHlEowJQ1DBfSKsDbABSjOhUr2ywB0qalN1C9Ouq8ChS 0aba2RAr1xq0F+z55ulK5Q+Ew/na14SmbOV4GYJX/aEBKVDoeGKHn/fNYGHeR+aM1jZn iy1hyjVVGpdjNPh8pI+7GkbII1/Y+Afql9zzp8r+3tsQRZ0LGCmynrzbQ+oDfJRot44C juCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=ET8GrRfRE+NFYL1TiQ2Hx55/C+2d2lVJHYXdWpyop1c=; b=bY4xJnWMkZhmL4AuJfC9hgQTzZhhcE3FMGvUbIhwnUR44WKSZL/R94dtXlmD9ss3j4 20SwRHhEHzIxFFS1Pu4zZM+2qVxc5NfWsN23YX9Z9ARypvMT7imZFMdpxaR4jokklvn7 Wz5EhD78XkujhCuhrDnm6qwTUtYQ6IrVGNZACkJvR8EdM8iedAS66imzakTKH/NRpqZW y8XqYc5znxHBCeoULVsGpB631liUy0HDwy4/0dTyVYtCbG/m2Ll9Qhg0uq+aAWkYR4YH fzptLGnPSbS+Lk6Kp11MXigWKwA6eMb99IGWPwRJxF/w7SvrnQBquWobIvxthlAWpnuO Hf5A== X-Gm-Message-State: AOAM532ny2nfI5EAhLN7lWKWY2MXimrM+XosEgkHHM2jKr4KDw4fGccp L4ZRTrCoY1BzNtmah2X1dBUFBS/tr+Ku7A== X-Google-Smtp-Source: ABdhPJzc+ASKxP6Y+mKdXhXiEA9T8i90i7YyXEuEc2c4x6x2rEEjWIYVvhQyjJYHhn8/4vb/hmh+bg== X-Received: by 2002:ac8:6f21:: with SMTP id i1mr33499074qtv.89.1600464214780; Fri, 18 Sep 2020 14:23:34 -0700 (PDT) Received: from raichu (bras-vprn-toroon0560w-lp130-18-184-148-120-111.dsl.bell.ca. [184.148.120.111]) by smtp.gmail.com with ESMTPSA id g18sm2850135qtu.69.2020.09.18.14.23.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Sep 2020 14:23:34 -0700 (PDT) Sender: Mark Johnston Date: Fri, 18 Sep 2020 17:23:32 -0400 From: Mark Johnston To: Ian Lepore Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, kan@freebsd.org Subject: Re: svn commit: r365889 - head/share/mk Message-ID: <20200918212332.GB36442@raichu> References: <202009181903.08IJ3YTM014958@repo.freebsd.org> <311b50c504a5dc1e54a1442aa4f759b14f263b67.camel@freebsd.org> <20200918194400.GA36442@raichu> <3a0ce4f32721320bd04e400cfda05f2383d46220.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3a0ce4f32721320bd04e400cfda05f2383d46220.camel@freebsd.org> X-Rspamd-Queue-Id: 4BtRdz6WKTz488d X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Sep 2020 21:23:36 -0000 On Fri, Sep 18, 2020 at 02:10:13PM -0600, Ian Lepore wrote: > On Fri, 2020-09-18 at 15:44 -0400, Mark Johnston wrote: > > On Fri, Sep 18, 2020 at 01:27:23PM -0600, Ian Lepore wrote: > > > On Fri, 2020-09-18 at 19:03 +0000, Mark Johnston wrote: > > > > Author: markj > > > > Date: Fri Sep 18 19:03:34 2020 > > > > New Revision: 365889 > > > > URL: https://svnweb.freebsd.org/changeset/base/365889 > > > > > > > > Log: > > > > Install library symlinks atomically. > > > > > > > > As we do for shared library binaries, pass -S to install(1) when > > > > installing symlinks. Doing so helps avoid transient failures when > > > > libraries are being reinstalled, which seems to be the root cause > > > > of > > > > spurious libgcc_s.so link failures during CI builds. > > > > > > > > > > Actually, I believe this will just paper over the real problem, which > > > is the fact that we're building and installing the same library > > > multiple times during early build steps. Nobody has yet been able to > > > explain why we do that, and I haven't been able to figure out why by > > > analyzing any differences in the generated files, but I'm not done > > > trying. > > > > Sorry, this deserved more commentary in the commit log. > > > > Indeed, we install libgcc_s.so multiple times during buildworld. As far > > as I can see, it's intentional insofar as libgcc_s.so is listed in > > multiple library targets: it's in both _prereq_libs and _startup_libs in > > Makefile.inc1. In particular, from scanning the log from a recent > > instance of the failure, libgcc_s gets built once but installed multiple > > times. > > > > Why that's the case, or whether it's still necessary, is not clear to > > me. But adding -S seemed like a desirable change in general, so I went > > forward with it. > > Yeah, it has been listed in both prereq and startup list since it was > first added by kan@ in 2007 or so. I wonder if he remembers why (added > to cc list)? Digging a bit deeper, _prereq_libs are built with -DNO_PIC. For lib/libgcc_s this actually does nothing: libgcc_s/Makefile does not define LIB so no static library gets built. The problem seems to be that libgcc_s gets built+installed for _startup_libs and then again as part of _generic_libs. Again, the constituent object files only get built once, but the library binary and symlink get installed multiple times into the worldtmp dir. libc.so is not a symlink and was already getting installed with -S, so despite also getting installed to worldtmp multiple times it does not cause similar build failures. So it seems that: - libgcc_s can be dropped from _prereq_libs. - _generic_libs revisits libraries that were built during earlier stages of the libraries target, i.e., the problem is not specific to libgcc_s.