From owner-freebsd-toolchain@freebsd.org Wed Sep 4 20:38:18 2019 Return-Path: Delivered-To: freebsd-toolchain@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2BA75CD3EC for ; Wed, 4 Sep 2019 20:38:18 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-24.consmr.mail.ne1.yahoo.com (sonic310-24.consmr.mail.ne1.yahoo.com [66.163.186.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Nwd42wvyz3N61 for ; Wed, 4 Sep 2019 20:38:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: x2YKiNQVM1n06jP4pi82OB8KtL2DfRLLkuIcJDdGqQ3fVYK73k124nyQ9Z4MIUJ IXjTDXNiriOFjgcK9oyu5sTn1kJas3dYcKN_8kQKtx6pp16moaLWuP364WZbiY6FPCKHRMg9KNCC E_pMLWIVx2vxSJBAJ8gyzqFbfOnkXKiaweNsxXz48KIWi_NIrOUPlU3yDZH6mgkp4d8luh_bmxJ6 WUD4EPmAG8l0BPjMtqlEOiqNZca5W3a9hRn_cQBVhDLAGoJBaMKAa9jN5dIz_fqoLEsEwyFuiIJC YDDyFkgtI8zDTtL0dBKjJBFfOsSkOqtI6dYcKPlzTSdOxb7jE0J.QRqoxVn..FUqcWQ5MP72.AoO uWf27Zov.AuVgz7jGnbEhF9UULsTy_ICnVVG_89keXK0V1gm7fumll4OoNtBHroLKAoyFOzQnbcN NmBRjsf4VYaYX8YJvOxa9E.9j8mTUx2EDLbGDhMLuKtAhPA1K.OmTJxXa3UeBYCniCQ_gTQmlQ0U aGyu.3IwmvOVRw.TKfIeBexB0Th5S1pq06Hxcj1nN6WVTHdzhYYVT97MNr_q9s.iWBla7Pt6k3OF 3AiGosgFfKYRKH_UM6LCZkaQrPj.Tc_AQWjySBiCnT0NEgOkuAg4Fz0d9uPqAf4sbqzgv56TQCvx T4zD6ikrVGamN1.39pDmiQKZUjKBE4650YjZ9szUxAqwvMGHCfDwrMmhCE69v9PlShc72m72y_tx RgnrZcN6UClGnGpy2.eHlAAxxVD_h3dPxGOS7XvW9JG00koQ0YdFx.lsAyzjGR7BwqOixdOx6XvM c.kiUGPCFdfSmeB7zp5lcZ7j3uBmpwzRzbkdOfj.A1MpWbxJOWsBb9nHlcAbICeBkrgNKeZEVUzn HL7sWDhsirnCD4o2G9Y5a4uOFd8enHoPRZepeXMiIwN6b7uou0zV.kiiX3z9xIH89pwQbyLxG.8p ygXxbeHPpnGrvTHFHt8QQ3EXiJiPvUhFcgxlLmujv3lQjZJxOR5GejrieEEQV0__WYyPnWMji1um tONMUyXTLlLk4Y2KtOCSTZhb3plE84iL5n8dcLxNh38Hi6zVdTh.5XXumOE5xkd_6W.umMiifSSx U5aFgtAfUvJM84_WHCUUxkUa3Rejra4SSYmZtOFmHiU49exrA_T2P4YYCkRjchg6S3il6mfqZyjV 1YZWW.M.EhNMoyNCfFrksXsSykAzRshl_2m2MbCY4RUKMzmJ6QNgKaxpm5fHytpJ8wPRn5Ybuzkr C3YKAvHeX3s1XTrV3q5y1vEEN1ftPhzUqYZCXpS4PYs7lntJ9d09476Iuq5oXTk6L.rA- Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.ne1.yahoo.com with HTTP; Wed, 4 Sep 2019 20:38:13 +0000 Received: by smtp409.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 18ba08b5993958123b33a92bc684e6f4; Wed, 04 Sep 2019 20:38:10 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: linker not using make.conf From: Mark Millard X-Priority: 3 In-Reply-To: Date: Wed, 4 Sep 2019 13:38:08 -0700 Cc: FreeBSD Toolchain Content-Transfer-Encoding: quoted-printable Message-Id: References: <20190904151842.GA71523@spindle.one-eyed-alien.net> To: Sid X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46Nwd42wvyz3N61 X-Spamd-Bar: / X-Spamd-Result: default: False [-0.61 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; HAS_X_PRIO_THREE(0.00)[3]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[bsdmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.58)[0.578,0]; NEURAL_HAM_LONG(-0.82)[-0.820,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ip: (2.48), ipnet: 66.163.184.0/21(1.30), asn: 36646(1.04), country: US(-0.05)]; NEURAL_SPAM_MEDIUM(0.13)[0.132,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[205.186.163.66.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Sep 2019 20:38:18 -0000 On 2019-Sep-4, at 11:35, Sid wrote: > When the base linker is not available, the link needs to be from = /usr/bin/ld rather than from /usr/local/*/ to /usr/local/bin/ld.lld80 or = variant of that. Also, programs being compiled do look for /usr/bin/ld = or maybe another ld under /usr/local/*. There were about two locations = that could be used for each version of the compiler and its toolchain, = one like you described, because another also had a soft link. >=20 > Mine works, since I have a soft link from /usr/bin/ld to the needed = one in /usr/local/bin/* >=20 > If I remember correctly, installing xtoolchain-llvm didn't do much, = except for give hints on what to put as XLD. =46rom trial an error, I = found that, LD covers everything, including what XLD covers, except when = XLD is used, it overrides LD only for ports. In other words, LD in = make.conf covered building the kernel, base, and ports. When XLD was = set, it overrode LD's settings only for ports. At least the X version = (XCC, XCXX, compared to CC, CXX) of the other make.conf setting. >=20 > The point is, I wonder how much confusion is being caused for = developers, when LD and XLD are not working as they are supposed to. = When one gets fixed, but ends up with the same problem, because the base = linker is used, rather than the one make.conf is intended to make work. = I wonder if this has to do with why some ports require llvm80, and = others llvm60, when the assumption is on the wrong needed update, when = it's not using the linkers from those. I also guess that, this is = causing difficulties for when trying to make clang's utils work for = different architectures. A problem like this doesn't help, and it likely = slows down development, that a new release must be waited for before = significant improvements can be made. The LD setting in make.conf not = working properly is a fundamental problem, that can cause other = problems, and false assumptions. >=20 > It's more difficult to see the problem, if the base ld is available. >=20 >=20 >> Sent: Wednesday, September 04, 2019 at 10:18 AM >> From: >> Cc: freebsd-toolchain@freebsd.org >> Subject: Re: linker not using make.conf >=20 >> The LD variable only effects the very few cases where the linker is = called >> directly. The linker is almost always run via clang. If you install = the >> xtoolchain-llvm80 port it will install a link from >> /usr/local/llvm80/bin/ld.lld to /usr/local/llvm80/bin/ld which I = think will >> be sufficient for your use case. >>=20 >> On Tue, Sep 03, 2019 at 11:04:08PM +0200, Sid wrote: >>> In /etc/make.conf, I have >>> LD=3D /usr/local/bin/ld.lld80 >>>=20 >>> This is not used for ports. It may be used for building the kernel = and world. >>>=20 >>> clang-8: error: unable to execute command: Executable "ld" doesn't = exist! >>> clang-8: error: linker command failed with exit code 1 (use -v to = see invocation) >>> *** Error code 1 >>>=20 >>> XLD=3D /usr/local/bin/ld.lld80 being set as well also provides the = same error. XD sets it for all, but XLD is only applicable if a = different compiler is used for ports than kernel and the base. When LD = is set, XLD only applies when it is set as well, but this suggests that = XLD is not working correctly either. >>>=20 >>> I have to manually link /usr/bin/ld to /usr/local/bin/ld.lld80 for = ports to build correctly. This is with both make, and with portmaster. >>>=20 >>> I built my computer without ld in the base system, and this has = worked well. make.conf should reference the chosen linker without having = to manually link it. Otherwise, LD in make.conf is not working = correctly, and gives the impression that one linker is used, when it's = not. This can cause faulty conclusions and confusion for developers as = well, who think one linker is set, when it's not. May be what brooks was referring to is not familiar. So I show examples for system-clang++, llvm90's clang++90, and g++9. Here is an example of building and linking a program: # c++ -std=3Dc++17 -pedantic -O3 -pthread -c cpp_thousandslocale.cpp # c++ -std=3Dc++17 -pedantic -O3 -pthread -c cpp_clockinfo.cpp # c++ -std=3Dc++17 -pedantic -O3 -pthread cpp_thousandslocale.o = cpp_clockinfo.o -o cpp_clockinfo_main cpp_clockinfo_main.cpp That last runs a linker aa part of its activity. It make s no use of makefile macros or environment variables LD or XLD. This would be true even if if comamnds were in a makefile. Use of the -### option for the last of those commands shows the link command that clang used ("/usr/bin/ld"): FBSDFHUGE# c++ -### -std=3Dc++17 -pedantic -O3 -pthread = cpp_thousandslocale.o cpp_clockinfo.o -o cpp_clockinfo_main = cpp_clockinfo_main.cpp FreeBSD clang version 8.0.1 (tags/RELEASE_801/final 366581) (based on = LLVM 8.0.1) Target: x86_64-unknown-freebsd13.0 Thread model: posix InstalledDir: /usr/bin "/usr/bin/c++" "-cc1" "-triple" "x86_64-unknown-freebsd13.0" = "-emit-obj" "-disable-free" "-main-file-name" "cpp_clockinfo_main.cpp" = "-mrelocation-model" "static" "-mthread-model" "posix" = "-mdisable-fp-elim" "-masm-verbose" "-mconstructor-aliases" = "-munwind-tables" "-fuse-init-array" "-target-cpu" "x86-64" = "-dwarf-column-info" "-debugger-tuning=3Dgdb" "-resource-dir" = "/usr/lib/clang/8.0.1" "-internal-isystem" "/usr/include/c++/v1" "-O3" = "-pedantic" "-std=3Dc++17" "-fdeprecated-macro" = "-fdebug-compilation-dir" "/root/c_tests" "-ferror-limit" "19" = "-fmessage-length" "200" "-pthread" "-fobjc-runtime=3Dgnustep" = "-fcxx-exceptions" "-fexceptions" "-fdiagnostics-show-option" = "-fcolor-diagnostics" "-vectorize-loops" "-vectorize-slp" "-o" = "/tmp/cpp_clockinfo_main-bf9d80.o" "-x" "c++" "cpp_clockinfo_main.cpp" = "-faddrsig" "/usr/bin/ld" "--eh-frame-hdr" "-dynamic-linker" "/libexec/ld-elf.so.1" = "--hash-style=3Dboth" "--enable-new-dtags" "-o" "cpp_clockinfo_main" = "/usr/lib/crt1.o" "/usr/lib/crti.o" "/usr/lib/crtbegin.o" "-L/usr/lib" = "cpp_thousandslocale.o" "cpp_clockinfo.o" = "/tmp/cpp_clockinfo_main-bf9d80.o" "-lc++" "-lm" "-lgcc" "--as-needed" = "-lgcc_s" "--no-as-needed" "-lpthread" "-lc" "-lgcc" "--as-needed" = "-lgcc_s" "--no-as-needed" "/usr/lib/crtend.o" "/usr/lib/crtn.o" Systenm-clang makes no use of ${LD} or ${XLD} but directly tries to use = /usr/bin/ld instead. Similarly for devel/llvm90 : its clang++90 uses "/usr/local/llvm90/bin/ld" directly: # clang++90 -std=3Dc++17 -pedantic -O3 -pthread -c = cpp_thousandslocale.cpp # clang++90 -std=3Dc++17 -pedantic -O3 -pthread -c cpp_clockinfo.cpp # clang++90 -std=3Dc++17 -pedantic -O3 -pthread cpp_thousandslocale.o = cpp_clockinfo.o -o cpp_clockinfo_main cpp_clockinfo_main.cpp Again using -### : # clang++90 -### -std=3Dc++17 -pedantic -O3 -pthread = cpp_thousandslocale.o cpp_clockinfo.o -o cpp_clockinfo_main = cpp_clockinfo_main.cpp clang version 9.0.0 (tags/RELEASE_900/rc2) Target: x86_64-portbld-freebsd13.0 Thread model: posix InstalledDir: /usr/local/llvm90/bin "/usr/local/llvm90/bin/clang-9" "-cc1" "-triple" = "x86_64-portbld-freebsd13.0" "-emit-obj" "-disable-free" = "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" = "cpp_clockinfo_main.cpp" "-mrelocation-model" "static" "-mthread-model" = "posix" "-mdisable-fp-elim" "-masm-verbose" "-mconstructor-aliases" = "-munwind-tables" "-fuse-init-array" "-target-cpu" "x86-64" = "-dwarf-column-info" "-debugger-tuning=3Dgdb" "-resource-dir" = "/usr/local/llvm90/lib/clang/9.0.0" "-internal-isystem" = "/usr/include/c++/v1" "-O3" "-pedantic" "-std=3Dc++17" = "-fdeprecated-macro" "-fdebug-compilation-dir" "/root/c_tests" = "-ferror-limit" "19" "-fmessage-length" "200" "-pthread" = "-fobjc-runtime=3Dgnustep" "-fcxx-exceptions" "-fexceptions" = "-fdiagnostics-show-option" "-fcolor-diagnostics" "-vectorize-loops" = "-vectorize-slp" "-faddrsig" "-o" "/tmp/cpp_clockinfo_main-bb1750.o" = "-x" "c++" "cpp_clockinfo_main.cpp" "/usr/local/llvm90/bin/ld" "--eh-frame-hdr" "-dynamic-linker" = "/libexec/ld-elf.so.1" "--hash-style=3Dboth" "--enable-new-dtags" "-o" = "cpp_clockinfo_main" "/usr/lib/crt1.o" "/usr/lib/crti.o" = "/usr/lib/crtbegin.o" "-L/usr/lib" "cpp_thousandslocale.o" = "cpp_clockinfo.o" "/tmp/cpp_clockinfo_main-bb1750.o" "-lc++" "-lm" = "-lgcc" "--as-needed" "-lgcc_s" "--no-as-needed" "-lpthread" "-lc" = "-lgcc" "--as-needed" "-lgcc_s" "--no-as-needed" "/usr/lib/crtend.o" = "/usr/lib/crtn.o" Similarly for g++9 which uses "/usr/local/bin/ld" directly: # g++9 -std=3Dc++17 -pedantic -O3 -pthread -c cpp_thousandslocale.cpp # g++9 -std=3Dc++17 -pedantic -O3 -pthread -c cpp_clockinfo.cpp # g++9 -std=3Dc++17 -pedantic -O3 -pthread cpp_thousandslocale.o = cpp_clockinfo.o -o cpp_clockinfo_main cpp_clockinfo_main.cpp Again using -### : # g++9 -### -std=3Dc++17 -pedantic -O3 -pthread cpp_thousandslocale.o = cpp_clockinfo.o -o cpp_clockinfo_main cpp_clockinfo_main.cpp Using built-in specs. COLLECT_GCC=3Dg++9 = COLLECT_LTO_WRAPPER=3D/usr/local/libexec/gcc9/gcc/x86_64-portbld-freebsd13= .0/9.2.0/lto-wrapper Target: x86_64-portbld-freebsd13.0 Configured with: /wrkdirs/usr/ports/lang/gcc9/work/gcc-9.2.0/configure = --enable-multilib --with-build-config=3Dbootstrap-debug --disable-nls = --enable-gnu-indirect-function --libdir=3D/usr/local/lib/gcc9 = --libexecdir=3D/usr/local/libexec/gcc9 --program-suffix=3D9 = --with-as=3D/usr/local/bin/as --with-gmp=3D/usr/local = --with-gxx-include-dir=3D/usr/local/lib/gcc9/include/c++/ = --with-ld=3D/usr/local/bin/ld --with-pkgversion=3D'FreeBSD Ports = Collection' --with-system-zlib --enable-languages=3Dc,c++,objc,fortran = --prefix=3D/usr/local --localstatedir=3D/var --mandir=3D/usr/local/man = --infodir=3D/usr/local/share/info/gcc9 = --build=3Dx86_64-portbld-freebsd13.0 Thread model: posix gcc version 9.2.0 (FreeBSD Ports Collection)=20 COLLECT_GCC_OPTIONS=3D'-std=3Dc++17' '-Wpedantic' '-O3' '-pthread' '-o' = 'cpp_clockinfo_main' '-shared-libgcc' '-mtune=3Dgeneric' '-march=3Dx86-64'= /usr/local/libexec/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/cc1plus = -quiet cpp_clockinfo_main.cpp -quiet -dumpbase cpp_clockinfo_main.cpp = "-mtune=3Dgeneric" "-march=3Dx86-64" -auxbase cpp_clockinfo_main -O3 = -Wpedantic "-std=3Dc++17" -o /tmp//ccqXOrjE.s COLLECT_GCC_OPTIONS=3D'-std=3Dc++17' '-Wpedantic' '-O3' '-pthread' '-o' = 'cpp_clockinfo_main' '-shared-libgcc' '-mtune=3Dgeneric' '-march=3Dx86-64'= /usr/local/bin/as -o /tmp//ccxktS2W.o /tmp//ccqXOrjE.s = COMPILER_PATH=3D/usr/local/libexec/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2= .0/:/usr/local/libexec/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/:/usr/loc= al/libexec/gcc9/gcc/x86_64-portbld-freebsd13.0/:/usr/local/lib/gcc9/gcc/x8= 6_64-portbld-freebsd13.0/9.2.0/:/usr/local/lib/gcc9/gcc/x86_64-portbld-fre= ebsd13.0/:/usr/local/lib/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/../../.= ./../../x86_64-portbld-freebsd13.0/bin/ = LIBRARY_PATH=3D/usr/local/lib/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/:/= usr/local/lib/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/../../../../../x86= _64-portbld-freebsd13.0/lib/:/usr/local/lib/gcc9/gcc/x86_64-portbld-freebs= d13.0/9.2.0/../../../:/lib/:/usr/lib/ COLLECT_GCC_OPTIONS=3D'-std=3Dc++17' '-Wpedantic' '-O3' '-pthread' '-o' = 'cpp_clockinfo_main' '-shared-libgcc' '-mtune=3Dgeneric' '-march=3Dx86-64'= /usr/local/libexec/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/collect2 = -plugin = /usr/local/libexec/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/liblto_plugin= .so = "-plugin-opt=3D/usr/local/libexec/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.= 0/lto-wrapper" "-plugin-opt=3D-fresolution=3D/tmp//ccA61h1w.res" = "-plugin-opt=3D-pass-through=3D-lgcc_s" = "-plugin-opt=3D-pass-through=3D-lgcc" = "-plugin-opt=3D-pass-through=3D-lpthread" = "-plugin-opt=3D-pass-through=3D-lc" "-plugin-opt=3D-pass-through=3D-lgcc_s= " "-plugin-opt=3D-pass-through=3D-lgcc" --eh-frame-hdr -m = elf_x86_64_fbsd -dynamic-linker /libexec/ld-elf.so.1 -o = cpp_clockinfo_main /usr/lib/crt1.o /usr/lib/crti.o = /usr/local/lib/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/crtbegin.o = -L/usr/local/lib/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0 = -L/usr/local/lib/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/../../../../../= x86_64-portbld-freebsd13.0/lib = -L/usr/local/lib/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/../../.. = cpp_thousandslocale.o cpp_clockinfo.o /tmp//ccxktS2W.o "-lstdc++" -lm = -lgcc_s -lgcc -lpthread -lc -lgcc_s -lgcc = /usr/local/lib/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/crtend.o = /usr/lib/crtn.o COLLECT_GCC_OPTIONS=3D'-std=3Dc++17' '-Wpedantic' '-O3' '-pthread' '-o' = 'cpp_clockinfo_main' '-shared-libgcc' '-mtune=3Dgeneric' '-march=3Dx86-64'= Although here it is less obvious what: /usr/local/libexec/gcc9/gcc/x86_64-portbld-freebsd13.0/9.2.0/collect2 ends up using. So using truss produced the evidence: 3472: execve("/usr/local/bin/ld",0x800b35180,0x7fffffffd380) EJUSTRETURN Again: no use of ${LD} or ${XLD} . Builds that want to control which linker is used need to avoid using such commands and instead use ${LD} or ${XLD} or such explicitly. Many do not do this. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)