From nobody Mon Jul 5 15:09:18 2021 X-Original-To: freebsd-toolchain@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 45A6F11D77F3; Mon, 5 Jul 2021 15:09:33 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJTcX1fq8z4RxW; Mon, 5 Jul 2021 15:09:32 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-il1-f182.google.com with SMTP id e13so1222868ilc.1; Mon, 05 Jul 2021 08:09:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=RUNYyPwAWHPMEvH6m9X36t5VSUOWMvaV5YQEAlVqRrY=; b=dQ+m73yuc+BXFvfJGahE/8y9pcTZRA0VsDiMo0nNMp8rmGn57ADf4EXM9pKEO74nSp sR2wC4Sh4yRtRdbButsBuSou7hHnT6xUDqa2sh53iq+8pZFckApsS0vw4B9zIi5f457o d7NJX0HD1EtjzgYw+gsyh6u3yUdQI/VXsMPBCiKG4uUmM7ePdR1SGM1HrPDP9+1Fjb7Z tILGKu81gZfSLaS14RMoDRqs5OsvDOKogZkqbGdxvQyjFgJNGUrdmpPElgcZXnCdJJY/ BnCn0ji/RJwR8NjOBkykEStI/zjyDkHIIOR6BnD1Dl+rkEou/vZ30Z3rElyzR9yZ16YG AOCQ== X-Gm-Message-State: AOAM533rTnUdet/hIzPNH3PElvG996IZT/+EIFRrOwW1H9PCXcE4PHuO sXPwrC30QsmuL4Va7kJWMLZVNxNEN9RveM+r3xZV58y0 X-Google-Smtp-Source: ABdhPJzk9FiLnMr4gyZ4/EqetI1k4pKHvc9kg2VdiAkpkbi9CAFv/Zt3NBXGCb0uHxlUQiqrYx0+Gh7K0VQuJTnHZ4Q= X-Received: by 2002:a92:d0d1:: with SMTP id y17mr11372496ila.256.1625497770284; Mon, 05 Jul 2021 08:09:30 -0700 (PDT) List-Id: Maintenance List-Archive: https://lists.freebsd.org/archives/freebsd-toolchain List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-toolchain@freebsd.org X-BeenThere: freebsd-toolchain@freebsd.org MIME-Version: 1.0 From: Ed Maste Date: Mon, 5 Jul 2021 11:09:18 -0400 Message-ID: Subject: Migrating to LLVM binutils tools (ar, nm, addr2line, etc.) To: "freebsd-toolchain@FreeBSD.org" , FreeBSD Hackers , Dimitry Andric , Alexander Richardson Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GJTcX1fq8z4RxW X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.166.182 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com X-Spamd-Result: default: False [-0.98 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.166.182:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[carpeddiem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.97)[-0.974]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; NEURAL_SPAM_MEDIUM(0.99)[0.992]; SPAMHAUS_ZRD(0.00)[209.85.166.182:from:127.0.2.255]; R_DKIM_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.182:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.182:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-toolchain,freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: N FreeBSD migrated from GNU binutils to versions from ELF Tool Chain, starting in 2014. At that time there were no usable LLVM versions of those tools, but they have been developing rapidly since then. Now I think it may be prudent to migrate to the LLVM tools where they exist, for both functionality and maintainability reasons. I'd like to allow use of link-time optimization (LTO) in the FreeBSD base system. LTO runs optimization passes over the entire executable (or library) at link time and thus allows for more effective optimization than when performed on individual compilation units. When using LTO object files (.o) including those contained in static library archives (.a) contain LLVM IR bitcode rather than target object code. This means that utilities that operate on object files need to support LLVM IR; we currently use a number of bespoke tools and ones obtained from ELF Tool Chain that do not have this support. Alex Richardson also pointed out that asan (address sanitizer) produces a useful backtrace only if addr2line is llvm-symbolizer. Like ELF Tool Chain the LLVM tools aim for command line and output format compatibility with GNU binutils, although there are a few minor differences. Where these cause a material issue (breaking a port or eliminating required functionality) we can submit LLVM bugs and work on patches. In the past we provided build knobs to control individual utilities (e.g. WITH_LLD_IS_LD); I'd like to avoid perpetuating that here. It seems individual knobs (WITH_LLVM_AR_IS_AR, WITH_LLVM_NM_IS_NM, WITH_LLVM_SYMBOLIZER_IS_ADDR2LINE etc.) will introduce extra complexity without adding much value. Alex is working on a patch now and will follow up shortly, but I wanted to email the list as a heads-up, and see if there are any comments or concerns. Potential next steps are: - Introduce new build knob - Iterate on exp-runs and call for testing - Switch to LLVM tools by default - Major release (14.0) - Retire knob, leaving only the LLVM implementation. From nobody Mon Jul 5 17:56:10 2021 X-Original-To: freebsd-toolchain@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9D36911D5A8D; Mon, 5 Jul 2021 17:56:23 +0000 (UTC) (envelope-from arichardson.kde@gmail.com) Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJYK33qkjz3FLm; Mon, 5 Jul 2021 17:56:23 +0000 (UTC) (envelope-from arichardson.kde@gmail.com) Received: by mail-ej1-f41.google.com with SMTP id bg14so30018650ejb.9; Mon, 05 Jul 2021 10:56:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tyeG0+uP3jlC9MrCLlel3VHagTU1hREcZfOK3elRXwA=; b=HVJOE24O28XoxatAi8OjQqBAvL1GerMyE7COahU7twRdAzSaQDzuQaZOIUYbT9qC7j G7pC5rsngGuwhVs3KRIhLEMvrxNZmEcFA/HzUnxMKKAc/IabsY9J18eL7bb0cxDa+G6H UGTiRX8jmYPeGGa5JKQNFxNX+QD894ChkPn4Jtl1NN14qeXFnbml2iJ96pMMjudpPy3C qQ393EAifztOD4BPBkErjoEZurkD/MPeUSk1+3sEPIdBRCYl4EE+1qZV6BhAbhxlDnwB PbVXeZi4EUV0thPOkE828SS5LCCNfYDeHi6ArwmewR3hCFu+gZwhWJJqg3u+PdETClcN 0cEA== X-Gm-Message-State: AOAM530EBTzagdz4t8nQ5NYVx/F3WSyC5jA3tTRmx+8Lyq296eOql+g8 QpYZrnvV6ex/bFAjxWKb8TJ2BWUNZUSQ8g== X-Google-Smtp-Source: ABdhPJxjrQM1Awz3AepQlQPCbVXaP+457AYm9RZXGKdeYP0JvbHagISHQR63OBtTYfIDFoOLFc6wEw== X-Received: by 2002:a17:906:b190:: with SMTP id w16mr14631217ejy.332.1625507782113; Mon, 05 Jul 2021 10:56:22 -0700 (PDT) Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com. [209.85.128.52]) by smtp.gmail.com with ESMTPSA id f9sm4101537edw.88.2021.07.05.10.56.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 05 Jul 2021 10:56:21 -0700 (PDT) Received: by mail-wm1-f52.google.com with SMTP id j16-20020a05600c1c10b0290204b096b0caso493949wms.1; Mon, 05 Jul 2021 10:56:21 -0700 (PDT) X-Received: by 2002:a1c:9d8f:: with SMTP id g137mr282085wme.13.1625507781650; Mon, 05 Jul 2021 10:56:21 -0700 (PDT) List-Id: Maintenance List-Archive: https://lists.freebsd.org/archives/freebsd-toolchain List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-toolchain@freebsd.org X-BeenThere: freebsd-toolchain@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alexander Richardson Date: Mon, 5 Jul 2021 18:56:10 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Migrating to LLVM binutils tools (ar, nm, addr2line, etc.) To: Ed Maste Cc: "freebsd-toolchain@FreeBSD.org" , FreeBSD Hackers , Dimitry Andric Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GJYK33qkjz3FLm X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Mon, 5 Jul 2021 at 16:09, Ed Maste wrote: > > FreeBSD migrated from GNU binutils to versions from ELF Tool Chain, > starting in 2014. At that time there were no usable LLVM versions of > those tools, but they have been developing rapidly since then. Now I > think it may be prudent to migrate to the LLVM tools where they exist, > for both functionality and maintainability reasons. > > I'd like to allow use of link-time optimization (LTO) in the FreeBSD > base system. LTO runs optimization passes over the entire executable > (or library) at link time and thus allows for more effective > optimization than when performed on individual compilation units. > > When using LTO object files (.o) including those contained in static > library archives (.a) contain LLVM IR bitcode rather than target > object code. This means that utilities that operate on object files > need to support LLVM IR; we currently use a number of bespoke tools > and ones obtained from ELF Tool Chain that do not have this support. > > Alex Richardson also pointed out that asan (address sanitizer) > produces a useful backtrace only if addr2line is llvm-symbolizer. > > Like ELF Tool Chain the LLVM tools aim for command line and output > format compatibility with GNU binutils, although there are a few minor > differences. Where these cause a material issue (breaking a port or > eliminating required functionality) we can submit LLVM bugs and work > on patches. > > In the past we provided build knobs to control individual utilities > (e.g. WITH_LLD_IS_LD); I'd like to avoid perpetuating that here. It > seems individual knobs (WITH_LLVM_AR_IS_AR, WITH_LLVM_NM_IS_NM, > WITH_LLVM_SYMBOLIZER_IS_ADDR2LINE etc.) will introduce extra > complexity without adding much value. > > Alex is working on a patch now and will follow up shortly, but I > wanted to email the list as a heads-up, and see if there are any > comments or concerns. Thanks for writing this up, I've posted the initial patch as https://reviews.freebsd.org/D31060. > > Potential next steps are: > - Introduce new build knob > - Iterate on exp-runs and call for testing > - Switch to LLVM tools by default > - Major release (14.0) > - Retire knob, leaving only the LLVM implementation. From nobody Sun Jul 11 21:00:53 2021 X-Original-To: toolchain@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1AF8812A269A for ; Sun, 11 Jul 2021 21:00:55 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNK794znpz3Nbr for ; Sun, 11 Jul 2021 21:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 79A29225BE for ; Sun, 11 Jul 2021 21:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 16BL0rPi032733 for ; Sun, 11 Jul 2021 21:00:53 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 16BL0rdc032732 for toolchain@FreeBSD.org; Sun, 11 Jul 2021 21:00:53 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202107112100.16BL0rdc032732@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: toolchain@FreeBSD.org Subject: Problem reports for toolchain@FreeBSD.org that need special attention Date: Sun, 11 Jul 2021 21:00:53 +0000 List-Id: Maintenance List-Archive: https://lists.freebsd.org/archives/freebsd-toolchain List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-toolchain@freebsd.org X-BeenThere: freebsd-toolchain@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="16260372530.edbdbfd.30613" Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N --16260372530.edbdbfd.30613 Date: Sun, 11 Jul 2021 21:00:53 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 234232 | clang Assertion failed when building the port dev Open | 245179 | lld: wrong/misleading "SHF_MERGE section size mus Open | 247665 | emulators/rpcs3: clang 10 crashes during build 3 problems total for which you should take action. --16260372530.edbdbfd.30613--