From owner-freebsd-toolchain@FreeBSD.ORG Tue Sep 30 11:35:49 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D0A9267; Tue, 30 Sep 2014 11:35:49 +0000 (UTC) Received: from mail-la0-x22d.google.com (mail-la0-x22d.google.com [IPv6:2a00:1450:4010:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 57C0AB96; Tue, 30 Sep 2014 11:35:48 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id q1so8364207lam.18 for ; Tue, 30 Sep 2014 04:35:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=x4TLYILdZDTf3eXdV10v0WSDAq/LpQdPKZp2nCFRrDM=; b=KHYHN/AbVZduM0xjHDvyWZBtR2cysP5oSDTCBFF01dWkYxQc2TXzvnMnv9V5wunWGy SyJ2A+1q3o0aFBOiywgLHPDU/9IN+UyhC9/n3bMxDFvT/ncMLjx/0I5bhoH+MsG/k6hN 8juwBhLFvjcstwe1fN5GO6kTvHsa/plP+elFSZ5n3F37n13lvZXJZ6a0AmLM67G/WTMY f4ULYBlZX47jZEaMHsC3ENJuH4HzVY8bCkYxtZ6KTGOAdhUv6NSnLgGFdd8V7g27U3sX fFkNZEbmuVm09lbYakjE7Gl9GicRXrNDbbKEi+kpiVJnGT+ZNtXkBU6YBlnP7YjR78mU 6aeQ== X-Received: by 10.152.87.10 with SMTP id t10mr23577909laz.22.1412076946302; Tue, 30 Sep 2014 04:35:46 -0700 (PDT) Received: from localhost (s83-179-26-16.cust.tele2.se. [83.179.26.16]) by mx.google.com with ESMTPSA id d7sm5927724lad.4.2014.09.30.04.35.45 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 30 Sep 2014 04:35:45 -0700 (PDT) Sender: Kai Wang Received: from localhost.localdomain ([127.0.0.1] helo=localhost.my.domain) by localhost with esmtp (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1XYvib-0001yT-3E; Tue, 30 Sep 2014 13:35:45 +0200 Received: (from kaiw@localhost) by localhost.my.domain (8.14.5/8.14.5/Submit) id s8UBZid7007592; Tue, 30 Sep 2014 13:35:44 +0200 (CEST) (envelope-from kaiw@FreeBSD.org) X-Authentication-Warning: localhost.my.domain: kaiw set sender to kaiw@FreeBSD.org using -f Date: Tue, 30 Sep 2014 13:35:44 +0200 From: Kai Wang To: Will Andrews Subject: Re: elftoolchain update? Message-ID: <20140930113544.GA7285@soulhacker> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: jkoshy@FreeBSD.org, freebsd-toolchain@FreeBSD.org X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Sep 2014 11:35:49 -0000 Hello Will, My apologies for the delayed reply. I've totally missed these few emails. I can investigate the C++ object files issue and make fixes for them ASAP. Regarding header, it was used for elftoolchain to build on various OS independently. When merging elftoolchain back to FreeBSD, I found it's very hard to use the header because of the conflicts. So I decided to continue using the existing system ELF headers. You approach of merging the missing defines from to the elfXX.h headers looks good to me. Thanks, Kai On Fri, Sep 26, 2014 at 11:32:20AM -0600, Will Andrews wrote: > Hi, > > I've created a git-svn clone of the current elftoolchain, and applied > some fixes in a branch: > https://github.com/wca/elftoolchain/tree/freebsd > > Any objections if I update the copy in head using this branch? I've > heard nothing from either Kai or Joseph. > > I'm not done testing yet -- it looks like there are some more bugfixes > needed to get ctfconvert to at least run against C++ object files > without bailing. Just wanted to know if there are any specific > concerns that people might have. > > A related review involves an update for the ELF headers: > https://reviews.freebsd.org/D844 > > I haven't finished testing this either (need to do an universe build > to check for conflicts), but my goal here is to achieve header parity > with from elftoolchain, which is largely duplicate. > The elftoolchain header exports many more symbols that are used by its > userland programs. This would include arch-specific interpretations > of some ELF structures. I believe it's appropriate to export these on > a global basis, given that userland programs can legitimately be run > on object files built for architectures other than the system they're > running on. > > Thanks! > --Will. > > On Wed, Sep 17, 2014 at 5:01 PM, Will Andrews wrote: > > Hi, > > > > I see there have been a lot of updates & fixes to elftoolchain since > > the last import into FreeBSD/head nearly 8 months ago. Are there any > > plans to update the import? > > > > I'm asking because it appears that ctfconvert currently crashes > > (specifically, due to a bug in dwarf_attrval_unsigned()), if you try > > to use it on C++ object files. > > > > This is easily demonstrated by applying this patch to FreeBSD/head and > > building sbin/devd with WITH_CTF=1: > > http://people.freebsd.org/~will/add-ctfconvert-to-cpp-object-files.diff > > > > Justin Gibbs (cc'd) posted about this issue in February, and it's > > still a problem: > > http://lists.freebsd.org/pipermail/freebsd-toolchain/2014-February/001121.html > > > > Thanks, > > --Will. From owner-freebsd-toolchain@FreeBSD.ORG Tue Sep 30 15:10:27 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3B7A5D2 for ; Tue, 30 Sep 2014 15:10:27 +0000 (UTC) Received: from mail-qg0-f43.google.com (mail-qg0-f43.google.com [209.85.192.43]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F1EA59C9 for ; Tue, 30 Sep 2014 15:10:26 +0000 (UTC) Received: by mail-qg0-f43.google.com with SMTP id j107so4918264qga.2 for ; Tue, 30 Sep 2014 08:10:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=I1iKo4JLtO0nW+taA6rrVtyszjZRmwchFAnCqha9EO4=; b=mf07TsxETcZSUjz1x4i0BeJfkNTB/rwHa5MZ8lVePqFHQJWGndSs55l/caSG6wXNpr fSdTdnSC4Q8WG4/23QknH3At9f1jAHajnWPLWs0gjwd/h1snVMph04q2SDRi09yJ29U4 Q0zwibx+nOLxzTJqDXWSQ+5fX6WBydvqVyIhzBMbJCBEa86SIjArqni7TSI3rwINGiAN SAPwuUdQs0NNeluSBUmXHeuAKzBQTXigA42G1tCnwgt0rSSvyd9rJMRlKqgVqM5t/nAN DI8bymaJJR44OEAdR67jfYYSOkmjnZlCjwoeURQP1KHzvCuHuplNfOwW6qFyoijdcl63 Wbuw== X-Gm-Message-State: ALoCoQnJtYcvhYMb6U0Dzm+APVgLGPxVrpjcIXuwfrO9Gw5TqzOrh3tRGZwuem+vefozanGjWh3/ MIME-Version: 1.0 X-Received: by 10.224.127.131 with SMTP id g3mr20151899qas.81.1412089816240; Tue, 30 Sep 2014 08:10:16 -0700 (PDT) Received: by 10.140.16.183 with HTTP; Tue, 30 Sep 2014 08:10:16 -0700 (PDT) In-Reply-To: <20140930113544.GA7285@soulhacker> References: <20140930113544.GA7285@soulhacker> Date: Tue, 30 Sep 2014 09:10:16 -0600 Message-ID: Subject: Re: elftoolchain update? From: Will Andrews To: Kai Wang Content-Type: text/plain; charset=UTF-8 Cc: jkoshy@freebsd.org, freebsd-toolchain@freebsd.org X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Sep 2014 15:10:27 -0000 Thanks Kai. Could you please review the changes I made on my branch? I'm still working on this issue. --Will. On Tue, Sep 30, 2014 at 5:35 AM, Kai Wang wrote: > Hello Will, > > My apologies for the delayed reply. I've totally missed these few > emails. > > I can investigate the C++ object files issue and make fixes for > them ASAP. > > Regarding header, it was used for elftoolchain to > build on various OS independently. When merging elftoolchain back to > FreeBSD, I found it's very hard to use the header > because of the conflicts. So I decided to continue using the existing > system ELF headers. You approach of merging the missing defines from > to the elfXX.h headers looks good to me. > > Thanks, > Kai > > On Fri, Sep 26, 2014 at 11:32:20AM -0600, Will Andrews wrote: >> Hi, >> >> I've created a git-svn clone of the current elftoolchain, and applied >> some fixes in a branch: >> https://github.com/wca/elftoolchain/tree/freebsd >> >> Any objections if I update the copy in head using this branch? I've >> heard nothing from either Kai or Joseph. >> >> I'm not done testing yet -- it looks like there are some more bugfixes >> needed to get ctfconvert to at least run against C++ object files >> without bailing. Just wanted to know if there are any specific >> concerns that people might have. >> >> A related review involves an update for the ELF headers: >> https://reviews.freebsd.org/D844 >> >> I haven't finished testing this either (need to do an universe build >> to check for conflicts), but my goal here is to achieve header parity >> with from elftoolchain, which is largely duplicate. >> The elftoolchain header exports many more symbols that are used by its >> userland programs. This would include arch-specific interpretations >> of some ELF structures. I believe it's appropriate to export these on >> a global basis, given that userland programs can legitimately be run >> on object files built for architectures other than the system they're >> running on. >> >> Thanks! >> --Will. >> >> On Wed, Sep 17, 2014 at 5:01 PM, Will Andrews wrote: >> > Hi, >> > >> > I see there have been a lot of updates & fixes to elftoolchain since >> > the last import into FreeBSD/head nearly 8 months ago. Are there any >> > plans to update the import? >> > >> > I'm asking because it appears that ctfconvert currently crashes >> > (specifically, due to a bug in dwarf_attrval_unsigned()), if you try >> > to use it on C++ object files. >> > >> > This is easily demonstrated by applying this patch to FreeBSD/head and >> > building sbin/devd with WITH_CTF=1: >> > http://people.freebsd.org/~will/add-ctfconvert-to-cpp-object-files.diff >> > >> > Justin Gibbs (cc'd) posted about this issue in February, and it's >> > still a problem: >> > http://lists.freebsd.org/pipermail/freebsd-toolchain/2014-February/001121.html >> > >> > Thanks, >> > --Will. From owner-freebsd-toolchain@FreeBSD.ORG Thu Oct 2 06:20:54 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D46CCD8 for ; Thu, 2 Oct 2014 06:20:54 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14645B17 for ; Thu, 2 Oct 2014 06:20:54 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s926KrXx089877 for ; Thu, 2 Oct 2014 06:20:53 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Thu, 02 Oct 2014 06:20:52 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: Needs MFC X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: imp@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Oct 2014 06:20:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 Garrett Cooper changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hsn@sendmail.cz --- Comment #6 from Garrett Cooper --- *** Bug 193558 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Thu Oct 2 06:24:19 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6CFCAD67 for ; Thu, 2 Oct 2014 06:24:19 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 53F7EBC3 for ; Thu, 2 Oct 2014 06:24:19 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s926OJ1U094458 for ; Thu, 2 Oct 2014 06:24:19 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Thu, 02 Oct 2014 06:24:19 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: gjb@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: imp@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Oct 2014 06:24:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 Glen Barber changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved Resolution|--- |FIXED --- Comment #7 from Glen Barber --- r272372 -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Thu Oct 2 09:11:55 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A3B59196; Thu, 2 Oct 2014 09:11:55 +0000 (UTC) Received: from mail-yh0-x230.google.com (mail-yh0-x230.google.com [IPv6:2607:f8b0:4002:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 26D7214C; Thu, 2 Oct 2014 09:11:55 +0000 (UTC) Received: by mail-yh0-f48.google.com with SMTP id v1so548597yhn.7 for ; Thu, 02 Oct 2014 02:11:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Cy++0p8RjpW05cvB1X1BTIs5XPUXlyfBlLnWUaNa6HQ=; b=tGGCYCXmXGORpYLrI3t1cuxIwFyZirWu7+3oo/vVAd8ArrQbRH+YYDaRmTHOeaixVQ WDGfGImPJrpT68Yz2qFd/i/f8Yuh8fbGkeom+0/gLM4HIHMlRglkPmRglbqRqasgtCF/ xprdQaOFoRgEpqPleWX0uKr4/jt+nFnonyviBWWqHNr9b7fg70P81jXxv5pwRQh3Nr3N u3v5GzO6zKmhhtcGap2vo6bahzamP5o3Zyu4P8I9et49/BhydldKNwkuDESTMvqVe81S b+GQT/2ctJXO6xxjOFVGBLjkJGXJP+ltoEqeavZFltpJgvgnTdwMbbeg8/8/4rWGNxQv xIiw== MIME-Version: 1.0 X-Received: by 10.236.208.2 with SMTP id p2mr71034yho.173.1412241114220; Thu, 02 Oct 2014 02:11:54 -0700 (PDT) Received: by 10.170.110.196 with HTTP; Thu, 2 Oct 2014 02:11:54 -0700 (PDT) In-Reply-To: References: Date: Thu, 2 Oct 2014 11:11:54 +0200 Message-ID: Subject: Re: elftoolchain update? From: Kai Wang To: Dimitry Andric Content-Type: multipart/mixed; boundary=001a11c1c7f6f3aeb405046d01fc X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Justin Gibbs , jkoshy@freebsd.org, freebsd-toolchain@freebsd.org X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Oct 2014 09:11:55 -0000 --001a11c1c7f6f3aeb405046d01fc Content-Type: text/plain; charset=UTF-8 Hello, Thanks for the backtrace and analysis. I attached a patch for libdwarf and ctfconvert to fix the crash issue. The libdwarf patch is the same as Will submitted, it adds check for NULL attribute. The ctfconvert patch fixes some issue with die_name(). We can't let die_name() return NULL because we need the empty string "" for type name comparison. Instead I added checks for empty string when creating variables and functions. However, this patch only fixes the crash issue. ctfconvert will still fail and complains "unresolved types" when invoked on devd (or other C++ objects) The problem is that ctfconvert doesn't understand any C++ DWARF types, for example: class, namespace, templates etc. Then I checked the Dtrace CTF format: sys/cddl/contrib/opensolaris/uts/common/sys/ctf.h It seems to me that CTF can only support ANSI C ? Did anyone ever use Dtrace with C++ program and get debugging info? /Kai 2014-09-18 20:46 GMT+02:00 Dimitry Andric : > On 18 Sep 2014, at 01:01, Will Andrews wrote: > > I see there have been a lot of updates & fixes to elftoolchain since > > the last import into FreeBSD/head nearly 8 months ago. Are there any > > plans to update the import? > > > > I'm asking because it appears that ctfconvert currently crashes > > (specifically, due to a bug in dwarf_attrval_unsigned()), if you try > > to use it on C++ object files. > > > > This is easily demonstrated by applying this patch to FreeBSD/head and > > building sbin/devd with WITH_CTF=1: > > http://people.freebsd.org/~will/add-ctfconvert-to-cpp-object-files.diff > > > > Justin Gibbs (cc'd) posted about this issue in February, and it's > > still a problem: > > > http://lists.freebsd.org/pipermail/freebsd-toolchain/2014-February/001121.html > > In that previous thread, I was not able to reproduce any problems with > ctfconvert or ctfmerge, but I have tried it again just now, and I think > it is a problem in libdwarf. > > The crash goes like this: > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 28803080 (LWP 100196)] > 0x280bb75d in dwarf_attrval_unsigned (die=0x28941f10, attr=73, > valp=0xbfbfdea0, err=0xbfbfe0a4) at > /usr/src/lib/libdwarf/../../contrib/elftoolchain/libdwarf/dwarf_attrval.c:186 > 186 switch (at->at_form) { > (gdb) bt > #0 0x280bb75d in dwarf_attrval_unsigned (die=0x28941f10, attr=73, > valp=0xbfbfdea0, err=0xbfbfe0a4) at > /usr/src/lib/libdwarf/../../contrib/elftoolchain/libdwarf/dwarf_attrval.c:186 > #1 0x08052a45 in die_attr_ref (dw=0xbfbfe0a0, die=0x28941f10, name=73) at > /usr/src/cddl/usr.bin/ctfconvert/../../../cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c:417 > #2 0x08052844 in die_lookup_pass1 (dw=0xbfbfe0a0, die=0x28941f10, > name=73) at > /usr/src/cddl/usr.bin/ctfconvert/../../../cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c:476 > #3 0x08052380 in die_variable_create (dw=0xbfbfe0a0, die=0x28941f10, > off=83907, tdp=0x0) at > /usr/src/cddl/usr.bin/ctfconvert/../../../cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c:1680 > #4 0x08050940 in die_create_one (dw=0xbfbfe0a0, die=0x28941f10) at > /usr/src/cddl/usr.bin/ctfconvert/../../../cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c:1793 > #5 0x0804fa94 in die_create (dw=0xbfbfe0a0, die=0x28941f10) at > /usr/src/cddl/usr.bin/ctfconvert/../../../cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c:1800 > #6 0x0804f368 in dw_read (td=0x2881c040, elf=0x28830040, > filename=0xbfbfe83e "devd.o") at > /usr/src/cddl/usr.bin/ctfconvert/../../../cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c:2003 > #7 0x0804eb6e in file_read (td=0x2881c040, filename=0xbfbfe83e "devd.o", > ignore_non_c=0) at > /usr/src/cddl/usr.bin/ctfconvert/../../../cddl/contrib/opensolaris/tools/ctf/cvt/ctfconvert.c:115 > #8 0x0804e7ca in main (argc=5, argv=0xbfbfe694) at > /usr/src/cddl/usr.bin/ctfconvert/../../../cddl/contrib/opensolaris/tools/ctf/cvt/ctfconvert.c:236 > (gdb) print at > $1 = (Dwarf_Attribute) 0x0 > > Looking at dwarf_attrval_unsigned(), you can see 'at' being NULL-checked > in line 163, but if the _dwarf_attr_find() call on line 164 then also > returns NULL, the switch on line 186 will segfault as above: > > 140 int > 141 dwarf_attrval_unsigned(Dwarf_Die die, Dwarf_Half attr, > Dwarf_Unsigned *valp, Dwarf_Error *err) > 142 { > 143 Dwarf_Attribute at; > ... > 157 if ((at = _dwarf_attr_find(die, attr)) == NULL && attr != > DW_AT_type) { > 158 DWARF_SET_ERROR(dbg, err, DW_DLE_NO_ENTRY); > 159 return (DW_DLV_NO_ENTRY); > 160 } > 161 > 162 die1 = NULL; > 163 if (at == NULL && > 164 (at = _dwarf_attr_find(die, DW_AT_abstract_origin)) != > NULL) { > ... > 184 } > 185 > 186 switch (at->at_form) { > ... > > I'm not sure what kind of error code should be returned when the second > _dwarf_attr_find() fails, though. Or if that is some sort of problem > with a symbol? If I go to frame 3 (die_variable_create), the name seems > to be the empty string, but not a NULL pointer: > > (gdb) frame 3 > #3 0x08052380 in die_variable_create (dw=0xbfbfe0a0, die=0x28941f10, > off=83907, tdp=0x0) at > /usr/src/cddl/usr.bin/ctfconvert/../../../cddl/contrib/opensolaris/tools/ctf/cvt/dwarf.c:1680 > 1680 ii->ii_dtype = die_lookup_pass1(dw, die, DW_AT_type); > (gdb) print name > $2 = 0x2892dc90 "" > > The name is looked up on line 1674, where nameless objects are supposed > to be skipped: > > 1666 static void > 1667 die_variable_create(dwarf_t *dw, Dwarf_Die die, Dwarf_Off off, > tdesc_t *tdp __unused) > 1668 { > 1669 iidesc_t *ii; > 1670 char *name; > 1671 > 1672 debug(3, "die %llu: creating object definition\n", off); > 1673 > 1674 if (die_isdecl(dw, die) || (name = die_name(dw, die)) == > NULL) > 1675 return; /* skip prototypes and nameless objects */ > 1676 > 1677 ii = xcalloc(sizeof (iidesc_t)); > 1678 ii->ii_type = die_isglobal(dw, die) ? II_GVAR : II_SVAR; > 1679 ii->ii_name = name; > 1680 ii->ii_dtype = die_lookup_pass1(dw, die, DW_AT_type); > > However, die_name() does not ever seem to return NULL (the code to > return the empty string was added by Kai in r261246): > > 425 static char * > 426 die_name(dwarf_t *dw, Dwarf_Die die) > 427 { > 428 char *str = NULL; > 429 > 430 (void) die_string(dw, die, DW_AT_name, &str, 0); > 431 if (str == NULL) > 432 str = xstrdup(""); > 433 > 434 return (str); > 435 } > > There are quite a lot of places in this file where the result of > die_name() is explicitly checked against NULL, so maybe always returning > an empty string was not such a good idea. It may have been done to > avoid another segfault. > > The way forward is probably to: > * fix the situation in dwarf_attrval_unsigned(), returning a sensible > error value if both lookups fail. > * make die_name() return a NULL pointer again, or explicitly check for > the empty string in die_variable_create(). > > -Dimitry > > --001a11c1c7f6f3aeb405046d01fc Content-Type: text/plain; charset=US-ASCII; name="libdwarf_ctfconvert.txt" Content-Disposition: attachment; filename="libdwarf_ctfconvert.txt" Content-Transfer-Encoding: base64 X-Attachment-Id: f_i0rvv6uy1 SW5kZXg6IGNkZGwvY29udHJpYi9vcGVuc29sYXJpcy90b29scy9jdGYvY3Z0L2R3YXJmLmMKPT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PQotLS0gY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3Rvb2xzL2N0Zi9jdnQvZHdhcmYu YwkocmV2aXNpb24gMjcyMzk0KQorKysgY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3Rvb2xzL2N0 Zi9jdnQvZHdhcmYuYwkod29ya2luZyBjb3B5KQpAQCAtOTkwLDggKzk5MCw3IEBACiAJCSAqIGlu Zm8gZm9yIGFub24gc3RydWN0cywgdGhvdWdoIHJlY2VudCB2ZXJzaW9ucyBhcmUgZml4ZWQgKGdj YwogCQkgKiBidWcgMTE4MTYpLgogCQkgKi8KLQkJaWYgKChtbC0+bWxfbmFtZSA9IGRpZV9uYW1l KGR3LCBtZW0pKSA9PSBOVUxMKQotCQkJbWwtPm1sX25hbWUgPSBOVUxMOworCQltbC0+bWxfbmFt ZSA9IGRpZV9uYW1lKGR3LCBtZW0pOwogCiAJCW1sLT5tbF90eXBlID0gZGllX2xvb2t1cF9wYXNz MShkdywgbWVtLCBEV19BVF90eXBlKTsKIAkJZGVidWcoMywgImRpZV9zb3VfY3JlYXRlKCk6IG1s X3R5cGUgPSAlcCB0X2lkID0gJWRcbiIsCkBAIC0xMTMyLDcgKzExMzEsMTEgQEAKIAlmb3IgKG1s ID0gdGRwLT50X21lbWJlcnM7IG1sICE9IE5VTEw7IG1sID0gbWwtPm1sX25leHQpIHsKIAkJaWYg KG1sLT5tbF9zaXplID09IDApIHsKIAkJCW10ID0gdGRlc2NfYmFzZXR5cGUobWwtPm1sX3R5cGUp OwotCisJCQlpZiAobXQgPT0gTlVMTCkgeworCQkJCS8qIFByb2JhYmx5IEMrKyB0eXBlcy4gKi8K KwkJCQlkdy0+ZHdfbnVucmVzKys7CisJCQkJcmV0dXJuICgxKTsKKwkJCX0KIAkJCWlmICgobWwt Pm1sX3NpemUgPSB0ZGVzY19iaXRzaXplKG10KSkgIT0gMCkKIAkJCQljb250aW51ZTsKIApAQCAt MTU5NCwxMyArMTU5NywxNyBAQAogCQl9CiAJfQogCi0JaWYgKGRpZV9pc2RlY2woZHcsIGRpZSkg fHwgKG5hbWUgPSBkaWVfbmFtZShkdywgZGllKSkgPT0gTlVMTCkgewotCQkvKgotCQkgKiBXZSBw cm9jZXNzIG5laXRoZXIgcHJvdG90eXBlcyBub3Igc3VicHJvZ3JhbXMgd2l0aG91dAotCQkgKiBu YW1lcy4KLQkJICovCisJLyoKKwkgKiBXZSBwcm9jZXNzIG5laXRoZXIgcHJvdG90eXBlcyBub3Ig c3VicHJvZ3JhbXMgd2l0aG91dAorCSAqIG5hbWVzLgorCSAqLworCW5hbWUgPSBkaWVfbmFtZShk dywgZGllKTsKKwlpZiAoKm5hbWUgPT0gJ1wwJykgeworCQlmcmVlKG5hbWUpOwogCQlyZXR1cm47 CiAJfQorCWlmIChkaWVfaXNkZWNsKGR3LCBkaWUpKQorCQlyZXR1cm47CiAKIAlpaSA9IHhjYWxs b2Moc2l6ZW9mIChpaWRlc2NfdCkpOwogCWlpLT5paV90eXBlID0gZGllX2lzZ2xvYmFsKGR3LCBk aWUpID8gSUlfR0ZVTiA6IElJX1NGVU47CkBAIC0xNjI2LDExICsxNjMzLDcgQEAKIAkJaWYgKGRp ZV90YWcoZHcsIGFyZykgIT0gRFdfVEFHX2Zvcm1hbF9wYXJhbWV0ZXIpCiAJCQljb250aW51ZTsK IAotCQlpZiAoKG5hbWUxID0gZGllX25hbWUoZHcsIGFyZykpID09IE5VTEwpIHsKLQkJCXRlcm1p bmF0ZSgiZGllICVsbHU6IGZ1bmMgYXJnICVkIGhhcyBubyBuYW1lXG4iLAotCQkJICAgIG9mZiwg aWktPmlpX25hcmdzICsgMSk7Ci0JCX0KLQorCQluYW1lMSA9IGRpZV9uYW1lKGR3LCBhcmcpOwog CQlpZiAoc3RyY21wKG5hbWUxLCAiLi4uIikgPT0gMCkgewogCQkJZnJlZShuYW1lMSk7CiAJCQlp aS0+aWlfdmFyZ3MgPSAxOwpAQCAtMTY3MSw4ICsxNjc0LDE0IEBACiAKIAlkZWJ1ZygzLCAiZGll ICVsbHU6IGNyZWF0aW5nIG9iamVjdCBkZWZpbml0aW9uXG4iLCBvZmYpOwogCi0JaWYgKGRpZV9p c2RlY2woZHcsIGRpZSkgfHwgKG5hbWUgPSBkaWVfbmFtZShkdywgZGllKSkgPT0gTlVMTCkKLQkJ cmV0dXJuOyAvKiBza2lwIHByb3RvdHlwZXMgYW5kIG5hbWVsZXNzIG9iamVjdHMgKi8KKwkvKiBz a2lwIHByb3RvdHlwZXMgYW5kIG5hbWVsZXNzIG9iamVjdHMgKi8KKwluYW1lID0gZGllX25hbWUo ZHcsIGRpZSk7CisJaWYgKCpuYW1lID09ICdcMCcpIHsKKwkJZnJlZShuYW1lKTsKKwkJcmV0dXJu OworCX0KKwlpZiAoZGllX2lzZGVjbChkdywgZGllKSkKKwkJcmV0dXJuOwogCiAJaWkgPSB4Y2Fs bG9jKHNpemVvZiAoaWlkZXNjX3QpKTsKIAlpaS0+aWlfdHlwZSA9IGRpZV9pc2dsb2JhbChkdywg ZGllKSA/IElJX0dWQVIgOiBJSV9TVkFSOwpAQCAtMTk5MSw3ICsyMDAwLDggQEAKIAkJZnJlZShw cm9kKTsKIAl9CiAKLQlpZiAoKGR3LmR3X2N1bmFtZSA9IGRpZV9uYW1lKCZkdywgY3UpKSAhPSBO VUxMKSB7CisJZHcuZHdfY3VuYW1lID0gZGllX25hbWUoJmR3LCBjdSk7CisJaWYgKCpkdy5kd19j dW5hbWUgIT0gJ1wwJykgewogCQljaGFyICpiYXNlID0geHN0cmR1cChiYXNlbmFtZShkdy5kd19j dW5hbWUpKTsKIAkJZnJlZShkdy5kd19jdW5hbWUpOwogCQlkdy5kd19jdW5hbWUgPSBiYXNlOwpJ bmRleDogY29udHJpYi9lbGZ0b29sY2hhaW4vbGliZHdhcmYvZHdhcmZfYXR0cnZhbC5jCj09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT0KLS0tIGNvbnRyaWIvZWxmdG9vbGNoYWluL2xpYmR3YXJmL2R3YXJmX2F0dHJ2YWwuYwko cmV2aXNpb24gMjcyMzk0KQorKysgY29udHJpYi9lbGZ0b29sY2hhaW4vbGliZHdhcmYvZHdhcmZf YXR0cnZhbC5jCSh3b3JraW5nIGNvcHkpCkBAIC0xNjAsOCArMTYwLDEyIEBACiAJfQogCiAJZGll MSA9IE5VTEw7Ci0JaWYgKGF0ID09IE5VTEwgJiYKLQkgICAgKGF0ID0gX2R3YXJmX2F0dHJfZmlu ZChkaWUsIERXX0FUX2Fic3RyYWN0X29yaWdpbikpICE9IE5VTEwpIHsKKwlpZiAoYXQgPT0gTlVM TCkgeworCQlhdCA9IF9kd2FyZl9hdHRyX2ZpbmQoZGllLCBEV19BVF9hYnN0cmFjdF9vcmlnaW4p OworCQlpZiAoYXQgPT0gTlVMTCkgeworCQkJRFdBUkZfU0VUX0VSUk9SKGRiZywgZXJyLCBEV19E TEVfTk9fRU5UUlkpOworCQkJcmV0dXJuIChEV19ETFZfTk9fRU5UUlkpOworCQl9CiAJCXN3aXRj aCAoYXQtPmF0X2Zvcm0pIHsKIAkJY2FzZSBEV19GT1JNX3JlZjE6CiAJCWNhc2UgRFdfRk9STV9y ZWYyOgo= --001a11c1c7f6f3aeb405046d01fc-- From owner-freebsd-toolchain@FreeBSD.ORG Thu Oct 2 13:18:20 2014 Return-Path: Delivered-To: freebsd-toolchain@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8D981EEC for ; Thu, 2 Oct 2014 13:18:20 +0000 (UTC) Received: from mail-qa0-f41.google.com (mail-qa0-f41.google.com [209.85.216.41]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BC35FDC for ; Thu, 2 Oct 2014 13:18:19 +0000 (UTC) Received: by mail-qa0-f41.google.com with SMTP id n8so1762015qaq.28 for ; Thu, 02 Oct 2014 06:18:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=ZslzI68vttYMig97G9LxzKcgwjvHYHt/chJL+SgOXcw=; b=jI2IqL41XBVDn4ImQB+fE1432NajuIjgl/yDGU0hsElPGmwkqErogll5CHbxj7htvs 6m94seeMZgnKDNJXfZ9sQWlMqXbiysXSDE5FJbxL7BEbIBe/0Mu9Qdihn1QcsLpvfNul rTgfGSHO0XjpaLddnckjaTBageaf/38Rcrx/gBiXBiXtf2wz/2PKbjoqjZ1tDEZ7X2Xc x9U4UbFTFku0krRHQeSe6B7W7izei0bzXFnxXBIkUOfgikpllwqd4qichBHBwOEUetO+ Utx81+N/D7Cz0uGAz7dhc1mscep5CXegY/dv9B/guACh+IOBZXBm2yHVpyQn0hiWrkXA mwkg== X-Gm-Message-State: ALoCoQmC6F2+PNI5y4LlQoqwFEdQ8q0yOXMsi7fKX4lksKrChcmS+1RXRrM+guiCDPZ5zeRCCJJg MIME-Version: 1.0 X-Received: by 10.140.106.130 with SMTP id e2mr2602364qgf.21.1412255898763; Thu, 02 Oct 2014 06:18:18 -0700 (PDT) Received: by 10.140.31.36 with HTTP; Thu, 2 Oct 2014 06:18:18 -0700 (PDT) In-Reply-To: References: Date: Thu, 2 Oct 2014 07:18:18 -0600 Message-ID: Subject: Re: elftoolchain update? From: Will Andrews To: Kai Wang Content-Type: text/plain; charset=UTF-8 Cc: Justin Gibbs , jkoshy@freebsd.org, freebsd-toolchain@freebsd.org, Dimitry Andric X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Oct 2014 13:18:20 -0000 Hi Kai, Thanks for checking in. On Thu, Oct 2, 2014 at 3:11 AM, Kai Wang wrote: > Thanks for the backtrace and analysis. > > I attached a patch for libdwarf and ctfconvert to fix the crash issue. > The libdwarf patch is the same as Will submitted, it adds check for NULL > attribute. > The ctfconvert patch fixes some issue with die_name(). We can't let > die_name() > return NULL because we need the empty string "" for type name comparison. > Instead I added checks for empty string when creating variables and > functions. I'm not sure this is the correct approach. There are many places in CTF that check for die_name() returning NULL. I believe the ones that don't should either generate the empty names themselves or explicitly check for a missing name. > However, this patch only fixes the crash issue. ctfconvert will still fail > and > complains "unresolved types" when invoked on devd (or other C++ objects) > The problem is that ctfconvert doesn't understand any C++ DWARF types, > for example: class, namespace, templates etc. Then I checked the Dtrace > CTF format: > > sys/cddl/contrib/opensolaris/uts/common/sys/ctf.h > > It seems to me that CTF can only support ANSI C ? Did anyone ever use > Dtrace with C++ program and get debugging info? You are correct, CTF only supports ANSI C. However, people have historically used it on C++ programs via pid and USDT probes. Non-POD C++ objects are not directly supported, although it is possible to access their data via pointers and pointer arithmetic. See, for example: http://www.oracle.com/technetwork/server-storage/solaris/dtrace-cc-138561.html (dated February 2005) Illumos uses the LGPL'd libdwarf, so I assume that is why it works for them to the extent that it does. I think there are probably also bugs in our port of CTF. I think that on FreeBSD, we should at least get ctfconvert to store whatever CTF data it can in C++ object files, instead of crashing or bailing out with errors. I think proper C++ support would require quite a bit of additional work in the DTrace stack, and should be considered a separate project. Thanks! --Will. From owner-freebsd-toolchain@FreeBSD.ORG Sat Oct 4 04:44:22 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1C3D99E for ; Sat, 4 Oct 2014 04:44:22 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 882DCCEF for ; Sat, 4 Oct 2014 04:44:22 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s944iMTG028364 for ; Sat, 4 Oct 2014 04:44:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Sat, 04 Oct 2014 04:44:22 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: imp@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Oct 2014 04:44:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 Garrett Cooper changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Issue Resolved |In Discussion Resolution|FIXED |--- --- Comment #8 from Garrett Cooper --- Reopening the bug because in retrospect the underlying issue has not been resolved -- it has been worked around with r268376. There are actually two bugs: 1. (Cause) rm -Rf is being run on a path and a subdirectory concurrently. 2. (Effect) rm -Rf is failing because fts_read isn't properly filtering out certain errors like EACCES, ENOENT and EPERM. imp@ resolved 2. (but there are other issues that were introduced with the commit as it ignores all errors with fts_* according to the rm(1) manpage). ian@ is looking into 1. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Sat Oct 4 05:26:38 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E602F20 for ; Sat, 4 Oct 2014 05:26:38 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 44AA52F for ; Sat, 4 Oct 2014 05:26:38 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s945Qcnt061354 for ; Sat, 4 Oct 2014 05:26:38 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Sat, 04 Oct 2014 05:26:38 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ian@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Oct 2014 05:26:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 Garrett Cooper changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|imp@FreeBSD.org |ian@FreeBSD.org -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Sat Oct 4 05:44:26 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2A3B14A1 for ; Sat, 4 Oct 2014 05:44:26 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 108D32E3 for ; Sat, 4 Oct 2014 05:44:26 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s945iPSr005421 for ; Sat, 4 Oct 2014 05:44:25 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Sat, 04 Oct 2014 05:44:25 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ngie@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ian@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: see_also Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Oct 2014 05:44:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 Garrett Cooper changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.freebsd.org/bu | |gzilla/show_bug.cgi?id=1941 | |32 -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Sat Oct 4 13:40:39 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D12F4172 for ; Sat, 4 Oct 2014 13:40:39 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B843435B for ; Sat, 4 Oct 2014 13:40:39 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s94Ded9M063448 for ; Sat, 4 Oct 2014 13:40:39 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Sat, 04 Oct 2014 13:40:39 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ian@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ian@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Oct 2014 13:40:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 --- Comment #9 from Ian Lepore --- I've been digging into the actual build system problem, and I'm starting to think that all the reported failures that contain enough of the log to be useful show that the build failed in a directory that has subdirectories. That is, one of the failures appeared to be caused by rm -rf running concurrently in usr.bin/lex and usr.bin/lex/lib. Another failure involves modules/aic7xxx and modules/aic7xxx/ahc. In another log it appeared that ata/atapci/chipsets was being deleted simulataneously with ata/atapci/chipsets/ataacard and several other subdirs under chipsets/. I didn't see any evidence that the exact same path was being multiply deleted at the same time. That is, no duplicated entries in SUBDIR lists or accidentally processing the entire sys/modules hiearchy twice in parallel somehow through two different parent paths or anything like that. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Sat Oct 4 20:56:59 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7B0DFA25 for ; Sat, 4 Oct 2014 20:56:59 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 444EE363 for ; Sat, 4 Oct 2014 20:56:59 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s94KuwPd040128 for ; Sat, 4 Oct 2014 20:56:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Sat, 04 Oct 2014 20:56:59 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: imp@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ian@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Oct 2014 20:56:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 --- Comment #10 from Warner Losh --- Ian: the problem is that directory foo has a subdir bar. So running rm -rf in both foo and foo/bar (especially when there are several bars) is what causes the issue. rm was too stupid to not generate an error when it was trying to read an entry that went away by some other agent. bde thinks this is a standards violation, but I'm not convinced by his reasoning. Adding actual debugging to rm shows that running in both is the issue, and it fails to properly ignore the ENOENT it gets when reading the subdir.. This is caused, I think, by not waiting for all subdirs to finish in clean targets before doing the current directory. Or maybe the -Rf in the first place is the bug that's masking this not waiting. I believe the -rf in question is in bsd.obj.mk where it deletes the CANONICALOBJDIR since CLEANDIRS isn't used in the sys/modules tree. It would appear that the for loop in bsd.subdir.mk that has the ${__target}: ${__subdir_targets} dependency isn't strong enough to force the recursion to finish before the current target is run. That might be a fruitful line to try to investigate. If you fix that, then the -r likely can go away if you think about it (though we'd need to add a seperate rmdir to get the same effect, which would have the added benefit of catching when the directory isn't empty due to missing CLEANFILES, since our build system has opted for the "you must list everything" approach elsewhere the -rf is a bit of a outliner in the current system). So by all means, look at the build system, but it is my belief that the fix in rm is actually standards-ly correct and fixes this bug. A build-system fix would help those systems with rm that isn't quite standards compliant. -- You are receiving this mail because: You are on the CC list for the bug. From owner-freebsd-toolchain@FreeBSD.ORG Sat Oct 4 21:23:52 2014 Return-Path: Delivered-To: freebsd-toolchain@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DE75D73 for ; Sat, 4 Oct 2014 21:23:52 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 041567FA for ; Sat, 4 Oct 2014 21:23:52 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s94LNpE0036588 for ; Sat, 4 Oct 2014 21:23:51 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-toolchain@FreeBSD.org Subject: [Bug 192490] [build] race condition with multiple instances of cleandir in subdirectories; results in failure like "rm: fts_read: No such file or directory" Date: Sat, 04 Oct 2014 21:23:52 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: conf X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: imp@FreeBSD.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ian@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Oct 2014 21:23:52 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192490 --- Comment #11 from Warner Losh --- P.S. If you want to paper over it in a different way, you could add a '-' after the @ in the rm -rf ${CANONICALOBJDIR} line. That would cause errors to just be ignored. -- You are receiving this mail because: You are on the CC list for the bug.