From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 16 13:01:11 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DD3EE247; Sun, 16 Dec 2012 13:01:11 +0000 (UTC) (envelope-from jlh@FreeBSD.org) Received: from smtp5-g21.free.fr (smtp5-g21.free.fr [IPv6:2a01:e0c:1:1599::14]) by mx1.freebsd.org (Postfix) with ESMTP id 859198FC13; Sun, 16 Dec 2012 13:01:08 +0000 (UTC) Received: from endor.tataz.chchile.org (unknown [82.233.239.98]) by smtp5-g21.free.fr (Postfix) with ESMTP id 3E061D480FD; Sun, 16 Dec 2012 14:01:02 +0100 (CET) Received: from felucia.tataz.chchile.org (felucia.tataz.chchile.org [192.168.1.9]) by endor.tataz.chchile.org (Postfix) with ESMTP id 207BE1E6; Sun, 16 Dec 2012 14:01:01 +0100 (CET) Received: by felucia.tataz.chchile.org (Postfix, from userid 1000) id F1F821005E; Sun, 16 Dec 2012 13:01:00 +0000 (UTC) Date: Sun, 16 Dec 2012 14:01:00 +0100 From: Jeremie Le Hen To: Baptiste Daroussin Subject: Re: Fix overlinking in base aka import pkgconf Message-ID: <20121216130100.GD15112@felucia.tataz.chchile.org> Mail-Followup-To: Baptiste Daroussin , Konstantin Belousov , hackers@freebsd.org References: <20121214235418.GF18884@ithaqua.etoilebsd.net> <20121215012233.GP71906@kib.kiev.ua> <20121215105643.GG18884@ithaqua.etoilebsd.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20121215105643.GG18884@ithaqua.etoilebsd.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Konstantin Belousov , hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2012 13:01:12 -0000 Hi bapt, kib, On Sat, Dec 15, 2012 at 11:56:43AM +0100, Baptiste Daroussin wrote: > On Sat, Dec 15, 2012 at 03:22:34AM +0200, Konstantin Belousov wrote: > > On Sat, Dec 15, 2012 at 12:54:19AM +0100, Baptiste Daroussin wrote: > > > Hi, > > > > > > Some of our binary are overlinked, the way we handle the linking doesn't help > > > for that. > > What do you mean there ? Do you mean that some libraries specified for the > > linking stage of the final binary are not needed for the execution ? > > I mean some library are registrer in the binary with DT_NEEDED tag where they > don't need to. > > > > > > > > > On proposition could be to use pkgconf https://github.com/pkgconf/pkgconf which > > > is BSD license pkg-config implementation 100% compatible with pkg-config. > > > > > > What I propose is to create a new PCADD variable for the Makefiles. > > > > > > PCADD will invoke pkgconf to gather the libraries and the cflags for a given > > > project. > > > > > > The second thing would be to create .pc files for all of our libraries. > > > > > > for example: > > > usr.bin/fstat dynamic build is overlinked > > And how this is better than just removing the unneeded library from > > the Makefile ? > > In that case to statically build you need -lkvm -lutil -lprocstat but if you > build dynamically you will only need -lproctast meaning you don't need to be > directly linked to libutil and libkvm. This means a breakage of libutil will > only have inpact on procstat and no more on fstat for example. I think this could be solved by an implicit linker script contained in .so and .a files, pointing to the real libraries. We have already the SHLIB_LDSCRIPT variable to accomplish this for .so files. It may be possible to do the same for .a files, though this would require renaming the real archives to something like .a.0 (here I'm just taking a similar scheme to the one used for DSO). As a matter of fact, libprocstat.a would contain: GROUP ( /usr/lib/libprocstat.a.0 /usr/lib/libkvm.a /usr/lib/libutil.a ) Note that libkvm.a and libutil.a could be linker scripts as well. Kib, do you see any problem to this proposition? -- Jeremie Le Hen Scientists say the world is made up of Protons, Neutrons and Electrons. They forgot to mention Morons. From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 16 14:03:45 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D7AFA13C; Sun, 16 Dec 2012 14:03:45 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 2D6AA8FC0C; Sun, 16 Dec 2012 14:03:45 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id qBGE3e8k094819; Sun, 16 Dec 2012 16:03:40 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.7.3 kib.kiev.ua qBGE3e8k094819 Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id qBGE3egL094818; Sun, 16 Dec 2012 16:03:40 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 16 Dec 2012 16:03:40 +0200 From: Konstantin Belousov To: Baptiste Daroussin , hackers@freebsd.org Subject: Re: Fix overlinking in base aka import pkgconf Message-ID: <20121216140340.GY71906@kib.kiev.ua> References: <20121214235418.GF18884@ithaqua.etoilebsd.net> <20121215012233.GP71906@kib.kiev.ua> <20121215105643.GG18884@ithaqua.etoilebsd.net> <20121216130100.GD15112@felucia.tataz.chchile.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="zFHfQTJdDMtDGy0L" Content-Disposition: inline In-Reply-To: <20121216130100.GD15112@felucia.tataz.chchile.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2012 14:03:45 -0000 --zFHfQTJdDMtDGy0L Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Dec 16, 2012 at 02:01:00PM +0100, Jeremie Le Hen wrote: > Hi bapt, kib, >=20 > On Sat, Dec 15, 2012 at 11:56:43AM +0100, Baptiste Daroussin wrote: > > On Sat, Dec 15, 2012 at 03:22:34AM +0200, Konstantin Belousov wrote: > > > On Sat, Dec 15, 2012 at 12:54:19AM +0100, Baptiste Daroussin wrote: > > > > Hi, > > > >=20 > > > > Some of our binary are overlinked, the way we handle the linking do= esn't help > > > > for that. > > > What do you mean there ? Do you mean that some libraries specified fo= r the > > > linking stage of the final binary are not needed for the execution ? > >=20 > > I mean some library are registrer in the binary with DT_NEEDED tag wher= e they > > don't need to. > >=20 > > >=20 > > > >=20 > > > > On proposition could be to use pkgconf https://github.com/pkgconf/p= kgconf which > > > > is BSD license pkg-config implementation 100% compatible with pkg-c= onfig. > > > >=20 > > > > What I propose is to create a new PCADD variable for the Makefiles. > > > >=20 > > > > PCADD will invoke pkgconf to gather the libraries and the cflags fo= r a given > > > > project. > > > >=20 > > > > The second thing would be to create .pc files for all of our librar= ies. > > > >=20 > > > > for example: > > > > usr.bin/fstat dynamic build is overlinked > > > And how this is better than just removing the unneeded library from > > > the Makefile ? > >=20 > > In that case to statically build you need -lkvm -lutil -lprocstat but i= f you > > build dynamically you will only need -lproctast meaning you don't need = to be > > directly linked to libutil and libkvm. This means a breakage of libutil= will > > only have inpact on procstat and no more on fstat for example. >=20 > I think this could be solved by an implicit linker script contained in > .so and .a files, pointing to the real libraries. >=20 > We have already the SHLIB_LDSCRIPT variable to accomplish this for .so > files. It may be possible to do the same for .a files, though this > would require renaming the real archives to something like .a.0 (here > I'm just taking a similar scheme to the one used for DSO). >=20 > As a matter of fact, libprocstat.a would contain: >=20 > GROUP ( /usr/lib/libprocstat.a.0 /usr/lib/libkvm.a /usr/lib/libutil.a ) >=20 > Note that libkvm.a and libutil.a could be linker scripts as well. >=20 > Kib, do you see any problem to this proposition? Wouldn't you need to completely rewrite the handling of the .a files in the share/mk ? I somewhat dislike the mere thought that .a is not an archive any longer. Does it make sense from the overhead and complexity POV, for such small goal ? --zFHfQTJdDMtDGy0L Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJQzdS7AAoJEJDCuSvBvK1BHAkP/ir7TtycOje8jd5SSdEB2T3g ZGX9AS7xhwdG70z99KNL1kjuF26XNuQg7lJlpGb0+BUU2kt84pMq1ui5zUO7TlT/ 1st2hQoovRPjiss245j3f0iFIdRvqlvIAWTJrhi+a8V5ni+7NsgFGQoDiSLSgNdE PLwetkHCulyyu4v690lFmtlv83lUvko8HJ5x2PFBIp5jFf2vcKAiP2E22pUJD9bK sEKtKAwBm541IdqyvzFFlzxr3vdZgjCXEs69TLgyIL3FyvsBnkArQYlMG0DLonc1 Sya/px2p1iLuWMQCDpaRJlsJQ6NCxfTP8NltMmh3Q0BxDOa2Xh9c98gQGq2wIPv4 e53DcC9lOu4rE2WfEsdYUXl/Zrual8sMg5+IM3akJgBFoAOta9YOqIHKR2KKzCVF nqbTGtk9db+TU2e0dnzmQ6nP12qtHJqvrqaTG125pKtZS4DFjlIuL+PjlY3l0glb VCz+M5bM77QOIfghu8efy4fl5ooldA+PV2QyvoctQ9p7oDXwpWHWhY4HI9mdaXyS 9YOCmpCZhcvMnkuOlUzK+eAy82wpCWZKnq2cofBqv0yg+QShCYUJ0rXV7BxkUHJM C3042mhDAD+GLXzWV0K9LiguAnkRzdwsiTVxhaX8RyUVy2hot6v6U8aujF53fJN1 fTYwulHNb8pnCOilj69h =4LpY -----END PGP SIGNATURE----- --zFHfQTJdDMtDGy0L-- From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 16 14:15:18 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5D4708AF; Sun, 16 Dec 2012 14:15:18 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) by mx1.freebsd.org (Postfix) with ESMTP id E6F618FC13; Sun, 16 Dec 2012 14:15:17 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 58A04358C55; Sun, 16 Dec 2012 15:15:17 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 3AF9C2848C; Sun, 16 Dec 2012 15:15:17 +0100 (CET) Date: Sun, 16 Dec 2012 15:15:17 +0100 From: Jilles Tjoelker To: Konstantin Belousov Subject: Re: Fix overlinking in base aka import pkgconf Message-ID: <20121216141517.GA33592@stack.nl> References: <20121214235418.GF18884@ithaqua.etoilebsd.net> <20121215012233.GP71906@kib.kiev.ua> <20121215105643.GG18884@ithaqua.etoilebsd.net> <20121216130100.GD15112@felucia.tataz.chchile.org> <20121216140340.GY71906@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20121216140340.GY71906@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Baptiste Daroussin , hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2012 14:15:18 -0000 On Sun, Dec 16, 2012 at 04:03:40PM +0200, Konstantin Belousov wrote: > On Sun, Dec 16, 2012 at 02:01:00PM +0100, Jeremie Le Hen wrote: > > I think this could be solved by an implicit linker script contained in > > .so and .a files, pointing to the real libraries. > > We have already the SHLIB_LDSCRIPT variable to accomplish this for .so > > files. It may be possible to do the same for .a files, though this > > would require renaming the real archives to something like .a.0 (here > > I'm just taking a similar scheme to the one used for DSO). > > As a matter of fact, libprocstat.a would contain: > > GROUP ( /usr/lib/libprocstat.a.0 /usr/lib/libkvm.a /usr/lib/libutil.a ) > > Note that libkvm.a and libutil.a could be linker scripts as well. > > Kib, do you see any problem to this proposition? > Wouldn't you need to completely rewrite the handling of the .a files > in the share/mk ? I somewhat dislike the mere thought that .a is not > an archive any longer. > Does it make sense from the overhead and complexity POV, for such small > goal ? For base this may not make much sense since there is not much need to run an old base binary against newer base libraries. Apart from that the only advantage is some form of philosophical correctness. For ports this may be a good way to keep static linking working after fixing overlinking caused by libtool's dependency_libs in .la files. -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 16 20:01:51 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6211546A for ; Sun, 16 Dec 2012 20:01:51 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 3DC718FC12 for ; Sun, 16 Dec 2012 20:01:50 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 332695081D for ; Sun, 16 Dec 2012 12:01:43 -0800 (PST) To: freebsd-hackers@freebsd.org Subject: Another WTF moment Date: Sun, 16 Dec 2012 12:01:43 -0800 Message-ID: <24507.1355688103@tristatelogic.com> From: "Ronald F. Guilmette" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2012 20:01:51 -0000 I have two Seagate ST380011A drives, both in the same single system. On that system, I boot to the FreeBSD 9.1-RC3 LiveCD. The resulting dmesg messages indicate the following regarding the two drives: ada0 at ata0 bus 0 scbus2 target 0 lun 0 ada0: ATA-6 device ada0: 100.000MB/s transfers (UDMA5, PIO 8192bytes) ada0: 76318MB (156299375 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad0 ada1 at ata0 bus 0 scbus2 target 1 lun 0 ada1: ATA-6 device ada1: 100.000MB/s transfers (UDMA5, PIO 8192bytes) ada1: 76319MB (156301488 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad1 So, um, WTF? One ST380011A is 156299375 sectors big, and the other one is 156301488 big. How exactly does this happen? From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 16 20:43:43 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B8EDCC90 for ; Sun, 16 Dec 2012 20:43:43 +0000 (UTC) (envelope-from claudiu.vasadi@gmail.com) Received: from mail-pb0-f54.google.com (mail-pb0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 87E978FC12 for ; Sun, 16 Dec 2012 20:43:43 +0000 (UTC) Received: by mail-pb0-f54.google.com with SMTP id wz12so3106031pbc.13 for ; Sun, 16 Dec 2012 12:43:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kiLd+xlFOcoVg0QCVOonNROV12uaOW2RbGBc8eh4AL0=; b=e/JB36MSN8V5y6X5Gprh7C9MLNhSEquezrXfHCbud6atKOzyPYzq0sy+HU4xVsfczj sXWYthZvNLup3ZBvLKP+Mrf656OAJyhJ5l83RPd/tbFy9J1iiqRgPqGlh2eg4yLLPXlY /b+wjyRVoqJwN/gvP1jVJ4BUISBWqcNk1hgk2J4GkFWyQpA9YGs0dxDcU+qMOX4BEvIp s4gNs9kh4SUDyLAOw4WNRfSXjZo4RxsyKG2CJvDvbrMO1BDdCdLahvO5Y5ZatYR7bLXB 5t0Mk9F3g/naWx6elRn1Cky+bDC47mi6MFxEtaW6HhsnhHyqy1WTYaqqPDdqk5sSmBZz rJMA== MIME-Version: 1.0 Received: by 10.68.227.97 with SMTP id rz1mr36444266pbc.54.1355690622984; Sun, 16 Dec 2012 12:43:42 -0800 (PST) Received: by 10.66.233.197 with HTTP; Sun, 16 Dec 2012 12:43:42 -0800 (PST) In-Reply-To: <24507.1355688103@tristatelogic.com> References: <24507.1355688103@tristatelogic.com> Date: Sun, 16 Dec 2012 21:43:42 +0100 Message-ID: Subject: Re: Another WTF moment From: claudiu vasadi To: "Ronald F. Guilmette" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2012 20:43:43 -0000 Hi, Maybe manufacturing problem? Can you try the disks on another system and see if the problem persists? -- Best regards, Claudiu Vasadi From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 16 20:54:33 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4DC7AF89 for ; Sun, 16 Dec 2012 20:54:33 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id ED3D38FC17 for ; Sun, 16 Dec 2012 20:54:24 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qBGKsG9E099503 for ; Sun, 16 Dec 2012 13:54:16 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qBGKrreo061787; Sun, 16 Dec 2012 13:53:53 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Subject: Re: Another WTF moment From: Ian Lepore To: "Ronald F. Guilmette" In-Reply-To: <24507.1355688103@tristatelogic.com> References: <24507.1355688103@tristatelogic.com> Content-Type: text/plain; charset="us-ascii" Date: Sun, 16 Dec 2012 13:53:53 -0700 Message-ID: <1355691233.1198.126.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2012 20:54:33 -0000 On Sun, 2012-12-16 at 12:01 -0800, Ronald F. Guilmette wrote: > > I have two Seagate ST380011A drives, both in the same single system. > > On that system, I boot to the FreeBSD 9.1-RC3 LiveCD. > > The resulting dmesg messages indicate the following regarding the two drives: > > ada0 at ata0 bus 0 scbus2 target 0 lun 0 > ada0: ATA-6 device > ada0: 100.000MB/s transfers (UDMA5, PIO 8192bytes) > ada0: 76318MB (156299375 512 byte sectors: 16H 63S/T 16383C) > ada0: Previously was known as ad0 > ada1 at ata0 bus 0 scbus2 target 1 lun 0 > ada1: ATA-6 device > ada1: 100.000MB/s transfers (UDMA5, PIO 8192bytes) > ada1: 76319MB (156301488 512 byte sectors: 16H 63S/T 16383C) > ada1: Previously was known as ad1 > > > So, um, WTF? One ST380011A is 156299375 sectors big, and the other one > is 156301488 big. > > How exactly does this happen? Assuming the 3.06 and 3.54 are firmware revision numbers, one might speculate that ongoing testing showed higher sector failure rates than intially expected, and thus newer firmware sets aside a few more sectors as spares. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 16 21:04:36 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A988D690 for ; Sun, 16 Dec 2012 21:04:36 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 852498FC14 for ; Sun, 16 Dec 2012 21:04:36 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 0E8F75081D for ; Sun, 16 Dec 2012 13:04:35 -0800 (PST) cc: freebsd-hackers@freebsd.org Subject: Re: NetBSD's boot select MBR In-Reply-To: <20121215175447.310780@gmx.com> Date: Sun, 16 Dec 2012 13:04:35 -0800 Message-ID: <24969.1355691875@tristatelogic.com> From: "Ronald F. Guilmette" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2012 21:04:36 -0000 In message <20121215175447.310780@gmx.com>, "Dieter BSD" wrote: >...Some sysadmins want >the MBR to be read-only for various reasons. Apparently, at least one sizeable company headquartered in Redmond, Washington does too. Can't imagine why. :-) http://www.zdnet.com/linux-foundation-uefi-secure-boot-key-for-windows-8-pcs-delays-explained-7000007841/ From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 16 21:55:03 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7BBF513E for ; Sun, 16 Dec 2012 21:55:03 +0000 (UTC) (envelope-from ap@bnc.net) Received: from mailomat.net (mailomat.net [81.20.89.254]) by mx1.freebsd.org (Postfix) with ESMTP id 07CD88FC0C for ; Sun, 16 Dec 2012 21:55:02 +0000 (UTC) X-Mailomat-SpamCatcher-Score: 2 [X] X-Mailomat-Junk-Score: 0 [] X-Mailomat-Cloudmark-Score: 0 [] Received: from [194.39.192.125] (account bnc-mail@mailrelay.mailomat.net HELO bnc.net) by mailomat.net (CommuniGate Pro SMTP 6.0c2) with ESMTPSA id 59492954; Sun, 16 Dec 2012 22:55:01 +0100 X-Junk-Score: 2 [X] X-SpamCatcher-Score: 2 [X] Received: from [192.168.200.188] (account ap@bnc.net HELO [192.168.200.188]) by bnc.net (CommuniGate Pro SMTP 6.0c1) with ESMTPSA id 6875952; Sun, 16 Dec 2012 22:56:11 +0100 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) Subject: Re: Another WTF moment From: Achim Patzner In-Reply-To: <24507.1355688103@tristatelogic.com> Date: Sun, 16 Dec 2012 22:55:00 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <8A165A1F-335D-4C5A-9CB7-6C9F5181728F@bnc.net> References: <24507.1355688103@tristatelogic.com> To: "Ronald F. Guilmette" X-Mailer: Apple Mail (2.1498) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2012 21:55:03 -0000 Am 16.12.2012 um 21:01 schrieb "Ronald F. Guilmette" = : > ada0: ATA-6 device > ada1: ATA-6 device You might consider taking a closer look at the firmware versions=85 Achim From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 16 22:08:13 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 28DBF6CB for ; Sun, 16 Dec 2012 22:08:13 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 048CA8FC12 for ; Sun, 16 Dec 2012 22:08:12 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 5B6DF50840 for ; Sun, 16 Dec 2012 14:08:12 -0800 (PST) To: freebsd-hackers@freebsd.org Subject: Re: Another WTF moment In-Reply-To: <1355691233.1198.126.camel@revolution.hippie.lan> Date: Sun, 16 Dec 2012 14:08:12 -0800 Message-ID: <25526.1355695692@tristatelogic.com> From: "Ronald F. Guilmette" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Dec 2012 22:08:13 -0000 In message <1355691233.1198.126.camel@revolution.hippie.lan>, Ian Lepore wrote: >On Sun, 2012-12-16 at 12:01 -0800, Ronald F. Guilmette wrote: >> So, um, WTF? One ST380011A is 156299375 sectors big, and the other one >> is 156301488 big. >> >> How exactly does this happen? > >Assuming the 3.06 and 3.54 are firmware revision numbers, one might >speculate that ongoing testing showed higher sector failure rates than >intially expected, and thus newer firmware sets aside a few more sectors >as spares. Well, as it turns out, no. I did some more research and managed to find out that the discrepancy is apparently due to the presence of a Host Protected Area (HPA) on the first drive. I don't know who or what created this HPA on that drive, but I do suspect that it was Windoze. (I have WindowsME and also Windows2000 partitions on that drive, and both of those OSese were installed onto that drive at various prior points in time.) So anyway, nevermind. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 17 09:49:30 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A0612B31 for ; Mon, 17 Dec 2012 09:49:30 +0000 (UTC) (envelope-from rsimmons0@gmail.com) Received: from mail-vb0-f54.google.com (mail-vb0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5103D8FC0C for ; Mon, 17 Dec 2012 09:49:30 +0000 (UTC) Received: by mail-vb0-f54.google.com with SMTP id l1so6814523vba.13 for ; Mon, 17 Dec 2012 01:49:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Xz80LdmzSpd/6+7m4aJLsvaD/3/i7wUDpCJFOMdOEPc=; b=eelZ30FRcdsDxaNhoMzmI4X7RnC68xkMRsmFI01dHbc+CZy+YSj+7FGi1vT+RSIN/U Yezwbel6xcB6x/ymk3vL0UiiUXGqy+1IBqQmKF1uYdCYGi+n2CszhMg6EBtpm1SLogN0 iQ87h8eWczKF51TY1W0k1qpyC7lH+D/EgpT/+mxNbaER6NqS+uUULf7Z1x7pshzEhOVz Af+uK3unROq5VPu+CkcQPwHlVRjWj4QkCKetzlrEOvnYD6rMLgEw6x4rmhfMa5oQPIWJ MzVI9VyEvs4Hp0oSZ/zGutY4dBpCheiOnNrduppwrSsyKW4Wxq4ixHAIv89FdoAN03mV 6a9A== MIME-Version: 1.0 Received: by 10.52.66.70 with SMTP id d6mr19354338vdt.30.1355737769397; Mon, 17 Dec 2012 01:49:29 -0800 (PST) Received: by 10.52.26.48 with HTTP; Mon, 17 Dec 2012 01:49:29 -0800 (PST) In-Reply-To: References: Date: Mon, 17 Dec 2012 04:49:29 -0500 Message-ID: Subject: Re: Building v8 on FreeBSD with clang From: Robert Simmons To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 09:49:30 -0000 On Tue, Dec 11, 2012 at 5:55 AM, Tom Evans wrote: > On Mon, Dec 10, 2012 at 6:30 PM, Robert Simmons wrote: >> I'm having a problem building google v8 on FreeBSD with clang. I get >> this error: >> >> /usr/bin/ld: final link failed: Nonrepresentable section on output >> clang++: error: linker command failed with exit code 1 (use -v to see >> invocation) >> gmake[1]: *** [/root/v8/out/x64.release/cctest] Error 1 >> gmake[1]: Leaving directory `/root/v8/out' >> gmake: *** [x64.release] Error 2 >> >> I needed to use gmake rather than make, and I added the following to use clang: >> setenv CC /usr/bin/clang >> setenv CXX /usr/bin/clang++ >> setenv GYP_DEFINES "clang=1" > > setenv LINK /usr/bin/clang++ > > (as grokked from the v8 ports makefile - there are also patches in > there to fix building on fbsd10 and adding pthread to the libraries to > link to, which may help you next...) I got past the problems above by building with GCC 4.6 from ports. V8 built just fine, and so did pyv8 the python wrapper. Now, when I try to use it in a python script, I get this error: ImportError: /usr/lib/libstdc++.so.6: version GLIBCXX_3.4.11 required by /root/.python-eggs/PyV8-1.0-py2.7-freebsd-9.1-RELEASE-amd64.egg-tmp/_PyV8.so not found How can I update the gnu libc++ to the right version that this needs? From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 17 14:07:09 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5939A3DC for ; Mon, 17 Dec 2012 14:07:09 +0000 (UTC) (envelope-from freebsd-hackers@m.gmane.org) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) by mx1.freebsd.org (Postfix) with ESMTP id 0251B8FC17 for ; Mon, 17 Dec 2012 14:07:08 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1TkbLh-0008RL-Vt for freebsd-hackers@freebsd.org; Mon, 17 Dec 2012 15:07:17 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 17 Dec 2012 15:07:17 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 17 Dec 2012 15:07:17 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-hackers@freebsd.org From: Ivan Voras Subject: Re: iSCSI vs. SMB with ZFS. Date: Mon, 17 Dec 2012 15:06:55 +0100 Lines: 57 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig62AD3874C4FF67C726A81368" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:14.0) Gecko/20120812 Thunderbird/14.0 In-Reply-To: X-Enigmail-Version: 1.4.3 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 14:07:09 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig62AD3874C4FF67C726A81368 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 12/12/2012 17:57, Zaphod Beeblebrox wrote: > The performance of the iSCSI disk is > about the same as the local disk for some operations --- faster for > some, slower for others. The workstation has 12G of memory and it's > my perception that iSCSI is heavily cached and that this enhances it's > performance. The second launch of a game ... or the second switch > into an area (ie: loading a specific piece of geometry again) is very > fast. > The performance on the SMB share is abysmal compared to the > performance on the iSCSI share. At the very least, there seems to be > little benifit to launching the same application twice --- which is > most likely windows fault. Think about what you have there: With iSCSI you have a block device, which is seen on your workstation as a disk drive, on which it creates a "local" file system (NTFS), and does *everything* like it is using a local disk drive. This includes caching, access permission calculations, file locking, etc. With a network file system (either SMB or NFS, it doesn't matter), you need to ask the server for *each* of the following situations: * to ask the server if a file has been changed so the client can use cached data (if the protocol supports it) * to ask the server if a file (or a portion of a file) has been locked by another client This basically means that for almost every single IO, you need to ask the server for something, which involves network traffic and round-trip delays. (there are smarter network protocols, and even extensions to SMB and NFS, but they are not widely used) --------------enig62AD3874C4FF67C726A81368 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAlDPJwAACgkQ/QjVBj3/HSwYPwCfQDo/0dNhAEO80b1FxsXf6ymb D80Ani4IzL/emewWeNCFV0u4etX2G8G0 =K/Pd -----END PGP SIGNATURE----- --------------enig62AD3874C4FF67C726A81368-- From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 17 16:46:32 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BDE23134; Mon, 17 Dec 2012 16:46:32 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 2E7888FC0A; Mon, 17 Dec 2012 16:46:31 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBHGkRwA001592; Mon, 17 Dec 2012 17:46:27 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBHGkRw5001589; Mon, 17 Dec 2012 17:46:27 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Mon, 17 Dec 2012 17:46:26 +0100 (CET) From: Wojciech Puchar To: Ivan Voras Subject: Re: iSCSI vs. SMB with ZFS. In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Mon, 17 Dec 2012 17:46:27 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 16:46:32 -0000 > With a network file system (either SMB or NFS, it doesn't matter), you > need to ask the server for *each* of the following situations: > * to ask the server if a file has been changed so the client can use > cached data (if the protocol supports it) > * to ask the server if a file (or a portion of a file) has been locked > by another client not really if there is only one user of file - then windows know this, but change to behaviour you described when there are more users. AND FINALLY the latter behaviour fails to work properly since windows XP (worked fine with windows 98). If you use programs that read/write share same files you may be sure data corruption would happen. you have to set locking = yes oplocks = no level2 oplocks = no to make it work properly but even more slow!. > This basically means that for almost every single IO, you need to ask > the server for something, which involves network traffic and round-trip > delays. Not that. The problem is that windows do not use all free memory for caching as with local or "local" (iSCSI) disk. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 17 19:31:06 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AC2FD49E for ; Mon, 17 Dec 2012 19:31:06 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 5154E8FC0C for ; Mon, 17 Dec 2012 19:31:05 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBHGgO6s001568; Mon, 17 Dec 2012 17:42:24 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBHGgNfV001565; Mon, 17 Dec 2012 17:42:24 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Mon, 17 Dec 2012 17:42:23 +0100 (CET) From: Wojciech Puchar To: "Ronald F. Guilmette" Subject: Re: Another WTF moment In-Reply-To: <24507.1355688103@tristatelogic.com> Message-ID: References: <24507.1355688103@tristatelogic.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Mon, 17 Dec 2012 17:42:24 +0100 (CET) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 19:31:06 -0000 > ada1: 76319MB (156301488 512 byte sectors: 16H 63S/T 16383C) > ada1: Previously was known as ad1 > > > So, um, WTF? One ST380011A is 156299375 sectors big, and the other one > is 156301488 big. > > How exactly does this happen? because this is how they report sector count. ask seagate why. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 17 20:09:51 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 60C3F50E for ; Mon, 17 Dec 2012 20:09:51 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id E6EF88FC0A for ; Mon, 17 Dec 2012 20:09:50 +0000 (UTC) Received: by mail-we0-f182.google.com with SMTP id u54so2995919wey.13 for ; Mon, 17 Dec 2012 12:09:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=o+2UZuVX+6lyuMNTvUdhTF3l+ZDfAzzI59vc8dyhnbI=; b=O6QpH4ukEKzApS4EA/2YYFOAmsVojt49tOi79ZQEDma6dioX6yWsVPngBghozH3yNu DSwQ/MBI2nWpAw8epFrTZ1ZBFAx4/p94PPweC5tO4zvNFLNeSm+Ws5xC/YLxd3xybgns IbVF+kgBMjYm274ep2gCzE8gxCZFjMLPKqzVxYsppSiCx+pdbrknj3swWIhsVUjXEtJQ dy0lNYJ1I4Mq39YMBksHQ6i5uLUUcaYjyh2+jhsX3hMA9kwwhyJi2P/HmBZ6/5S/BW1k 5zoOroes63SwWRenX+xZBnTrEFdpEfm6A9kETjUIKzkHl9P7hiUNJyE23Ee67rqDAaSt WVsQ== MIME-Version: 1.0 Received: by 10.180.88.71 with SMTP id be7mr17978647wib.17.1355774988065; Mon, 17 Dec 2012 12:09:48 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.217.57.9 with HTTP; Mon, 17 Dec 2012 12:09:47 -0800 (PST) In-Reply-To: <25526.1355695692@tristatelogic.com> References: <1355691233.1198.126.camel@revolution.hippie.lan> <25526.1355695692@tristatelogic.com> Date: Mon, 17 Dec 2012 12:09:47 -0800 X-Google-Sender-Auth: -XB5Vfx-E_2iuYZJqDv1ngUd9bw Message-ID: Subject: Re: Another WTF moment From: Adrian Chadd To: "Ronald F. Guilmette" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 20:09:51 -0000 Hi, Wow, I didn't even know this existed! Would you mind filing a doc PR with this information, along with how you figured out it was a HPA? I'm sure this is going to come up from time to time and it'd be great if someone would take the doc PR and turn it into a FAQ entry. Thanks, Adrian On 16 December 2012 14:08, Ronald F. Guilmette wrote: > > In message <1355691233.1198.126.camel@revolution.hippie.lan>, > Ian Lepore wrote: > >>On Sun, 2012-12-16 at 12:01 -0800, Ronald F. Guilmette wrote: >>> So, um, WTF? One ST380011A is 156299375 sectors big, and the other one >>> is 156301488 big. >>> >>> How exactly does this happen? >> >>Assuming the 3.06 and 3.54 are firmware revision numbers, one might >>speculate that ongoing testing showed higher sector failure rates than >>intially expected, and thus newer firmware sets aside a few more sectors >>as spares. > > Well, as it turns out, no. > > I did some more research and managed to find out that the discrepancy > is apparently due to the presence of a Host Protected Area (HPA) on the > first drive. > > I don't know who or what created this HPA on that drive, but I do suspect > that it was Windoze. (I have WindowsME and also Windows2000 partitions on > that drive, and both of those OSese were installed onto that drive at various > prior points in time.) > > So anyway, nevermind. > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 17 20:45:34 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C310A1CA; Mon, 17 Dec 2012 20:45:34 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 5AF318FC19; Mon, 17 Dec 2012 20:45:33 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAKyDz1CDaFvO/2dsb2JhbAA9CIY7uAJzgh4BAQEDAQEBASArIAsbDgoCAg0ZAikBCSYGCAcEARwEh2wGDKc6klyBIotQAgSDFYETA4hhinuCLoEcjyyDEYFGCRce X-IronPort-AV: E=Sophos;i="4.84,304,1355115600"; d="scan'208";a="5243221" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 17 Dec 2012 15:45:26 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D764EB3F15; Mon, 17 Dec 2012 15:45:26 -0500 (EST) Date: Mon, 17 Dec 2012 15:45:26 -0500 (EST) From: Rick Macklem To: Wojciech Puchar Message-ID: <1886761249.1455682.1355777126858.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: Subject: Re: iSCSI vs. SMB with ZFS. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-hackers@freebsd.org, Ivan Voras X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 20:45:34 -0000 Wojciech Puchar wrote: > > With a network file system (either SMB or NFS, it doesn't matter), > > you > > need to ask the server for *each* of the following situations: > > * to ask the server if a file has been changed so the client can > > use > > cached data (if the protocol supports it) > > * to ask the server if a file (or a portion of a file) has been > > locked > > by another client > > not really if there is only one user of file - then windows know this, > but > change to behaviour you described when there are more users. > > AND FINALLY the latter behaviour fails to work properly since windows > XP > (worked fine with windows 98). If you use programs that read/write > share > same files you may be sure data corruption would happen. > > you have to set > locking = yes > oplocks = no > level2 oplocks = no > > to make it work properly but even more slow!. > Btw, NFSv4 has delegations, which are essentially level2 oplocks. They can be enabled for a server if the volumes exported via NFSv4 are not being accessed locally (including Samba). For them to work, the nfscbd needs to be running on the client(s) and the clients must have IP addresses visible to the server for a callback TCP connection (no firewalls or NAT gateways). Even with delegations working, the client caching is limited to the buffer cache. I have an experimental patch that uses on-disk caching in the client for delegated files (I call it packrats), but it is not ready for production use. Now that I have the 4.1 client in place, I plan to get back to working on it. rick > > This basically means that for almost every single IO, you need to > > ask > > the server for something, which involves network traffic and > > round-trip > > delays. > Not that. The problem is that windows do not use all free memory for > caching as with local or "local" (iSCSI) disk. > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 17 22:00:21 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1D1936FC; Mon, 17 Dec 2012 22:00:21 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-lb0-f180.google.com (mail-lb0-f180.google.com [209.85.217.180]) by mx1.freebsd.org (Postfix) with ESMTP id 7858B8FC26; Mon, 17 Dec 2012 22:00:10 +0000 (UTC) Received: by mail-lb0-f180.google.com with SMTP id gj3so57484lbb.39 for ; Mon, 17 Dec 2012 14:00:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=rQuLQSymZGM5PcOht69XOMMWR9MVhjQszVn8U2T+Us8=; b=pRQpyhkeHY2X0CFC94pmRcNb5+ch5Ep2IEf/Ag1Jmxw4K+sNui22u/IzkmXOCspgMD MLLwcw1NVq4m8MH8c8ozeerhTtd3ezBk41Xyl0jlkRwu/dYnZf0sj80F+705N+1G+Qh/ HN6YFZAsj5W7ygnGoKLIG3aqLDnNeDbso6/Hn1Ic582sktOCBpSqx7XckDfFo89Cr2XV tkqCPE+J0OIh75atWoTx/WCPf5nbRktSNiOSnLdUui6eWjQhw/xeEDbcsJ9/9Im5bY/z ShpM1JzWsRLIs5KGatagL12B1jqzhjyOBVw1Q+AIPJprK8m+gt/9437Jh1T+BIxHPL7q iDrQ== MIME-Version: 1.0 Received: by 10.112.26.41 with SMTP id i9mr563lbg.77.1355781604139; Mon, 17 Dec 2012 14:00:04 -0800 (PST) Received: by 10.112.109.198 with HTTP; Mon, 17 Dec 2012 14:00:04 -0800 (PST) In-Reply-To: <1886761249.1455682.1355777126858.JavaMail.root@erie.cs.uoguelph.ca> References: <1886761249.1455682.1355777126858.JavaMail.root@erie.cs.uoguelph.ca> Date: Mon, 17 Dec 2012 17:00:04 -0500 Message-ID: Subject: Re: iSCSI vs. SMB with ZFS. From: Zaphod Beeblebrox To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Wojciech Puchar , Ivan Voras , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 22:00:21 -0000 Does windows 7 support nfs v4, then? Is it expected (ie: is it worthwhile trying) that nfsv4 would perform at a similar speed to iSCSI? It would seem that this at least requires active directory (or this user name mapping ... which I remember being hard). From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 17 22:22:52 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6CCCB82E; Mon, 17 Dec 2012 22:22:52 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 130828FC15; Mon, 17 Dec 2012 22:22:51 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAMmaz1CDaFvO/2dsb2JhbABFhju4AnOCHgEBAQQBAQEgKyALBRYOCgICDQUUAikBCRgBDQYIBwQBHASHcgynRZJrgSKLVnsNgg2BEwOIYYp7gi6BHI8sgxGBTzU X-IronPort-AV: E=Sophos;i="4.84,305,1355115600"; d="scan'208";a="5258745" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 17 Dec 2012 17:22:50 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id BDEF4B4023; Mon, 17 Dec 2012 17:22:50 -0500 (EST) Date: Mon, 17 Dec 2012 17:22:50 -0500 (EST) From: Rick Macklem To: Zaphod Beeblebrox Message-ID: <1469458445.1458725.1355782970762.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: Subject: Re: iSCSI vs. SMB with ZFS. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Linux)/6.0.10_GA_2692) Cc: Wojciech Puchar , Ivan Voras , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 22:22:52 -0000 Zaphod Beeblebrox wrote: > Does windows 7 support nfs v4, then? Is it expected (ie: is it > worthwhile > trying) that nfsv4 would perform at a similar speed to iSCSI? It would > seem that this at least requires active directory (or this user name > mapping ... which I remember being hard). As far as I know, there is no NFSv4 in Windows. I only made the comment (which I admit was a bit off topic), because the previous post had stated "SMB or NFS, they're the same" or something like that.) There was work on an NFSv4 client for Windows being done by CITI at the Univ. of Michigan funded by Microsoft research, but I have no idea if it was ever released. rick > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to > "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 17 23:00:19 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CA901606; Mon, 17 Dec 2012 23:00:19 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 610A18FC16; Mon, 17 Dec 2012 23:00:19 +0000 (UTC) Received: by mail-vc0-f182.google.com with SMTP id fy27so5907188vcb.13 for ; Mon, 17 Dec 2012 15:00:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=YoX1V7P7JeMUQeP1FPGr9McxkgLJYZ0DJWNo/0HcXpc=; b=Zj0P69aZ9CR1yR5VamTOu8vmlfbX7PMT1UVXxniYV3bISa+6TDcCWzj1TQeoQfHdxT ekZlGcsWH1U1bPzSuWeYEvxdBjhXFbX1K4/u5jbO6tpk61GKYiDKCQpIbMAdbMurj2rk CJb4DyIxrpKI3BRKfae8roDmX5fJUAnthkwyCObFfm3ggxC4G/U8gWiAAitYwZ6BTz4n 5J1VALUiJrVRSvTcUXEo7chx1MJA6pJLHEShzGMa3WG65wjIyaLP+UHYhUjb13/xQET0 dimO7T4AyMwTLbeTBsCoOQULYx+fun8+XbGJyufz8O1papbUkGSC5Wki2t8mae7JKjwj /dCA== MIME-Version: 1.0 Received: by 10.52.93.41 with SMTP id cr9mr21088763vdb.118.1355785218551; Mon, 17 Dec 2012 15:00:18 -0800 (PST) Received: by 10.58.218.35 with HTTP; Mon, 17 Dec 2012 15:00:18 -0800 (PST) In-Reply-To: <1469458445.1458725.1355782970762.JavaMail.root@erie.cs.uoguelph.ca> References: <1469458445.1458725.1355782970762.JavaMail.root@erie.cs.uoguelph.ca> Date: Mon, 17 Dec 2012 15:00:18 -0800 Message-ID: Subject: Re: iSCSI vs. SMB with ZFS. From: Mehmet Erol Sanliturk To: Rick Macklem Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Wojciech Puchar , Zaphod Beeblebrox , Ivan Voras , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Dec 2012 23:00:19 -0000 On Mon, Dec 17, 2012 at 2:22 PM, Rick Macklem wrote: > Zaphod Beeblebrox wrote: > > Does windows 7 support nfs v4, then? Is it expected (ie: is it > > worthwhile > > trying) that nfsv4 would perform at a similar speed to iSCSI? It would > > seem that this at least requires active directory (or this user name > > mapping ... which I remember being hard). > > As far as I know, there is no NFSv4 in Windows. I only made the comment > (which I admit was a bit off topic), because the previous post had stated > "SMB or NFS, they're the same" or something like that.) > > There was work on an NFSv4 client for Windows being done by CITI at the > Univ. of Michigan funded by Microsoft research, but I have no idea if it > was ever released. > > rick > > > > > http://www.citi.umich.edu/projects/nfsv4/ Projects: NFS Version 4 Open Source Reference Implementation We are developing an implementation of NFSv4 and NFSv4.1 for Linux http://www.citi.umich.edu/projects/nfsv4/windows/ http://www.citi.umich.edu/projects/nfsv4/windows/readme.html http://www.citi.umich.edu/projects/ Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 18 07:29:39 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 617EA410; Tue, 18 Dec 2012 07:29:39 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id AE8E38FC16; Tue, 18 Dec 2012 07:29:38 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5) with ESMTP id qBI7TXPn016542; Tue, 18 Dec 2012 08:29:34 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.5/8.14.5/Submit) with ESMTP id qBI7TXVF016539; Tue, 18 Dec 2012 08:29:33 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Tue, 18 Dec 2012 08:29:33 +0100 (CET) From: Wojciech Puchar To: Zaphod Beeblebrox Subject: Re: iSCSI vs. SMB with ZFS. In-Reply-To: Message-ID: References: <1886761249.1455682.1355777126858.JavaMail.root@erie.cs.uoguelph.ca> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="2456600518-72300972-1355815773=:16538" X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Tue, 18 Dec 2012 08:29:34 +0100 (CET) Cc: freebsd-hackers@freebsd.org, Rick Macklem , Ivan Voras X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2012 07:29:39 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --2456600518-72300972-1355815773=:16538 Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT you cannot compare file serving and block device serving. On Mon, 17 Dec 2012, Zaphod Beeblebrox wrote: > Does windows 7 support nfs v4, then?  Is it expected (ie: is it worthwhile trying) that nfsv4 would perform at a similar speed to > iSCSI?  It would seem that this at least requires active directory (or this user name mapping ... which I remember being hard). > > > --2456600518-72300972-1355815773=:16538-- From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 18 14:23:21 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 75016774; Tue, 18 Dec 2012 14:23:21 +0000 (UTC) (envelope-from fred@bws.bws.com) Received: from fipsb01.cogeco.net (smtp1.cogeco.ca [216.221.81.28]) by mx1.freebsd.org (Postfix) with ESMTP id 1A12A8FC18; Tue, 18 Dec 2012 14:23:20 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EAPF60FAYlk8q/2dsb2JhbABEviUXc4IeAQEFOhwjEAsOChYYFCUQAROIJqgPkEuMSYEWDYI/YQOIYY0okEmDEYFP X-IronPort-AV: E=Sophos;i="4.84,309,1355115600"; d="scan'208";a="365397063" Received: from d24-150-79-42.home.cgocable.net (HELO bws.bws.com) ([24.150.79.42]) by fipsb01.cogeco.net with ESMTP; 18 Dec 2012 09:22:09 -0500 Received: from bws.bws.com (localhost.bws.com [127.0.0.1]) by bws.bws.com (8.13.6/8.13.1) with ESMTP id qBIEM9YG062854; Tue, 18 Dec 2012 09:22:09 -0500 (EST) (envelope-from fred@bws.bws.com) Received: (from fred@localhost) by bws.bws.com (8.13.6/8.13.1/Submit) id qBIEM9EL062853; Tue, 18 Dec 2012 09:22:09 -0500 (EST) (envelope-from fred) Date: Tue, 18 Dec 2012 09:22:09 -0500 From: Fred Whiteside To: Rick Macklem Subject: Re: iSCSI vs. SMB with ZFS. Message-ID: <20121218142208.GA62382@bws.bws.com> References: <1469458445.1458725.1355782970762.JavaMail.root@erie.cs.uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1469458445.1458725.1355782970762.JavaMail.root@erie.cs.uoguelph.ca> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Wojciech Puchar , Zaphod Beeblebrox , Ivan Voras , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Dec 2012 14:23:21 -0000 On Mon, Dec 17, 2012 at 05:22:50PM -0500, Rick Macklem wrote: > Zaphod Beeblebrox wrote: > > Does windows 7 support nfs v4, then? Is it expected (ie: is it > > worthwhile > > trying) that nfsv4 would perform at a similar speed to iSCSI? It would > > seem that this at least requires active directory (or this user name > > mapping ... which I remember being hard). > > As far as I know, there is no NFSv4 in Windows. I only made the comment > (which I admit was a bit off topic), because the previous post had stated > "SMB or NFS, they're the same" or something like that.) > > There was work on an NFSv4 client for Windows being done by CITI at the > Univ. of Michigan funded by Microsoft research, but I have no idea if it > was ever released. There appears to be an implementation of NFSV4 {client,server} for Windows available from OpenText (via their acquisition of Hummingbird). This would not be a free product. I have no experience with their NFSV4 stuff, so have no comments on the speed... -Fred Whiteside From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 19 21:25:11 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 363A8A89; Wed, 19 Dec 2012 21:25:11 +0000 (UTC) (envelope-from rfg@tristatelogic.com) Received: from outgoing.tristatelogic.com (segfault.tristatelogic.com [69.62.255.118]) by mx1.freebsd.org (Postfix) with ESMTP id 0CCB58FC0A; Wed, 19 Dec 2012 21:25:10 +0000 (UTC) Received: from segfault-nmh-helo.tristatelogic.com (localhost [127.0.0.1]) by segfault.tristatelogic.com (Postfix) with ESMTP id 4B2F05081D; Wed, 19 Dec 2012 13:25:03 -0800 (PST) To: Adrian Chadd Subject: Re: Another WTF moment In-Reply-To: Date: Wed, 19 Dec 2012 13:25:03 -0800 Message-ID: <57462.1355952303@tristatelogic.com> From: "Ronald F. Guilmette" Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Dec 2012 21:25:11 -0000 In message Adrian Chadd wrote: >Wow, I didn't even know this existed! My apologies for not replying sooner. It has been a hectic few days. Anyway, yes, there are two kinds of "hidden" areas that may be present on a hard drive, Host Protected Area (HPA) and then separately also the Device Configuration Overlay (DCO). To be honest, I only found out about these myself a couple of months ago as I was researching the proper way of _completely_ eraseing drives prior to disposal. (Hint: Search out a program called "HDDErase". It clobbers _everything_, including even these two hiddfen areas.) >Would you mind filing a doc PR with this information, along with how >you figured out it was a HPA? Ummm... I'm not sure. I only fill out PRs when I see some specific FreeBSD document that I _know_ is either incorrect or incomplete in some way. I am not immediately aware of what specific FreeBSD document either can or should document this particular somewhat obscure aspect of modern hard disk drives. I mean it is just an obscure hardware feature, not unlike the specialized codes that might be transmitted by pressing button #7 on your 10 button "gamer" mouse. I don't think it is incumbant upon FreeBSD to document all such obscure things. As regards to how I verified my suspicion that the size discrepancy might be due to the presence of an HPA on one of the drives, the following page shows exactly how to do that. (Too bad that it assumes that you have booted to some flavor of Linux, but oh well. There are plenty of LiveCD versions of Linux out there which one can boot from, when & if one needs to check for the presence of an HPA.) http://www.forensicswiki.org/wiki/DCO_and_HPA >I'm sure this is going to come up from time to time and it'd be great >if someone would take the doc PR and turn it into a FAQ entry. Someone? Are you volunteering? Regards, rfg From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 20 07:13:38 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D1D448AB for ; Thu, 20 Dec 2012 07:13:38 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-la0-f45.google.com (mail-la0-f45.google.com [209.85.215.45]) by mx1.freebsd.org (Postfix) with ESMTP id 44BDE8FC0C for ; Thu, 20 Dec 2012 07:13:37 +0000 (UTC) Received: by mail-la0-f45.google.com with SMTP id p9so2380896laa.4 for ; Wed, 19 Dec 2012 23:13:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:from:date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=9wNkIED45gPxngVEdCkNqAyI4hMeyyeqTleJzOJ6vnQ=; b=JjBnEdM3NKofpOQdWmmCE6fRqDKf1no57M8Bkod1VIx52aJOA8bTjPl4t4z6J9Xu3B mufyskv1tkQsY+Xpyb+PKoj1wjwzr4R6SkvDw/SPH8Gg5kwsw116szSc2P8OkqQ8aERa uiSQPpbZxwzd/S+6n2joQs5PMHFwouPpoLlUk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:from:date:message-id:subject:to:cc:content-type :content-transfer-encoding:x-gm-message-state; bh=9wNkIED45gPxngVEdCkNqAyI4hMeyyeqTleJzOJ6vnQ=; b=a4TDpwGaz87i1e5FIRikWg2zDbfB8KrkgK9bnbGj/dQ5jcUcHFjeFDOE5urlmQCXA+ tvF/HCsIU+CTetn6lixKCHrp5hGe+6osIRS/G2/WDIev/xNgbOVoEJdynR9ERRwcIWx1 ffNxzOz77trd9pwAD1dDHdQrbRYPJnzYyd5ukW+dHuiPnKd+Cmjwe2TQ6Ork/TR/yean kHUop9E+5fz+imRkkyzwmXRrbkir5RLqlMzUJrRj+83pdMDwo4zuhc/UQ/a6pN1VLtFx 4ng8gIzsLHqoRI8fHCLcUWJ0C0DVQz2pTPI8noQObZOp1vSTG2PPs8440tOJg9lqc6bn r1mg== Received: by 10.112.46.199 with SMTP id x7mr3394944lbm.109.1355987611175; Wed, 19 Dec 2012 23:13:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.162.100 with HTTP; Wed, 19 Dec 2012 23:13:01 -0800 (PST) From: Eitan Adler Date: Thu, 20 Dec 2012 02:13:01 -0500 Message-ID: Subject: use after free in grep? To: Gabor Kovesdan Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Gm-Message-State: ALoCoQmjvZ08ItuJ78wKXNxLdoeoNeXsU0sPJXFuZ8Bko/0hJuU3oVW5oy7ZYH50Kz2Gk2dpjcsA Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Dec 2012 07:13:38 -0000 Hey, in xrealloc_impl 338 new_ptr =3D realloc(ptr, new_size); 339 if (new_ptr !=3D NULL) 340 { 341 hash_table_del(xmalloc_table, ptr); ^^^ isn't this a use-after-free of ptr? =C2=A77.22.3.5.2 says that ptr is deallocated after the call to realloc. 342 hash_table_add(xmalloc_table, new_ptr, (int)new_size, file, line, func); 343 } --=20 Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 20 09:34:20 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8E9BBB3B; Thu, 20 Dec 2012 09:34:20 +0000 (UTC) (envelope-from redcrash@gmail.com) Received: from mail-vc0-f175.google.com (mail-vc0-f175.google.com [209.85.220.175]) by mx1.freebsd.org (Postfix) with ESMTP id 1E48D8FC12; Thu, 20 Dec 2012 09:34:19 +0000 (UTC) Received: by mail-vc0-f175.google.com with SMTP id fy7so3435239vcb.20 for ; Thu, 20 Dec 2012 01:34:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=g2rlBNw1opwDMCUGGI/wD5k7Y2Y9kDjLxvKn6rWeFxo=; b=cJjsYnnIGn132DTVbPJZSxt8h3eE9kQAxywibBndnl5nzrT6yu45fUhZjCG+ph0TIw 7s9XjDMsvHmsSH7/pEB3jajhiKdTOWvZW/Jf9wJxVwz3yLrLtIh2e2G/wuCocs26TDLZ uNggWkWeP2LlfmvuuVlCDrSlEyrVabumd2fDQ/coooMlkFPIkkF7uT9OMEE8TSeNZpQP JlOFbehN18kw47VtkH7siTWhQ60vbHxw/i9unRdQ2qFb8aJr0BALkUAkniGTpZ4uvZPk 8VLOn31IncwIxOn68GgDl7NeQPPmTQ+xSCcHdN6NLjM+3Q6Kq6PZ9c9837zyCDf2H8O5 R3gA== MIME-Version: 1.0 Received: by 10.58.161.113 with SMTP id xr17mr13850098veb.3.1355996058993; Thu, 20 Dec 2012 01:34:18 -0800 (PST) Received: by 10.220.5.75 with HTTP; Thu, 20 Dec 2012 01:34:18 -0800 (PST) In-Reply-To: References: Date: Thu, 20 Dec 2012 10:34:18 +0100 Message-ID: Subject: Re: use after free in grep? From: Harald Servat To: Eitan Adler Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers , Gabor Kovesdan X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Dec 2012 09:34:20 -0000 Hello Eitan, ptr is not changed in realloc (although its allocated memory region is freed). Is it possible that hash_table_del only takes the value of ptr and removes such an entry from the hash table? While *ptr is not accessed, that should be fine, shouldn't it? Regards. 2012/12/20 Eitan Adler > Hey, > > in xrealloc_impl > > 338 new_ptr =3D realloc(ptr, new_size); > 339 if (new_ptr !=3D NULL) > 340 { > 341 hash_table_del(xmalloc_table, ptr); > > ^^^ isn't this a use-after-free of ptr? > > =A77.22.3.5.2 says that ptr is deallocated after the call to realloc. > > 342 hash_table_add(xmalloc_table, new_ptr, (int)new_size, file, > line, func); > 343 } > > > > -- > Eitan Adler > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " --=20 Fry: You can see how I lived before I met you. Bender: You lived before you met me?! Fry: Yeah, lots of people did. Bender: Really?! From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 20 12:19:14 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B9BCB30D; Thu, 20 Dec 2012 12:19:14 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) by mx1.freebsd.org (Postfix) with ESMTP id 6F9BC8FC0C; Thu, 20 Dec 2012 12:19:12 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7:0:d537:ea34:110c:5177] (unknown [IPv6:2001:7b8:3a7:0:d537:ea34:110c:5177]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id DD4F15C5A; Thu, 20 Dec 2012 13:19:04 +0100 (CET) Message-ID: <50D3023B.8090407@FreeBSD.org> Date: Thu, 20 Dec 2012 13:19:07 +0100 From: Dimitry Andric Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20121128 Thunderbird/18.0 MIME-Version: 1.0 To: Eitan Adler Subject: Re: use after free in grep? References: In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Hackers , Gabor Kovesdan X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Dec 2012 12:19:14 -0000 On 2012-12-20 08:13, Eitan Adler wrote: > in xrealloc_impl > > 338 new_ptr = realloc(ptr, new_size); > 339 if (new_ptr != NULL) > 340 { > 341 hash_table_del(xmalloc_table, ptr); > > ^^^ isn't this a use-after-free of ptr? Yes, realloc does not guarantee the realloc'd space will be at the same address, so it may free ptr at its discretion. Also, there is a memory leak if realloc() returns NULL. This is a very usual mistake when using realloc(). :-) Probably, the code should do the hash_table_del() before the realloc(), but I am not sure if hash_table_del() will already free ptr. From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 20 12:42:58 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A1928B2C; Thu, 20 Dec 2012 12:42:58 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (unknown [IPv6:2001:610:1108:5012::107]) by mx1.freebsd.org (Postfix) with ESMTP id 3A92C8FC0C; Thu, 20 Dec 2012 12:42:58 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 9958E1203C7; Thu, 20 Dec 2012 13:42:54 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 854B62848C; Thu, 20 Dec 2012 13:42:54 +0100 (CET) Date: Thu, 20 Dec 2012 13:42:54 +0100 From: Jilles Tjoelker To: Dimitry Andric Subject: Re: use after free in grep? Message-ID: <20121220124254.GA99616@stack.nl> References: <50D3023B.8090407@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50D3023B.8090407@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Eitan Adler , Gabor Kovesdan , FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Dec 2012 12:42:58 -0000 On Thu, Dec 20, 2012 at 01:19:07PM +0100, Dimitry Andric wrote: > On 2012-12-20 08:13, Eitan Adler wrote: > > in xrealloc_impl > > 338 new_ptr = realloc(ptr, new_size); > > 339 if (new_ptr != NULL) > > 340 { > > 341 hash_table_del(xmalloc_table, ptr); > > ^^^ isn't this a use-after-free of ptr? > Yes, realloc does not guarantee the realloc'd space will be at the same > address, so it may free ptr at its discretion. Even if you somehow know realloc() is not going to move the block, it is still wrong to use any pointer not derived from its return value to access the block. Comparing the old and the new pointers (normally or with memcmp()) does not help; it has an indeterminate result. See http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_260.htm > Also, there is a memory leak if realloc() returns NULL. This is a > very usual mistake when using realloc(). :-) No, this would be correct if a successful realloc() call did not make the old pointer indeterminate. The hash table remains unchanged if realloc() fails. > Probably, the code should do the hash_table_del() before the realloc(), > but I am not sure if hash_table_del() will already free ptr. Yes, and add it back if realloc() fails. A smarter internal interface to the hash table would avoid freeing and reallocating hash table entries here (which might fail). -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 22 03:37:59 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3C838CAA; Sat, 22 Dec 2012 03:37:59 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 11B428FC0A; Sat, 22 Dec 2012 03:37:58 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id EB4221A3C1C; Fri, 21 Dec 2012 19:37:51 -0800 (PST) Message-ID: <50D52B10.1060205@mu.org> Date: Fri, 21 Dec 2012 19:37:52 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Daniel Eischen , Jason Evans , hackers@freebsd.org Subject: malloc+utrace, tracking memory leaks in a running program. Content-Type: multipart/mixed; boundary="------------000103040406040707070503" X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Dec 2012 03:37:59 -0000 This is a multi-part message in MIME format. --------------000103040406040707070503 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hey guys. So the other day in an effort to debug a memory leak I decided to take a look at malloc+utrace(2) and decided to make a tool to debug where leaks are coming from. A few hours later I have: 1) a new version of utrace(2) (utrace2(2)) that uses structured data to prevent overloading of data. (utrace2.diff) 2) changes to ktrace and kdump to decode the new format. (also in utrace2.diff) 3) changes to jemalloc to include the new format AND the function caller so it's easy to get the source of the leaks. (also in utrace2.diff) 4) a program that can take a pipe of kdump(1) and figure out what memory has leaked. (alloctrace.py) 5) simple test program (test_utrace.c) If you want to get a trace now you can do this: gcc -Wall -O ./test_utrace.c env MALLOC_CONF='utrace:true' ktrace ./a.out kdump | ./alloctrace.py Now the problem I am having is making this work on a running program: 1) turning on the "opt_utrace" in a running program is almost impossible. This is because libc is installed stripped. Unfortunately my gdb-foo is weak and I was unable to load the symbol file without a really bad hack. The only way I could get it done was to use a trick from Ed Maste which was to: 1.1) install a debug copy of libc.so over the installed one. <- dislike! 1.2) then launching gdb ./a.out , 1.3) then set __jemalloc_opt_utrace = 1 1.4) enable ktrace on the running binary: ktrace -p -t U # this is utrace2 enabled 1.5) run 'cont' in gdb to proceed. There has to be an easier way to access the symbol __jemalloc_opt_utrace besides copying over the installed libc. Is there a workaround for 1.1? Is it time to start installing with some form of debug symbols? This would help us also with dtrace. Ideas? -Alfred --------------000103040406040707070503 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="utrace2.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="utrace2.diff" Index: contrib/jemalloc/src/jemalloc.c =================================================================== --- contrib/jemalloc/src/jemalloc.c (revision 244503) +++ contrib/jemalloc/src/jemalloc.c (working copy) @@ -78,20 +78,26 @@ static malloc_mutex_t init_lock = MALLOC_MUTEX_INITIALIZER; #endif +#ifdef JEMALLOC_UTRACE typedef struct { + int ut_type; /* utrace type UTRACE_MALLOC */ + int ut_version; /* utrace malloc version */ void *p; /* Input pointer (as in realloc(p, s)). */ size_t s; /* Request size. */ void *r; /* Result pointer. */ + void *ut_caller; /* Caller */ } malloc_utrace_t; -#ifdef JEMALLOC_UTRACE # define UTRACE(a, b, c) do { \ if (opt_utrace) { \ malloc_utrace_t ut; \ + ut.ut_type = UTRACE_MALLOC; \ + ut.ut_version = 2; \ ut.p = (a); \ ut.s = (b); \ ut.r = (c); \ - utrace(&ut, sizeof(ut)); \ + ut.ut_caller = __builtin_return_address(0); \ + utrace2(&ut, sizeof(ut)); \ } \ } while (0) #else @@ -975,7 +981,6 @@ } if (config_prof && opt_prof && result != NULL) prof_malloc(result, usize, cnt); - UTRACE(0, size, result); return (ret); } @@ -985,6 +990,7 @@ int ret = imemalign(memptr, alignment, size, sizeof(void *)); JEMALLOC_VALGRIND_MALLOC(ret == 0, *memptr, isalloc(*memptr, config_prof), false); + UTRACE(0, size, *memptr); return (ret); } @@ -1000,6 +1006,7 @@ } JEMALLOC_VALGRIND_MALLOC(err == 0, ret, isalloc(ret, config_prof), false); + UTRACE(0, size, ret); return (ret); } @@ -1265,6 +1272,7 @@ void *ret JEMALLOC_CC_SILENCE_INIT(NULL); imemalign(&ret, alignment, size, 1); JEMALLOC_VALGRIND_MALLOC(ret != NULL, ret, size, false); + UTRACE(0, size, ret); return (ret); } #endif @@ -1276,6 +1284,7 @@ void *ret JEMALLOC_CC_SILENCE_INIT(NULL); imemalign(&ret, PAGE, size, 1); JEMALLOC_VALGRIND_MALLOC(ret != NULL, ret, size, false); + UTRACE(0, size, ret); return (ret); } #endif Index: sys/compat/freebsd32/syscalls.master =================================================================== --- sys/compat/freebsd32/syscalls.master (revision 244503) +++ sys/compat/freebsd32/syscalls.master (working copy) @@ -1004,4 +1004,5 @@ int *status, int options, \ struct wrusage32 *wrusage, \ siginfo_t *info); } +533 AUE_NULL STD { int utrace2(const void *addr, size_t len); } Index: sys/kern/init_sysent.c =================================================================== --- sys/kern/init_sysent.c (revision 244503) +++ sys/kern/init_sysent.c (working copy) @@ -567,4 +567,5 @@ { AS(posix_fallocate_args), (sy_call_t *)sys_posix_fallocate, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 530 = posix_fallocate */ { AS(posix_fadvise_args), (sy_call_t *)sys_posix_fadvise, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 531 = posix_fadvise */ { AS(wait6_args), (sy_call_t *)sys_wait6, AUE_WAIT6, NULL, 0, 0, 0, SY_THR_STATIC }, /* 532 = wait6 */ + { AS(utrace2_args), (sy_call_t *)sys_utrace2, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 533 = utrace2 */ }; Index: sys/kern/kern_ktrace.c =================================================================== --- sys/kern/kern_ktrace.c (revision 244503) +++ sys/kern/kern_ktrace.c (working copy) @@ -123,6 +123,7 @@ sizeof(struct ktr_cap_fail), /* KTR_CAPFAIL */ sizeof(struct ktr_fault), /* KTR_FAULT */ sizeof(struct ktr_faultend), /* KTR_FAULTEND */ + 0, /* KTR_USER2 */ }; static STAILQ_HEAD(, ktr_request) ktr_free; @@ -1025,7 +1026,43 @@ #endif /* KTRACE */ } +/* ARGSUSED */ +int +sys_utrace2(td, uap) + struct thread *td; + register struct utrace2_args *uap; +{ + #ifdef KTRACE + struct ktr_request *req; + void *cp; + int error; + + if (!KTRPOINT(td, KTR_USER2)) + return (0); + if (uap->len > KTR_USER_MAXLEN) + return (EINVAL); + cp = malloc(uap->len, M_KTRACE, M_WAITOK); + error = copyin(uap->addr, cp, uap->len); + if (error) { + free(cp, M_KTRACE); + return (error); + } + req = ktr_getrequest(KTR_USER2); + if (req == NULL) { + free(cp, M_KTRACE); + return (ENOMEM); + } + req->ktr_buffer = cp; + req->ktr_header.ktr_len = uap->len; + ktr_submitrequest(td, req); + return (0); +#else /* !KTRACE */ + return (ENOSYS); +#endif /* KTRACE */ +} + +#ifdef KTRACE static int ktrops(td, p, ops, facs, vp) struct thread *td; Index: sys/kern/syscalls.c =================================================================== --- sys/kern/syscalls.c (revision 244503) +++ sys/kern/syscalls.c (working copy) @@ -540,4 +540,5 @@ "posix_fallocate", /* 530 = posix_fallocate */ "posix_fadvise", /* 531 = posix_fadvise */ "wait6", /* 532 = wait6 */ + "utrace2", /* 533 = utrace2 */ }; Index: sys/kern/syscalls.master =================================================================== --- sys/kern/syscalls.master (revision 244503) +++ sys/kern/syscalls.master (working copy) @@ -955,5 +955,6 @@ int *status, int options, \ struct __wrusage *wrusage, \ siginfo_t *info); } +533 AUE_NULL STD { int utrace2(const void *addr, size_t len); } ; Please copy any additions and changes to the following compatability tables: ; sys/compat/freebsd32/syscalls.master Index: sys/kern/systrace_args.c =================================================================== --- sys/kern/systrace_args.c (revision 244503) +++ sys/kern/systrace_args.c (working copy) @@ -3286,6 +3286,14 @@ *n_args = 6; break; } + /* utrace2 */ + case 533: { + struct utrace2_args *p = params; + uarg[0] = (intptr_t) p->addr; /* const void * */ + uarg[1] = p->len; /* size_t */ + *n_args = 2; + break; + } default: *n_args = 0; break; @@ -8745,6 +8753,19 @@ break; }; break; + /* utrace2 */ + case 533: + switch(ndx) { + case 0: + p = "const void *"; + break; + case 1: + p = "size_t"; + break; + default: + break; + }; + break; default: break; }; @@ -10638,6 +10659,11 @@ if (ndx == 0 || ndx == 1) p = "int"; break; + /* utrace2 */ + case 533: + if (ndx == 0 || ndx == 1) + p = "int"; + break; default: break; }; Index: sys/sys/ktrace.h =================================================================== --- sys/sys/ktrace.h (revision 244503) +++ sys/sys/ktrace.h (working copy) @@ -216,6 +216,8 @@ int result; }; +#define KTR_USER2 15 + /* * KTR_DROP - If this bit is set in ktr_type, then at least one event * between the previous record and this record was dropped. @@ -240,6 +242,7 @@ #define KTRFAC_CAPFAIL (1<ktr_type); type = unknown; @@ -1369,51 +1379,117 @@ } } -struct utrace_malloc { - void *p; - size_t s; - void *r; +struct utrace_malloc_old { + void *p; /* Input pointer (as in realloc(p, s)). */ + size_t s; /* Request size. */ + void *r; /* Result pointer. */ }; +struct utrace_malloc_v2 { + int utrace_type; /* utrace type UTRACE_MALLOC */ + int utrace_version; /* utrace malloc version */ + void *p; /* Input pointer (as in realloc(p, s)). */ + size_t s; /* Request size. */ + void *r; /* Result pointer. */ + void *caller; /* Caller */ +}; + void -ktruser_malloc(unsigned char *p) +ktruser_malloc(struct utrace_malloc_v2 *ut, int v2) { - struct utrace_malloc *ut = (struct utrace_malloc *)p; if (ut->p == (void *)(intptr_t)(-1)) - printf("malloc_init()\n"); + printf("malloc_init()"); else if (ut->s == 0) - printf("free(%p)\n", ut->p); + printf("free(%p)", ut->p); else if (ut->p == NULL) - printf("%p = malloc(%zu)\n", ut->r, ut->s); + printf("%p = malloc(%zu)", ut->r, ut->s); else - printf("%p = realloc(%p, %zu)\n", ut->r, ut->p, ut->s); + printf("%p = realloc(%p, %zu)", ut->r, ut->p, ut->s); + if (v2) + printf(", caller %p", ut->caller); + putchar('\n'); } void -ktruser(int len, unsigned char *p) +ktruser_data(size_t len, unsigned char *p) { + printf("%zd ", len); + while (len--) + if (decimal) + printf(" %d", *p++); + else + printf(" %02x", *p++); + printf("\n"); +} + +void +ktruser(int len, void *p) +{ + if (len >= 8 && bcmp(p, "RTLD", 4) == 0) { ktruser_rtld(len, p); return; } - if (len == sizeof(struct utrace_malloc)) { - ktruser_malloc(p); + if (len == sizeof(struct utrace_malloc_old)) { + struct utrace_malloc_old *utm_old; + struct utrace_malloc_v2 utm_v2; + + utm_old = p; + utm_v2.p = utm_old->p; + utm_v2.s = utm_old->s; + utm_v2.r = utm_old->r; + ktruser_malloc(&utm_v2, 0); return; } - printf("%d ", len); - while (len--) - if (decimal) - printf(" %d", *p++); - else - printf(" %02x", *p++); - printf("\n"); + ktruser_data(len, p); } void +ktruser2(size_t len, void *p) +{ + struct ktruser2_header { + int type; + int version; + } *ktru2_hd; + + if (len < sizeof(struct ktruser2_header)) { + warnx("utrace2 size too small: %zd (want: %zd)", + len, sizeof(struct ktruser2_header)); + ktruser_data(len, p); + return; + } + + ktru2_hd = p; + switch (ktru2_hd->type) { + case UTRACE_MALLOC: + if (ktru2_hd->version == 2) { + if (len == sizeof(struct utrace_malloc_v2)) { + ktruser_malloc(p, 1); + return; + } + warnx("ktruser2: UTRACE_MALLOC, version = 2, " + "size incorrect %zd (want: %zd)", + len, sizeof(struct utrace_malloc_v2)); + ktruser_data(len, p); + return; + } else { + warnx("ktruser2: UTRACE_MALLOC, unknown version = %d, " + "(want: %d)", ktru2_hd->version, 2); + ktruser_data(len, p); + return; + } + default: + break; + } + ktruser_data(len, p); + return; +} + +void ktrsockaddr(struct sockaddr *sa) { /* Index: usr.bin/ktrace/ktrace.h =================================================================== --- usr.bin/ktrace/ktrace.h (revision 244503) +++ usr.bin/ktrace/ktrace.h (working copy) @@ -32,7 +32,8 @@ #define DEF_POINTS (KTRFAC_SYSCALL | KTRFAC_SYSRET | KTRFAC_NAMEI | \ KTRFAC_GENIO | KTRFAC_PSIG | KTRFAC_USER | \ - KTRFAC_STRUCT | KTRFAC_SYSCTL | KTRFAC_CAPFAIL) + KTRFAC_STRUCT | KTRFAC_SYSCTL | KTRFAC_CAPFAIL | \ + KTRFAC_USER2) #define PROC_ABI_POINTS (KTRFAC_PROCCTOR | KTRFAC_PROCDTOR) Index: usr.bin/ktrace/subr.c =================================================================== --- usr.bin/ktrace/subr.c (revision 244503) +++ usr.bin/ktrace/subr.c (working copy) @@ -82,6 +82,9 @@ case 'u': facs |= KTRFAC_USER; break; + case 'U': + facs |= KTRFAC_USER2; + break; case 'w': facs |= KTRFAC_CSW; break; --------------000103040406040707070503 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0"; name="test_utrace.c" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="test_utrace.c" #include #include #include #define SZ 10 int main(void) { void *allocs[SZ]; int i; volatile int dummy = 0; for (i = 0; i < SZ; i++) allocs[i] = malloc(10); while (dummy) { printf("sleeping...\n"); sleep(1); } for (i = 0; i < SZ; i++) allocs[i] = realloc(allocs[i], 100); for (i = 0; i < SZ; i++) allocs[i] = realloc(allocs[i], 200); for (i = 0; i < SZ; i++) allocs[i] = realloc(allocs[i], 20); //for (i = 0; i < SZ; i++) free(allocs[i]); return 0; } --------------000103040406040707070503-- From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 22 16:46:03 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2139B253 for ; Sat, 22 Dec 2012 16:46:03 +0000 (UTC) (envelope-from emaste@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id F3D768FC12 for ; Sat, 22 Dec 2012 16:46:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id qBMGk2rH038755 for ; Sat, 22 Dec 2012 16:46:02 GMT (envelope-from emaste@freebsd.org) Received: (from emaste@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id qBMGk22d038754 for freebsd-hackers@freebsd.org; Sat, 22 Dec 2012 16:46:02 GMT (envelope-from emaste@freebsd.org) X-Authentication-Warning: freefall.freebsd.org: emaste set sender to emaste@freebsd.org using -f Date: Sat, 22 Dec 2012 16:46:02 +0000 From: Ed Maste To: freebsd-hackers@freebsd.org Subject: [PATCH] Add WITH_DEBUG_FILES knob to enable separate debug files Message-ID: <20121222164602.GB32022@sandvine.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="XsQoSWH+UP9D9v3l" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Dec 2012 16:46:03 -0000 --XsQoSWH+UP9D9v3l Content-Type: text/plain; charset=us-ascii Content-Disposition: inline When this knob is set standalone debug files for shared objects are built and installed in /usr/lib/debug/.debug. GDB searches this path for debug data. The -g flag is automatically added to CFLAGS if debug files are enabled (but the shared objects are still installed stripped, if DEBUG_FLAGS is not set). --- This is a refinement of my earlier change for shared object standalone debug. This patch also includes the following changes: - Change GDB's standalone debug file path to the default /usr/lib/debug. - Change debug file extension from 'symbols' to 'debug', in line with GDB's documentation. I initially followed the kernel build example in choosing .symbols, but .debug more accurately represents the use of these files. --XsQoSWH+UP9D9v3l Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="usr_lib_debug.diff" diff --git a/etc/mtree/BSD.usr.dist b/etc/mtree/BSD.usr.dist index 336d055..7b428d5 100644 --- a/etc/mtree/BSD.usr.dist +++ b/etc/mtree/BSD.usr.dist @@ -18,6 +18,22 @@ aout .. .. + debug + boot + .. + lib + geom + .. + .. + usr + lib + engines + .. + .. + lib32 + .. + .. + .. dtrace .. engines diff --git a/gnu/usr.bin/gdb/arch/amd64/config.h b/gnu/usr.bin/gdb/arch/amd64/config.h index ac81c54..ae3e104 100644 --- a/gnu/usr.bin/gdb/arch/amd64/config.h +++ b/gnu/usr.bin/gdb/arch/amd64/config.h @@ -440,7 +440,7 @@ #define PACKAGE "gdb" /* Global directory for separate debug files. */ -#define DEBUGDIR "/usr/local/lib/debug" +#define DEBUGDIR "/usr/lib/debug" /* Define to BFD's default architecture. */ #define DEFAULT_BFD_ARCH bfd_i386_arch diff --git a/gnu/usr.bin/gdb/arch/arm/config.h b/gnu/usr.bin/gdb/arch/arm/config.h index e1b128c..26b1891 100644 --- a/gnu/usr.bin/gdb/arch/arm/config.h +++ b/gnu/usr.bin/gdb/arch/arm/config.h @@ -452,7 +452,7 @@ #define PACKAGE "gdb" /* Global directory for separate debug files. */ -#define DEBUGDIR "/usr/local/lib/debug" +#define DEBUGDIR "/usr/lib/debug" /* Define to BFD's default architecture. */ #define DEFAULT_BFD_ARCH bfd_arm_arch diff --git a/gnu/usr.bin/gdb/arch/i386/config.h b/gnu/usr.bin/gdb/arch/i386/config.h index f21da4c..30e75c3 100644 --- a/gnu/usr.bin/gdb/arch/i386/config.h +++ b/gnu/usr.bin/gdb/arch/i386/config.h @@ -440,7 +440,7 @@ #define PACKAGE "gdb" /* Global directory for separate debug files. */ -#define DEBUGDIR "/usr/local/lib/debug" +#define DEBUGDIR "/usr/lib/debug" /* Define to BFD's default architecture. */ #define DEFAULT_BFD_ARCH bfd_i386_arch diff --git a/gnu/usr.bin/gdb/arch/ia64/config.h b/gnu/usr.bin/gdb/arch/ia64/config.h index 5faa96b..f8c84ab 100644 --- a/gnu/usr.bin/gdb/arch/ia64/config.h +++ b/gnu/usr.bin/gdb/arch/ia64/config.h @@ -440,7 +440,7 @@ #define PACKAGE "gdb" /* Global directory for separate debug files. */ -#define DEBUGDIR "/usr/local/lib/debug" +#define DEBUGDIR "/usr/lib/debug" /* Define to BFD's default architecture. */ #define DEFAULT_BFD_ARCH bfd_ia64_arch diff --git a/gnu/usr.bin/gdb/arch/mips/config.h b/gnu/usr.bin/gdb/arch/mips/config.h index 41a6731..01a7869 100644 --- a/gnu/usr.bin/gdb/arch/mips/config.h +++ b/gnu/usr.bin/gdb/arch/mips/config.h @@ -440,7 +440,7 @@ #define PACKAGE "gdb" /* Global directory for separate debug files. */ -#define DEBUGDIR "/usr/local/lib/debug" +#define DEBUGDIR "/usr/lib/debug" /* Define to BFD's default architecture. */ #define DEFAULT_BFD_ARCH bfd_mips_arch diff --git a/gnu/usr.bin/gdb/arch/powerpc/config.h b/gnu/usr.bin/gdb/arch/powerpc/config.h index f169fad..49472e7 100644 --- a/gnu/usr.bin/gdb/arch/powerpc/config.h +++ b/gnu/usr.bin/gdb/arch/powerpc/config.h @@ -440,7 +440,7 @@ #define PACKAGE "gdb" /* Global directory for separate debug files. */ -#define DEBUGDIR "/usr/local/lib/debug" +#define DEBUGDIR "/usr/lib/debug" /* Define to BFD's default architecture. */ #define DEFAULT_BFD_ARCH bfd_rs6000_arch diff --git a/gnu/usr.bin/gdb/arch/powerpc64/config.h b/gnu/usr.bin/gdb/arch/powerpc64/config.h index d8b9b6d..c904d1d 100644 --- a/gnu/usr.bin/gdb/arch/powerpc64/config.h +++ b/gnu/usr.bin/gdb/arch/powerpc64/config.h @@ -440,7 +440,7 @@ #define PACKAGE "gdb" /* Global directory for separate debug files. */ -#define DEBUGDIR "/usr/local/lib/debug" +#define DEBUGDIR "/usr/lib/debug" /* Define to BFD's default architecture. */ #define DEFAULT_BFD_ARCH bfd_rs6000_arch diff --git a/gnu/usr.bin/gdb/arch/sparc64/config.h b/gnu/usr.bin/gdb/arch/sparc64/config.h index 5527a79..ff87c28 100644 --- a/gnu/usr.bin/gdb/arch/sparc64/config.h +++ b/gnu/usr.bin/gdb/arch/sparc64/config.h @@ -440,7 +440,7 @@ #define PACKAGE "gdb" /* Global directory for separate debug files. */ -#define DEBUGDIR "/usr/local/lib/debug" +#define DEBUGDIR "/usr/lib/debug" /* Define to BFD's default architecture. */ #define DEFAULT_BFD_ARCH bfd_sparc_arch diff --git a/share/mk/bsd.lib.mk b/share/mk/bsd.lib.mk index aa3567b..a3fa175 100644 --- a/share/mk/bsd.lib.mk +++ b/share/mk/bsd.lib.mk @@ -43,6 +43,10 @@ CTFFLAGS+= -g STRIP?= -s .endif +.if defined(MK_DEBUG_FILES) && empty(${DEBUG_FLAGS:M-g}) +CFLAGS+= -g +.endif + .include # prefer .s to a .c, add .po, remove stuff not used in the BSD libraries @@ -173,8 +177,8 @@ SOLINKOPTS+= -Wl,--fatal-warnings -Wl,--warn-shared-textrel .if target(beforelinking) ${SHLIB_NAME}: beforelinking .endif -.if defined(DEBUG_FLAGS) -${SHLIB_NAME}.debug: ${SOBJS} +.if defined(MK_DEBUG_FILES) +${SHLIB_NAME}.full: ${SOBJS} .else ${SHLIB_NAME}: ${SOBJS} .endif @@ -196,13 +200,13 @@ ${SHLIB_NAME}: ${SOBJS} ${CTFMERGE} ${CTFFLAGS} -o ${.TARGET} ${SOBJS} .endif -.if defined(DEBUG_FLAGS) -${SHLIB_NAME}: ${SHLIB_NAME}.debug ${SHLIB_NAME}.symbols - ${OBJCOPY} --strip-debug --add-gnu-debuglink=${SHLIB_NAME}.symbols \ - ${SHLIB_NAME}.debug ${.TARGET} +.if defined(MK_DEBUG_FILES) +${SHLIB_NAME}: ${SHLIB_NAME}.full ${SHLIB_NAME}.debug + ${OBJCOPY} --strip-debug --add-gnu-debuglink=${SHLIB_NAME}.debug \ + ${SHLIB_NAME}.full ${.TARGET} -${SHLIB_NAME}.symbols: ${SHLIB_NAME}.debug - ${OBJCOPY} --only-keep-debug ${SHLIB_NAME}.debug ${.TARGET} +${SHLIB_NAME}.debug: ${SHLIB_NAME}.full + ${OBJCOPY} --only-keep-debug ${SHLIB_NAME}.full ${.TARGET} .endif .endif @@ -280,10 +284,10 @@ _libinstall: ${INSTALL} ${STRIP} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${_INSTALLFLAGS} ${_SHLINSTALLFLAGS} \ ${SHLIB_NAME} ${DESTDIR}${SHLIBDIR} -.if defined(DEBUG_FLAGS) +.if defined(MK_DEBUG_FILES) ${INSTALL} -o ${LIBOWN} -g ${LIBGRP} -m ${LIBMODE} \ ${_INSTALLFLAGS} \ - ${SHLIB_NAME}.symbols ${DESTDIR}${SHLIBDIR} + ${SHLIB_NAME}.debug ${DESTDIR}${DEBUGDIR}${SHLIBDIR} .endif .if defined(SHLIB_LINK) # ${_SHLIBDIRPREFIX} and ${_LDSCRIPTROOT} are both needed when cross-building diff --git a/share/mk/bsd.own.mk b/share/mk/bsd.own.mk index 78af8fb..cf83264 100644 --- a/share/mk/bsd.own.mk +++ b/share/mk/bsd.own.mk @@ -43,6 +43,9 @@ # LIBMODE Library mode. [${NOBINMODE}] # # +# DEBUGDIR Base path for standalone debug files. [/usr/lib/debug] +# +# # KMODDIR Base path for loadable kernel modules # (see kld(4)). [/boot/kernel] # @@ -147,6 +150,8 @@ LIBOWN?= ${BINOWN} LIBGRP?= ${BINGRP} LIBMODE?= ${NOBINMODE} +DEBUGDIR?= /usr/lib/debug + # Share files SHAREDIR?= /usr/share @@ -427,6 +432,7 @@ __DEFAULT_NO_OPTIONS = \ BSD_GREP \ CLANG_EXTRAS \ CTF \ + DEBUG_FILES \ HESIOD \ ICONV \ IDEA \ --XsQoSWH+UP9D9v3l-- From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 22 16:56:05 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EDB6D400 for ; Sat, 22 Dec 2012 16:56:05 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-ia0-f174.google.com (mail-ia0-f174.google.com [209.85.210.174]) by mx1.freebsd.org (Postfix) with ESMTP id AF7358FC13 for ; Sat, 22 Dec 2012 16:56:05 +0000 (UTC) Received: by mail-ia0-f174.google.com with SMTP id y25so4897655iay.33 for ; Sat, 22 Dec 2012 08:56:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=y8H/v58KsW8yh5o+5yhgojUQVV9SkF8Sed4BWe0GAEE=; b=mmRUI92RUeJPMb1ljvM1OZW9E9+La/qQrktrMoAJkc1MR6cepvH7sbJvvhNmvA6ckU 30W7S/uN1TYbTwhVfUSf/swd2qXNLsK/JIggP14HOhq+tlrvCWOpwKjf4XHUiGuQJn1H NWGVgCX1D2cL50KH32dcRH4RKxoEukY2zpqEYhK1dgFHQfQ6vplHaXmYISstIGxrRFSX ma6TH1nMu3BSa6Ssfq1kwTLrcWX0tm2LiAan2dYbp5shxlABlIEoJHh0s25Aj8yQhHdG 0VlU2r2xHT72ZqjxHdgm86OMDBU8JLIVLrPoEMiJqi5Kn/ImZxscQOWh7nPX0EIp9iYd owgQ== MIME-Version: 1.0 Received: by 10.50.169.106 with SMTP id ad10mr11601689igc.88.1356195365127; Sat, 22 Dec 2012 08:56:05 -0800 (PST) Sender: carpeddiem@gmail.com Received: by 10.50.112.134 with HTTP; Sat, 22 Dec 2012 08:56:05 -0800 (PST) In-Reply-To: <50D52B10.1060205@mu.org> References: <50D52B10.1060205@mu.org> Date: Sat, 22 Dec 2012 11:56:05 -0500 X-Google-Sender-Auth: kMVQV3NFCAHUHfCEE0s0NxNRICE Message-ID: Subject: Re: malloc+utrace, tracking memory leaks in a running program. From: Ed Maste To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Dec 2012 16:56:06 -0000 On 21 December 2012 22:37, Alfred Perlstein wrote: > Is it time to start installing with some form of debug symbols? This would > help us also with dtrace. I just posted a patch to add a knob to build and install standalone debug files. My intent is that we will build releases with this enabled, and add a base-dbg.txz distribution that contains the debug data for the base system, so that one can install it along with everything else, or add it later on when needed. We could perhaps teach dtrace to read its data from standalone .ctf files, or have it read DWARF directly and use the same debug files. From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 22 18:08:05 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 37B3535D; Sat, 22 Dec 2012 18:08:05 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 175A98FC13; Sat, 22 Dec 2012 18:08:04 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id B3D9D1A3C1A; Sat, 22 Dec 2012 10:08:03 -0800 (PST) Message-ID: <50D5F705.8020303@mu.org> Date: Sat, 22 Dec 2012 10:08:05 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Ed Maste Subject: Re: malloc+utrace, tracking memory leaks in a running program. References: <50D52B10.1060205@mu.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Rui Paulo X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Dec 2012 18:08:05 -0000 On 12/22/12 8:56 AM, Ed Maste wrote: > On 21 December 2012 22:37, Alfred Perlstein wrote: > >> Is it time to start installing with some form of debug symbols? This would >> help us also with dtrace. > I just posted a patch to add a knob to build and install standalone > debug files. My intent is that we will build releases with this > enabled, and add a base-dbg.txz distribution that contains the debug > data for the base system, so that one can install it along with > everything else, or add it later on when needed. > > We could perhaps teach dtrace to read its data from standalone .ctf > files, or have it read DWARF directly and use the same debug files. > Thank you. Added CC'd Rui Paulo. Rui, do you think it's easy to get dtrace to honor these conventions? -Alfred From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 22 18:09:54 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D85E345A; Sat, 22 Dec 2012 18:09:54 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id B99C68FC12; Sat, 22 Dec 2012 18:09:54 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id 282811A3C2A; Sat, 22 Dec 2012 10:09:54 -0800 (PST) Message-ID: <50D5F774.6070502@mu.org> Date: Sat, 22 Dec 2012 10:09:56 -0800 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Ed Maste Subject: Re: [PATCH] Add WITH_DEBUG_FILES knob to enable separate debug files References: <20121222164602.GB32022@sandvine.com> In-Reply-To: <20121222164602.GB32022@sandvine.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Dec 2012 18:09:54 -0000 On 12/22/12 8:46 AM, Ed Maste wrote: > When this knob is set standalone debug files for shared objects are > built and installed in /usr/lib/debug/.debug. GDB > searches this path for debug data. > > The -g flag is automatically added to CFLAGS if debug files are enabled > (but the shared objects are still installed stripped, if DEBUG_FLAGS is > not set). > --- > This is a refinement of my earlier change for shared object standalone > debug. This patch also includes the following changes: > > - Change GDB's standalone debug file path to the default /usr/lib/debug. > > - Change debug file extension from 'symbols' to 'debug', in line with > GDB's documentation. I initially followed the kernel build example > in choosing .symbols, but .debug more accurately represents the use > of these files. > This looks promising. After this patch, running "gdb ./a.linked.with.libc.so.out" should work without any extra thinking? I'll give it a shot. We should enable this flag by default. A big FreeBSD strength is our debugging system. -Alfred