From owner-freebsd-arch@FreeBSD.ORG Sun Mar 18 18:50:38 2012 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5DB1F106564A for ; Sun, 18 Mar 2012 18:50:38 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from theravensnest.org (theravensnest.org [109.169.23.128]) by mx1.freebsd.org (Postfix) with ESMTP id EC3338FC0A for ; Sun, 18 Mar 2012 18:50:37 +0000 (UTC) Received: from [192.168.0.2] (cpc1-cwma8-2-0-cust257.7-3.cable.virginmedia.com [82.20.153.2]) (authenticated bits=0) by theravensnest.org (8.14.4/8.14.4) with ESMTP id q2IIoZOa013285 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO) for ; Sun, 18 Mar 2012 18:50:36 GMT (envelope-from theraven@FreeBSD.org) From: David Chisnall Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Sun, 18 Mar 2012 18:50:38 +0000 Message-Id: <47F86CED-DE0E-47A0-93E8-93D35E89F9F3@FreeBSD.org> To: arch@FreeBSD.org Mime-Version: 1.0 (Apple Message framework v1257) X-Mailer: Apple Mail (2.1257) Cc: Subject: C++ ABI library switching X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Mar 2012 18:50:38 -0000 Hello the list, I am currently working on making it possible to switch C++ ABI library. = We currently ship one of these in FreeBSD 9.0 (libsupc++), and will = hopefully ship two (libsupc++ and libcxxrt) in 9.1 and one (libstdc++) = in 10. Currently, libc++ dynamically links to libcxxrt, libstdc++ statically = links to libsupc++. I would like to make us build libsupc++ as a .so = and link libstdc++.so with -Wl,-f,libsupc++.so.1. This makes it an = auxiliary filter so, if libsupc++.so.1 exists, it will be used in place = of the internal version. This preserves ABI compatibility and allows us = to switch between libcxxrt and libsupc++ with a libmap.conf entry, = making it easy for users of 9.1 to test the libstdc++ and libcxxrt = combination. With this done, it's then possible to link both libstdc++ and libc++ = into the same program, making the transition from libstdc++ to libc++ = relatively painless (everything still needs recompiling, but it doesn't = all need recompiling at the same time). For 10.0, we can then ship a = COMPAT version of libstdc++ that is linked against libcxxrt by default. Any objections / suggestions of better approaches? David= From owner-freebsd-arch@FreeBSD.ORG Sun Mar 18 23:23:34 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5B1821065673; Sun, 18 Mar 2012 23:23:34 +0000 (UTC) (envelope-from rmh.aybabtu@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0EA0C8FC12; Sun, 18 Mar 2012 23:23:33 +0000 (UTC) Received: by iahk25 with SMTP id k25so11972795iah.13 for ; Sun, 18 Mar 2012 16:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=8ISIL4RAIrU+YNVhQRo2WIXWF7yDG8r1tdp8Zv4TyRQ=; b=e2yZpT9ZFtnj8ikyBvT9Icjw5sX82XAg/ydKEHmXrhn89awGGas0judy2Lof+wGyMY 9ygeAGwYZT41ucmFRniBdPzwYxU892vJl7qRFzt4IBwpVlGYVggM/9XHrqFfhLFRa0aD 6tbJPcRZxVt54SyULp7BKG1czTdKeEcKj0hicltin7DawqvFNTK3zlG2A9KnB3uSG374 V9gMVRCArLVluWFVthFhtprYB6SLrb3SoyY1x5xIM5RA3QdmvqUJNoAFGkVZ4QE3N9Sv zom/NNmUFXna23FwNdgZioHikE3E4XQK8JcY+jDRyJHoyyo+LrNfC6smI6aKqikwN6ma gyLA== MIME-Version: 1.0 Received: by 10.50.185.228 with SMTP id ff4mr4638220igc.17.1332113013533; Sun, 18 Mar 2012 16:23:33 -0700 (PDT) Sender: rmh.aybabtu@gmail.com Received: by 10.43.130.201 with HTTP; Sun, 18 Mar 2012 16:23:33 -0700 (PDT) In-Reply-To: <20120312121852.P1122@besplex.bde.org> References: <20120312121852.P1122@besplex.bde.org> Date: Mon, 19 Mar 2012 00:23:33 +0100 X-Google-Sender-Auth: KBEs9KzAp96qR-hvEo4t9gAUAOA Message-ID: From: Robert Millan To: Bruce Evans Content-Type: multipart/mixed; boundary=14dae9340ccbf8ecd304bb8cbad3 Cc: Adrian Chadd , freebsd-arch@freebsd.org Subject: Re: [PATCH] Add compatibility X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Mar 2012 23:23:34 -0000 --14dae9340ccbf8ecd304bb8cbad3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable El 12 de mar=C3=A7 de 2012 3:18, Bruce Evans ha escr= it: > I would prefer to make it fail to build if it gets the arg order wrong, > but I don't see how to do that, since both args are integers. I don't think you can. E.g. consider: outb (0x60, 0x61); then only semantical analysis could tell. >> Looks good. =C2=A0So you suggest we tell userspace users to switch to >> bus_space_write_*? > > > The problem with that is that if you don't do the switch yourself then > most users won't even know that it is necessary. =C2=A0You have to remove > the functionaility in cpufunc.h and/or add messy userland ifdefs as > well as messy kernel ifdefs to unremove it, so that users who don't > know what they are doing and which you haven't adjusted get warned > by build failures. =C2=A0All users that knew what they were doing have to > do it differently. Okay, I see your point. Maybe we can try a conservative approach and just issue warnings. How does this look for a start? --=20 Robert Millan --14dae9340ccbf8ecd304bb8cbad3 Content-Type: text/plain; charset=US-ASCII; name="cpufunc.diff" Content-Disposition: attachment; filename="cpufunc.diff" Content-Transfer-Encoding: base64 X-Attachment-Id: f_gzylsu1x0 SW5kZXg6IHN5cy9pMzg2L2luY2x1ZGUvY3B1ZnVuYy5oCj09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9pMzg2 L2luY2x1ZGUvY3B1ZnVuYy5oCShyZXZpc2lvbiAyMzMwOTUpCisrKyBzeXMvaTM4Ni9pbmNsdWRl L2NwdWZ1bmMuaAkod29ya2luZyBjb3B5KQpAQCAtNDIsNiArNDIsMTAgQEAKICNlcnJvciB0aGlz IGZpbGUgbmVlZHMgc3lzL2NkZWZzLmggYXMgYSBwcmVyZXF1aXNpdGUKICNlbmRpZgogCisjaWZu ZGVmIF9LRVJORUwKKyN3YXJuaW5nICJObyB1c2VyLXNlcnZpY2VhYmxlIHBhcnRzIGluc2lkZS4g Rm9yIHVzZXItc3BhY2UgSS9PLCB1c2UgdGhlIGJ1c19zcGFjZSg5KSBmYW1pbHkgb2YgZnVuY3Rp b25zLiIKKyNlbmRpZgorCiAjaWZkZWYgWEVOCiBleHRlcm4gdm9pZCB4ZW5fY2xpKHZvaWQpOwog ZXh0ZXJuIHZvaWQgeGVuX3N0aSh2b2lkKTsKSW5kZXg6IHN5cy9hbWQ2NC9pbmNsdWRlL2NwdWZ1 bmMuaAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09Ci0tLSBzeXMvYW1kNjQvaW5jbHVkZS9jcHVmdW5jLmgJKHJldmlzaW9u IDIzMzA5NSkKKysrIHN5cy9hbWQ2NC9pbmNsdWRlL2NwdWZ1bmMuaAkod29ya2luZyBjb3B5KQpA QCAtNDMsNiArNDMsMTAgQEAKICNlcnJvciB0aGlzIGZpbGUgbmVlZHMgc3lzL2NkZWZzLmggYXMg YSBwcmVyZXF1aXNpdGUKICNlbmRpZgogCisjaWZuZGVmIF9LRVJORUwKKyN3YXJuaW5nICJObyB1 c2VyLXNlcnZpY2VhYmxlIHBhcnRzIGluc2lkZS4gRm9yIHVzZXItc3BhY2UgSS9PLCB1c2UgdGhl IGJ1c19zcGFjZSg5KSBmYW1pbHkgb2YgZnVuY3Rpb25zLiIKKyNlbmRpZgorCiBzdHJ1Y3QgcmVn aW9uX2Rlc2NyaXB0b3I7CiAKICNkZWZpbmUgcmVhZGIodmEpCSgqKHZvbGF0aWxlIHVpbnQ4X3Qg KikgKHZhKSkK --14dae9340ccbf8ecd304bb8cbad3-- From owner-freebsd-arch@FreeBSD.ORG Mon Mar 19 10:53:23 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5B885106564A; Mon, 19 Mar 2012 10:53:23 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail01.syd.optusnet.com.au (mail01.syd.optusnet.com.au [211.29.132.182]) by mx1.freebsd.org (Postfix) with ESMTP id CFAAA8FC1A; Mon, 19 Mar 2012 10:53:22 +0000 (UTC) Received: from c211-30-171-136.carlnfd1.nsw.optusnet.com.au (c211-30-171-136.carlnfd1.nsw.optusnet.com.au [211.30.171.136]) by mail01.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q2JArJrx011214 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 19 Mar 2012 21:53:20 +1100 Date: Mon, 19 Mar 2012 21:53:19 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Robert Millan In-Reply-To: Message-ID: <20120319204531.B1232@besplex.bde.org> References: <20120312121852.P1122@besplex.bde.org> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-147523586-1332154399=:1232" Cc: Adrian Chadd , freebsd-arch@FreeBSD.org Subject: Re: [PATCH] Add compatibility X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2012 10:53:23 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --0-147523586-1332154399=:1232 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Mon, 19 Mar 2012, Robert Millan wrote: > El 12 de mar=C3=A7 de 2012 3:18, Bruce Evans ha es= crit: >> I would prefer to make it fail to build if it gets the arg order wrong, >> but I don't see how to do that, since both args are integers. > > I don't think you can. E.g. consider: > > outb (0x60, 0x61); > > then only semantical analysis could tell. > >>> Looks good. =C2=A0So you suggest we tell userspace users to switch to >>> bus_space_write_*? >> >> >> The problem with that is that if you don't do the switch yourself then >> most users won't even know that it is necessary. =C2=A0You have to remov= e >> the functionaility in cpufunc.h and/or add messy userland ifdefs as >> well as messy kernel ifdefs to unremove it, so that users who don't >> know what they are doing and which you haven't adjusted get warned >> by build failures. =C2=A0All users that knew what they were doing have t= o >> do it differently. > > Okay, I see your point. Maybe we can try a conservative approach and > just issue warnings. How does this look for a start? % Index: sys/i386/include/cpufunc.h % =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D % --- sys/i386/include/cpufunc.h=09(revision 233095) % +++ sys/i386/include/cpufunc.h=09(working copy) % @@ -42,6 +42,10 @@ % #error this file needs sys/cdefs.h as a prerequisite % #endif %=20 % +#ifndef _KERNEL % +#warning "No user-serviceable parts inside. For user-space I/O, use the = bus_space(9) family of functions." % +#endif % + Too many style bugs for me :-) : - line too long - multiple sentences. These are unusual in error messages, and help give the previous bug - terminated message. Error messages are not terminated with a ".", except possibly if they have multiple sentences, each terminated with a ".". I don't know the exact rule for more normal use. This sentence is terminated, unlike the last one for the previous point. - Capitalized sentences. Again it is unclear what should be capitalized for multiple sentences. The one that started this point is capitalized, unlike all the others here except for the one at the end of the list. - sentence break of 1 space instead of 2. All of these style bugs are avoided in all user-anti-serviceable messages in sys/*.h. All 8 there say '#error "no user-serviceable parts inside"', except 1 says "no assembler-serviceable parts inside", and 5 misspell "serviceable" as "servicable". This phrasing is of course a joke. I know prefer the more correct "no user-serving parts inside". #warning must be used instead of #error here since we don't want to break this use completely, but there is a technical problem with this. #warning is only a gcc extension. This is handled in gratuitously different horrible ways in all places that #warning is used in sys/*.h: % dir.h: % #ifdef __CC_SUPPORTS_WARNING % #warning "The information in this file should be obtained from = " % #warning "and is provided solely (and temporarily) for backward compatibi= lity." % #endif The condition for #warning being available is given by the rather over-engineered feature test macro __CC_SUPPORTS_WARNING. To make the above correct, this file grew an otherwise-unnecessary include of just before the above. The warning message is rather formal and verbose. It is formatted nicely enough using multiple #warning statements to avoid the long lines. gcc prints the multiple #warning statements even with -Werror. It even prints multiple #error statements and doesn't abort on the first one. % syslimits.h: % #if !defined(_KERNEL) && !defined(_LIMITS_H_) && !defined(_SYS_PARAM_H_) % #ifndef _SYS_CDEFS_H_ % #error this file needs sys/cdefs.h as a prerequisite % #endif % #ifdef __CC_SUPPORTS_WARNING % #warning "No user-serviceable parts inside." % #endif % #endif Now the ifdefs are even more horrible. The #error for not including sys/cdefs.h is bogus. We are trying to trap the error of including this file directly. It must only be included by or . This has nothing to do with sys/cdefs.h, but we print a #error about the latter. The primary warning about the former is the usual cryptic "!user-serviceable" one, except it now has style bugs. This ifdef tangle is apparently the result of: - using #warning instead of #error for no reason - adding the __CC_SUPPORTS_WARNING ifdef to hide this bug. This ifdef was originally spelled __GNUC__ - when converting from the __GNUC__ spelling to __CC_SUPPORTS_WARNING, sys/cdefs.h became a prerequisite, but instead of just including it, an ifdef for its presence was added and the #error message for that - the _KERNEL ifdef is also bogus. Untangling this and fixing the message gives: % #if !defined(_LIMITS_H_) && !defined(_SYS_PARAM_H_) % #error "no user-serviceable parts inside" % #endif or the message could be decrytped to #error "this file must only be included directly by or " but I don't want to spell out the implementation details like this in all #error messages. % timeb.h: % #ifdef __GNUC__ % #warning "this file includes which is deprecated" % #endif The old spelling of __CC_SUPPORTS_WARNING here is an anachronism. The other ifdefs on #warning date from 1997 and 2002, but were changed to the new spelling in 2005. This one dates from 2010 but hasn't caught up with the 2005 change. I fixed the spelling errors long ago in my version, so I didn't notice them at first when I grepped for them while writing this. After adding a few more of these messages, I now have the following in sys/*.h: %%% _task.h:#error "no user-servicable parts inside" eventhandler.h:#error "no user-serviceable parts inside" libkern.h:#error "no user-serviceable parts inside" mutex.h:#error "no assembler-serviceable parts inside" pcpu.h:#error "no user-serviceable parts inside" pcpu.h:#error "no assembler-serviceable parts inside" smp.h:#error "no user-serviceable parts inside" smp.h:#error "no assembler-serviceable parts inside" syslimits.h:#error "no user-serviceable parts inside" taskqueue.h:#error "no user-serviceable parts inside" timetc.h:#error "no user-serviceable parts inside" %%% Kernel headers really shouldn't need these, but adding just one takes a lot of work since it tends to expose kernel headers leaking to userland with full kernel pollution like kernel mutexes and locks. The "no assembler-serviceable parts inside" are even less needed. I added them mainly in places where there were bogus LOCORE ifdefs, to detect any cases where asm files actually still include C headers (C except for a tiny part ifdefed by LOCORE). genassym mostly handles this better although it takes a few more seconds to hack on. Bruce --0-147523586-1332154399=:1232-- From owner-freebsd-arch@FreeBSD.ORG Mon Mar 19 21:00:00 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A564F1065703 for ; Mon, 19 Mar 2012 21:00:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5C6BB8FC15 for ; Mon, 19 Mar 2012 21:00:00 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id DFC4146B39; Mon, 19 Mar 2012 16:59:59 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id AB6B4B960; Mon, 19 Mar 2012 16:59:57 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Mon, 19 Mar 2012 16:59:34 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: <201203131729.q2DHTH3b066201@ambrisko.com> In-Reply-To: <201203131729.q2DHTH3b066201@ambrisko.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201203191659.34568.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 19 Mar 2012 16:59:59 -0400 (EDT) Cc: Subject: Re: rtld enhancement to add osversion, version 2 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2012 21:00:00 -0000 On Tuesday, March 13, 2012 1:29:17 pm Doug Ambrisko wrote: > This is round 2 of my rtld enhancements. The primary goal it to have > rtld look in different places for libraries depending on the legacy > binary that we want to run. This is especially a problem with binaries > linked to libraries from ports since the version of a library in > /usr/local/lib is the same whether it was FreeBSD 6, 7, 8 etc. until the > library itself changes version due to an interface change. At work we > need to run 3rd party binaries on our box of which we don't have source. > Having a FreeBSD 6 binaries load /usr/local/lib/libiconv.so.3 that was > built for FreeBSD 9 is not good. It is worse when libiconv.so.3 is > linked to libc.so.7 when the FreeBSD 6 binary needs libc.so.6. > > I solved this by having rtld extract the OSVERSION from the binary > and then use that to determine what to do. In my prior version, > I inserted that into the library directory. That meant to pull > in libc it would look at: > /lib/603000/libc.so.6 > /lib/6/libc.so.6 > /lib/libc.so.6 > to find libc.so.6. This meant a lot more look ups. Also I found it > had a problem in that if we had an ambiguous name say > /usr/local/lib/libcrypto.so.5 on the FreeBSD 6 machine and /lib/libcrypto.so.5 > on the new FreeBSD 8 system, just doing the search logic would get > a hit on /lib/libcrypto.so.5 before it got to /usr/local/lib/libcrypto.so.5. > So then it would mean we'd have to put all FreeBSD lib's in /lib/6 to > beat the search path. This wasn't a good solution. To solve the > performance and path issue, I now follow the 32 bit hints file name > model. Now it does a lookup of the hints file based on the osversion > and major. So now it looks for the hints file as: > /var/run/ld-elf-603000.so.hints > /var/run/ld-elf-6.so.hints > /var/run/ld-elf.so.hints > This is faster and has more unique paths for FreeBSD 6 libraries since > the FreeBSD 6 search paths would be in the hints file. I modified > ldconfig to accept an "-os=" option to create this hints file. > I tweaked /etc/rc* to make this easy to setup like this: > ldconfig_os_versions="6" > ldconfig_6_path="/usr/local/lib/compat/6" I think this is a definite improvement from before, thanks! > This doesn't solve the LD_LIBRARY_PATH or LD_PRELOAD. Solving that > I still insert and iterate over the osverion, major and none into the > path to find the library. The reason for this is that a FreeBSD 8 > binary could exec a FreeBSD 6 binary that execs a FreeBSD 7 binary. > If for the FreeBSD 6 binary we needed to set a custom LD_LIBRARY_PATH > and the FreeBSD 7 binary find a library via the FreeBSD 6 search path > then the FreeBSD 7 binary would die. By adding in the osversion search > path I can put the FreeBSD 6 library into the search path + the directory > 6. Then only FreeBSD 6 binaries can find it. An example: > LD_LIBRARY_PATH=/usr/custom_software/lib > /usr/custom_software/lib/6/libfoo.so.6 > then only the FreeBSD 6 binary could load it. Since the searches > would be for the FreeBSD 6 binary: > /usr/custom_software/lib/603000/libfoo.so.6 > /usr/custom_software/lib/6/libfoo.so.6 > /usr/custom_software/lib/libfoo.so.6 > and for FreeBSD 7 binary: > /usr/custom_software/lib/702000/libfoo.so.6 > /usr/custom_software/lib/7/libfoo.so.6 > /usr/custom_software/lib/libfoo.so.6 > Only the FreeBSD 6 binary would load /usr/custom_software/lib/6/libfoo.so.6. > I do the same search with LD_PRELOAD. Hmm, I'm still not quite fan of this. Perhaps you could add an extension to ldconfig and the hints file to handle this case? That is, a way to store path mappings so you could do something like: ldconfig -os=6 -p /usr/local/lib /usr/local/lib/6 Or maybe you could make it an extension of how 'm' worked, so you could make directories accept an optional set of colon-separated paths that they serve as aliases for: ldconfig -os=6 -m /usr/local/lib/6:/usr/local/lib:/usr/lib (That would even fit into your existing rc.d script changes I believe). Then rtld would keep this internal directory mapping and be able to map the '/usr/local/lib' and '/usr/lib' directories in LD_PRELOAD and LD_LIBRARY_PATH to /usr/local/lib/6. The advantage of this is the same as with your previous change that all the mappings are configurable and not hard-coded into rtld itself. > Final, is that prior binaries built on FreeBSD i386 but run on FreeBSD amd64 > that set LD_* environemnt variables would fail on FreeBSD amd64 as is > since it didn't set the equivalent LD32_*. To address this for COMPAT_32BIT > I have rtld look for LD32_* first and then check for the LD_* second. > This way legacy applications work fine. Hmm, so Yahoo! had some patches to handle this as well. I think Yahoo's patches were different though. They actually patched the 32-bit libc to capture attempts to get or set LD_* and convert them to actually get/set LD32_* instead. I'm not sure which approach is best, but it might be worth asking Peter why Yahoo! did it that way and if there were reasons for that approach vs. doing it in rtld. I think the primary reason was so that you could set LD_LIBRARY_PATH or LD_PRELOAD to reference 64-bit libraries and not have it break 32-bit apps, but if a 32-bit app tried to set a variable before launching another app it would still DTRT. I do think this is definitely a problem worth solving. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Mar 19 21:03:59 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BB42B106564A; Mon, 19 Mar 2012 21:03:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 1660D8FC14; Mon, 19 Mar 2012 21:03:58 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q2JL3tIJ064248; Mon, 19 Mar 2012 23:03:55 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q2JL3sjr029163; Mon, 19 Mar 2012 23:03:54 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q2JL3sIF029162; Mon, 19 Mar 2012 23:03:54 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 19 Mar 2012 23:03:54 +0200 From: Konstantin Belousov To: John Baldwin Message-ID: <20120319210354.GE2358@deviant.kiev.zoral.com.ua> References: <201203131729.q2DHTH3b066201@ambrisko.com> <201203191659.34568.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="orO6xySwJI16pVnm" Content-Disposition: inline In-Reply-To: <201203191659.34568.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-arch@freebsd.org Subject: Re: rtld enhancement to add osversion, version 2 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2012 21:03:59 -0000 --orO6xySwJI16pVnm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 19, 2012 at 04:59:34PM -0400, John Baldwin wrote: > On Tuesday, March 13, 2012 1:29:17 pm Doug Ambrisko wrote: > > This is round 2 of my rtld enhancements. The primary goal it to have > > rtld look in different places for libraries depending on the legacy > > binary that we want to run. This is especially a problem with binaries= =20 > > linked to libraries from ports since the version of a library in=20 > > /usr/local/lib is the same whether it was FreeBSD 6, 7, 8 etc. until th= e=20 > > library itself changes version due to an interface change. At work we= =20 > > need to run 3rd party binaries on our box of which we don't have source. > > Having a FreeBSD 6 binaries load /usr/local/lib/libiconv.so.3 that was= =20 > > built for FreeBSD 9 is not good. It is worse when libiconv.so.3 is > > linked to libc.so.7 when the FreeBSD 6 binary needs libc.so.6. > >=20 > > I solved this by having rtld extract the OSVERSION from the binary > > and then use that to determine what to do. In my prior version, > > I inserted that into the library directory. That meant to pull > > in libc it would look at: > > /lib/603000/libc.so.6 > > /lib/6/libc.so.6 > > /lib/libc.so.6 > > to find libc.so.6. This meant a lot more look ups. Also I found it > > had a problem in that if we had an ambiguous name say > > /usr/local/lib/libcrypto.so.5 on the FreeBSD 6 machine and=20 > /lib/libcrypto.so.5 > > on the new FreeBSD 8 system, just doing the search logic would get > > a hit on /lib/libcrypto.so.5 before it got to /usr/local/lib/libcrypto.= so.5. > > So then it would mean we'd have to put all FreeBSD lib's in /lib/6 to > > beat the search path. This wasn't a good solution. To solve the > > performance and path issue, I now follow the 32 bit hints file name=20 > > model. Now it does a lookup of the hints file based on the osversion > > and major. So now it looks for the hints file as: > > /var/run/ld-elf-603000.so.hints > > /var/run/ld-elf-6.so.hints > > /var/run/ld-elf.so.hints > > This is faster and has more unique paths for FreeBSD 6 libraries since > > the FreeBSD 6 search paths would be in the hints file. I modified > > ldconfig to accept an "-os=3D" option to create this hints fil= e. > > I tweaked /etc/rc* to make this easy to setup like this: > > ldconfig_os_versions=3D"6" > > ldconfig_6_path=3D"/usr/local/lib/compat/6" >=20 > I think this is a definite improvement from before, thanks! >=20 > > This doesn't solve the LD_LIBRARY_PATH or LD_PRELOAD. Solving that > > I still insert and iterate over the osverion, major and none into the > > path to find the library. The reason for this is that a FreeBSD 8=20 > > binary could exec a FreeBSD 6 binary that execs a FreeBSD 7 binary. =20 > > If for the FreeBSD 6 binary we needed to set a custom LD_LIBRARY_PATH > > and the FreeBSD 7 binary find a library via the FreeBSD 6 search path > > then the FreeBSD 7 binary would die. By adding in the osversion search > > path I can put the FreeBSD 6 library into the search path + the directo= ry > > 6. Then only FreeBSD 6 binaries can find it. An example: > > LD_LIBRARY_PATH=3D/usr/custom_software/lib > > /usr/custom_software/lib/6/libfoo.so.6 > > then only the FreeBSD 6 binary could load it. Since the searches > > would be for the FreeBSD 6 binary: > > /usr/custom_software/lib/603000/libfoo.so.6 > > /usr/custom_software/lib/6/libfoo.so.6 > > /usr/custom_software/lib/libfoo.so.6 > > and for FreeBSD 7 binary: > > /usr/custom_software/lib/702000/libfoo.so.6 > > /usr/custom_software/lib/7/libfoo.so.6 > > /usr/custom_software/lib/libfoo.so.6 > > Only the FreeBSD 6 binary would load /usr/custom_software/lib/6/libfoo.= so.6. > > I do the same search with LD_PRELOAD. >=20 > Hmm, I'm still not quite fan of this. Perhaps you could add an extension= to > ldconfig and the hints file to handle this case? That is, a way to store > path mappings so you could do something like: >=20 > ldconfig -os=3D6 -p /usr/local/lib /usr/local/lib/6 >=20 > Or maybe you could make it an extension of how 'm' worked, so you could m= ake > directories accept an optional set of colon-separated paths that they ser= ve > as aliases for: >=20 > ldconfig -os=3D6 -m /usr/local/lib/6:/usr/local/lib:/usr/lib >=20 > (That would even fit into your existing rc.d script changes I believe). = Then > rtld would keep this internal directory mapping and be able to map the > '/usr/local/lib' and '/usr/lib' directories in LD_PRELOAD and LD_LIBRARY_= PATH > to /usr/local/lib/6. The advantage of this is the same as with your prev= ious > change that all the mappings are configurable and not hard-coded into rtld > itself. >=20 > > Final, is that prior binaries built on FreeBSD i386 but run on FreeBSD = amd64 > > that set LD_* environemnt variables would fail on FreeBSD amd64 as is > > since it didn't set the equivalent LD32_*. To address this for COMPAT_= 32BIT > > I have rtld look for LD32_* first and then check for the LD_* second. > > This way legacy applications work fine. >=20 > Hmm, so Yahoo! had some patches to handle this as well. I think Yahoo's > patches were different though. They actually patched the 32-bit libc to > capture attempts to get or set LD_* and convert them to actually get/set > LD32_* instead. I'm not sure which approach is best, but it might be wor= th > asking Peter why Yahoo! did it that way and if there were reasons for that > approach vs. doing it in rtld. I think the primary reason was so that you > could set LD_LIBRARY_PATH or LD_PRELOAD to reference 64-bit libraries and > not have it break 32-bit apps, but if a 32-bit app tried to set a variable > before launching another app it would still DTRT. >=20 > I do think this is definitely a problem worth solving. Just a trivial note: r232831 added note parsing and stores osrel in the Obj_Entry. --orO6xySwJI16pVnm Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9nnzoACgkQC3+MBN1Mb4idNACePMp9ApNP3rEyZFnp5WL8POIs QOIAoNjapaNtDE0Jig9y0uK9htu9aXHm =8bnF -----END PGP SIGNATURE----- --orO6xySwJI16pVnm-- From owner-freebsd-arch@FreeBSD.ORG Tue Mar 20 17:16:47 2012 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3F731065674; Tue, 20 Mar 2012 17:16:47 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 422078FC1D; Tue, 20 Mar 2012 17:16:46 +0000 (UTC) Received: by ggnk4 with SMTP id k4so353652ggn.13 for ; Tue, 20 Mar 2012 10:16:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; bh=ZVvyUKt7btMo1FkRmNVtd+SqNmCi3aL0R6xA8mtYYNc=; b=YBNR186RG7cv0JW1JCFKrawUJpIzvuhJJVmQsEODnj6AZhAUL7Nl+CQ89GRP85lMoC PpF0SsKunze4i3k0rCHdV7FWLBthtnBzPtUGTRnyKVFtKtQBfqF3tCRMgQ13mH8snUH+ 9Ohx1h9Aea5eTugnKfsZEsVbNkFWp62lXejLLQMV0TR9JHB2nhnPnEF5xqriEokpWGz7 97Kb2Ep58KPM4jzY/jWVWzGUoU2HT2fuvSWhydj3+hnkfyOK+v4jYLH/C9JcaWbudLAn ScsBVULHYCK46WS43uhMIOAuHgjrxwQERrHypUwcaT85FES7CGboY6ZSrMcFeOZLdlSn Bn7Q== Received: by 10.224.208.130 with SMTP id gc2mr1826847qab.73.1332263800253; Tue, 20 Mar 2012 10:16:40 -0700 (PDT) Received: from kan.dyndns.org (c-24-63-226-98.hsd1.ma.comcast.net. [24.63.226.98]) by mx.google.com with ESMTPS id j17sm3610531qaj.9.2012.03.20.10.16.38 (version=SSLv3 cipher=OTHER); Tue, 20 Mar 2012 10:16:38 -0700 (PDT) Date: Tue, 20 Mar 2012 13:16:23 -0400 From: Alexander Kabaev To: David Chisnall Message-ID: <20120320131623.4264c11e@kan.dyndns.org> In-Reply-To: <47F86CED-DE0E-47A0-93E8-93D35E89F9F3@FreeBSD.org> References: <47F86CED-DE0E-47A0-93E8-93D35E89F9F3@FreeBSD.org> X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.6; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/s4Wk/AWoRN8gJ11nWofT9m="; protocol="application/pgp-signature" Cc: arch@FreeBSD.org Subject: Re: C++ ABI library switching X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Mar 2012 17:16:47 -0000 --Sig_/s4Wk/AWoRN8gJ11nWofT9m= Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Sun, 18 Mar 2012 18:50:38 +0000 David Chisnall wrote: > Hello the list, >=20 > I am currently working on making it possible to switch C++ ABI > library. We currently ship one of these in FreeBSD 9.0 (libsupc++), > and will hopefully ship two (libsupc++ and libcxxrt) in 9.1 and one > (libstdc++) in 10. >=20 > Currently, libc++ dynamically links to libcxxrt, libstdc++ statically > links to libsupc++. I would like to make us build libsupc++ as a .so > and link libstdc++.so with -Wl,-f,libsupc++.so.1. This makes it an > auxiliary filter so, if libsupc++.so.1 exists, it will be used in > place of the internal version. This preserves ABI compatibility and > allows us to switch between libcxxrt and libsupc++ with a libmap.conf > entry, making it easy for users of 9.1 to test the libstdc++ and > libcxxrt combination. >=20 > With this done, it's then possible to link both libstdc++ and libc++ > into the same program, making the transition from libstdc++ to libc++ > relatively painless (everything still needs recompiling, but it > doesn't all need recompiling at the same time). For 10.0, we can > then ship a COMPAT version of libstdc++ that is linked against > libcxxrt by default. >=20 > Any objections / suggestions of better approaches? >=20 > David_______________________________________________ We did discuss this approach though other channels and use of the filter was selected specifically because it is less intrusive of all considered alternatives. Unless someone else raises objections soon, I'd say go ahead. --=20 Alexander Kabaev --Sig_/s4Wk/AWoRN8gJ11nWofT9m= Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iD8DBQFPaLtyQ6z1jMm+XZYRAu50AKC/Yr5ivdkk53jFxi0BCTTxUZ+FzwCfa5G+ lx2UZ4Vxqvmu32K0TEf4vFk= =3zMa -----END PGP SIGNATURE----- --Sig_/s4Wk/AWoRN8gJ11nWofT9m=-- From owner-freebsd-arch@FreeBSD.ORG Wed Mar 21 12:03:47 2012 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6210C1065741 for ; Wed, 21 Mar 2012 12:03:47 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 943248FC0A for ; Wed, 21 Mar 2012 12:03:46 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA20613 for ; Wed, 21 Mar 2012 14:03:45 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <4F69C3A0.5080008@FreeBSD.org> Date: Wed, 21 Mar 2012 14:03:44 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.2) Gecko/20120221 Thunderbird/10.0.2 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org X-Enigmail-Version: 1.3.5 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: "stop scheduler on panic" for stable/[98] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 12:03:47 -0000 I feel that I can not assess a balance between risks and benefits of merging the "stop scheduler on panic" changes to stable/9 and stable/8 (after 8.3 release). What is your impression/opinion/assessment? Thank you! -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Wed Mar 21 16:17:35 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E7248106564A; Wed, 21 Mar 2012 16:17:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id BBB288FC0A; Wed, 21 Mar 2012 16:17:35 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 5D0F946B43; Wed, 21 Mar 2012 12:17:35 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E5532B94B; Wed, 21 Mar 2012 12:17:34 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Date: Wed, 21 Mar 2012 09:38:27 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: <4F69C3A0.5080008@FreeBSD.org> In-Reply-To: <4F69C3A0.5080008@FreeBSD.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201203210938.27193.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 21 Mar 2012 12:17:35 -0400 (EDT) Cc: Andriy Gapon Subject: Re: "stop scheduler on panic" for stable/[98] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 16:17:36 -0000 On Wednesday, March 21, 2012 8:03:44 am Andriy Gapon wrote: > > I feel that I can not assess a balance between risks and benefits of merging the > "stop scheduler on panic" changes to stable/9 and stable/8 (after 8.3 release). > What is your impression/opinion/assessment? We have merged it to 8 recently at work because crash dumps do not work for us at all on 8 currently. We don't have panics very often however, so it will be a while before I will know if this really helped. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Mar 21 17:11:43 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9C2F106564A; Wed, 21 Mar 2012 17:11:43 +0000 (UTC) (envelope-from ambrisko@ambrisko.com) Received: from mail.ambrisko.com (mail.ambrisko.com [70.91.206.90]) by mx1.freebsd.org (Postfix) with ESMTP id 8EB2B8FC16; Wed, 21 Mar 2012 17:11:43 +0000 (UTC) X-Ambrisko-Me: Yes Received: from server2.ambrisko.com (HELO internal.ambrisko.com) ([192.168.1.2]) by ironport.ambrisko.com with ESMTP; 21 Mar 2012 10:11:43 -0700 Received: from ambrisko.com (localhost [127.0.0.1]) by internal.ambrisko.com (8.14.4/8.14.4) with ESMTP id q2LHBbVf026328; Wed, 21 Mar 2012 10:11:37 -0700 (PDT) (envelope-from ambrisko@ambrisko.com) Received: (from ambrisko@localhost) by ambrisko.com (8.14.4/8.14.4/Submit) id q2LHBbO2026327; Wed, 21 Mar 2012 10:11:37 -0700 (PDT) (envelope-from ambrisko) From: Doug Ambrisko Message-Id: <201203211711.q2LHBbO2026327@ambrisko.com> In-Reply-To: <201203191659.34568.jhb@freebsd.org> To: John Baldwin Date: Wed, 21 Mar 2012 10:11:37 -0700 (PDT) X-Mailer: ELM [version 2.4ME+ PL124d (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="US-ASCII" Cc: freebsd-arch@freebsd.org Subject: Re: rtld enhancement to add osversion, version 2 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 17:11:43 -0000 John Baldwin writes: | On Tuesday, March 13, 2012 1:29:17 pm Doug Ambrisko wrote: | > This is round 2 of my rtld enhancements. The primary goal it to have | > rtld look in different places for libraries depending on the legacy | > binary that we want to run. This is especially a problem with binaries | > linked to libraries from ports since the version of a library in | > /usr/local/lib is the same whether it was FreeBSD 6, 7, 8 etc. until the | > library itself changes version due to an interface change. At work we | > need to run 3rd party binaries on our box of which we don't have source. | > Having a FreeBSD 6 binaries load /usr/local/lib/libiconv.so.3 that was | > built for FreeBSD 9 is not good. It is worse when libiconv.so.3 is | > linked to libc.so.7 when the FreeBSD 6 binary needs libc.so.6. | > | > I solved this by having rtld extract the OSVERSION from the binary | > and then use that to determine what to do. In my prior version, | > I inserted that into the library directory. That meant to pull | > in libc it would look at: | > /lib/603000/libc.so.6 | > /lib/6/libc.so.6 | > /lib/libc.so.6 | > to find libc.so.6. This meant a lot more look ups. Also I found it | > had a problem in that if we had an ambiguous name say | > /usr/local/lib/libcrypto.so.5 on the FreeBSD 6 machine and | /lib/libcrypto.so.5 | > on the new FreeBSD 8 system, just doing the search logic would get | > a hit on /lib/libcrypto.so.5 before it got to /usr/local/lib/libcrypto.so.5. | > So then it would mean we'd have to put all FreeBSD lib's in /lib/6 to | > beat the search path. This wasn't a good solution. To solve the | > performance and path issue, I now follow the 32 bit hints file name | > model. Now it does a lookup of the hints file based on the osversion | > and major. So now it looks for the hints file as: | > /var/run/ld-elf-603000.so.hints | > /var/run/ld-elf-6.so.hints | > /var/run/ld-elf.so.hints | > This is faster and has more unique paths for FreeBSD 6 libraries since | > the FreeBSD 6 search paths would be in the hints file. I modified | > ldconfig to accept an "-os=" option to create this hints file. | > I tweaked /etc/rc* to make this easy to setup like this: | > ldconfig_os_versions="6" | > ldconfig_6_path="/usr/local/lib/compat/6" | | I think this is a definite improvement from before, thanks! | | > This doesn't solve the LD_LIBRARY_PATH or LD_PRELOAD. Solving that | > I still insert and iterate over the osverion, major and none into the | > path to find the library. The reason for this is that a FreeBSD 8 | > binary could exec a FreeBSD 6 binary that execs a FreeBSD 7 binary. | > If for the FreeBSD 6 binary we needed to set a custom LD_LIBRARY_PATH | > and the FreeBSD 7 binary find a library via the FreeBSD 6 search path | > then the FreeBSD 7 binary would die. By adding in the osversion search | > path I can put the FreeBSD 6 library into the search path + the directory | > 6. Then only FreeBSD 6 binaries can find it. An example: | > LD_LIBRARY_PATH=/usr/custom_software/lib | > /usr/custom_software/lib/6/libfoo.so.6 | > then only the FreeBSD 6 binary could load it. Since the searches | > would be for the FreeBSD 6 binary: | > /usr/custom_software/lib/603000/libfoo.so.6 | > /usr/custom_software/lib/6/libfoo.so.6 | > /usr/custom_software/lib/libfoo.so.6 | > and for FreeBSD 7 binary: | > /usr/custom_software/lib/702000/libfoo.so.6 | > /usr/custom_software/lib/7/libfoo.so.6 | > /usr/custom_software/lib/libfoo.so.6 | > Only the FreeBSD 6 binary would load /usr/custom_software/lib/6/libfoo.so.6. | > I do the same search with LD_PRELOAD. | | Hmm, I'm still not quite fan of this. Perhaps you could add an extension to | ldconfig and the hints file to handle this case? That is, a way to store | path mappings so you could do something like: | | ldconfig -os=6 -p /usr/local/lib /usr/local/lib/6 I'll have to look to see how the hints file could update rtld. It is an interesting idea. Maybe libmap.conf would be better place for this. I haven't looked at how libmap works. Maybe introduce: /etc/libmap-.conf that maps paths as well. So with the above example. /etc/libmap-6.conf would contain /usr/custom_software/lib /usr/custom_software/lib/6:/usr/custom_software/lib for example. | Or maybe you could make it an extension of how 'm' worked, so you could make | directories accept an optional set of colon-separated paths that they serve | as aliases for: | | ldconfig -os=6 -m /usr/local/lib/6:/usr/local/lib:/usr/lib I don't really get how this is solving the LD_LIBRARY_PATH/LD_PRELOAD since "-m" is solving the general case with the hints file which I did first. | (That would even fit into your existing rc.d script changes I believe). Then | rtld would keep this internal directory mapping and be able to map the | '/usr/local/lib' and '/usr/lib' directories in LD_PRELOAD and LD_LIBRARY_PATH | to /usr/local/lib/6. The advantage of this is the same as with your previous | change that all the mappings are configurable and not hard-coded into rtld | itself. | > Final, is that prior binaries built on FreeBSD i386 but run on FreeBSD amd64 | > that set LD_* environemnt variables would fail on FreeBSD amd64 as is | > since it didn't set the equivalent LD32_*. To address this for COMPAT_32BIT | > I have rtld look for LD32_* first and then check for the LD_* second. | > This way legacy applications work fine. | | Hmm, so Yahoo! had some patches to handle this as well. I think Yahoo's | patches were different though. They actually patched the 32-bit libc to | capture attempts to get or set LD_* and convert them to actually get/set | LD32_* instead. I'm not sure which approach is best, but it might be worth | asking Peter why Yahoo! did it that way and if there were reasons for that | approach vs. doing it in rtld. I think the primary reason was so that you | could set LD_LIBRARY_PATH or LD_PRELOAD to reference 64-bit libraries and | not have it break 32-bit apps, but if a 32-bit app tried to set a variable | before launching another app it would still DTRT. This means you would have to have a modified libc in these environment and it wouldn't help in the static binary case. I think we are safe in the case LD_ for a 32 bit binary effecting a 64 bit since rtld should not allow loading a the wrong type of lib. into a binary. I can do some more testing around that area. I was trying to keep all changes in the host environment so we can run unchanged binaries from other vendors. | I do think this is definitely a problem worth solving. Thanks, Doug A. From owner-freebsd-arch@FreeBSD.ORG Wed Mar 21 20:59:10 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F427106564A for ; Wed, 21 Mar 2012 20:59:10 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0D81D8FC0C for ; Wed, 21 Mar 2012 20:59:10 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 43FFB46B1A; Wed, 21 Mar 2012 16:59:03 -0400 (EDT) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id CA18DB940; Wed, 21 Mar 2012 16:59:02 -0400 (EDT) From: John Baldwin To: Doug Ambrisko Date: Wed, 21 Mar 2012 16:55:46 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p10; KDE/4.5.5; amd64; ; ) References: <201203211711.q2LHBbO2026327@ambrisko.com> In-Reply-To: <201203211711.q2LHBbO2026327@ambrisko.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201203211655.46443.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 21 Mar 2012 16:59:02 -0400 (EDT) Cc: freebsd-arch@freebsd.org Subject: Re: rtld enhancement to add osversion, version 2 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Mar 2012 20:59:10 -0000 On Wednesday, March 21, 2012 1:11:37 pm Doug Ambrisko wrote: > | > This doesn't solve the LD_LIBRARY_PATH or LD_PRELOAD. Solving that > | > I still insert and iterate over the osverion, major and none into the > | > path to find the library. The reason for this is that a FreeBSD 8 > | > binary could exec a FreeBSD 6 binary that execs a FreeBSD 7 binary. > | > If for the FreeBSD 6 binary we needed to set a custom LD_LIBRARY_PATH > | > and the FreeBSD 7 binary find a library via the FreeBSD 6 search path > | > then the FreeBSD 7 binary would die. By adding in the osversion search > | > path I can put the FreeBSD 6 library into the search path + the directory > | > 6. Then only FreeBSD 6 binaries can find it. An example: > | > LD_LIBRARY_PATH=/usr/custom_software/lib > | > /usr/custom_software/lib/6/libfoo.so.6 > | > then only the FreeBSD 6 binary could load it. Since the searches > | > would be for the FreeBSD 6 binary: > | > /usr/custom_software/lib/603000/libfoo.so.6 > | > /usr/custom_software/lib/6/libfoo.so.6 > | > /usr/custom_software/lib/libfoo.so.6 > | > and for FreeBSD 7 binary: > | > /usr/custom_software/lib/702000/libfoo.so.6 > | > /usr/custom_software/lib/7/libfoo.so.6 > | > /usr/custom_software/lib/libfoo.so.6 > | > Only the FreeBSD 6 binary would load /usr/custom_software/lib/6/libfoo.so.6. > | > I do the same search with LD_PRELOAD. > | > | Hmm, I'm still not quite fan of this. Perhaps you could add an extension to > | ldconfig and the hints file to handle this case? That is, a way to store > | path mappings so you could do something like: > | > | ldconfig -os=6 -p /usr/local/lib /usr/local/lib/6 > > I'll have to look to see how the hints file could update rtld. It is > an interesting idea. Maybe libmap.conf would be better place for this. > I haven't looked at how libmap works. Maybe introduce: > /etc/libmap-.conf > that maps paths as well. So with the above example. > /etc/libmap-6.conf > would contain > /usr/custom_software/lib /usr/custom_software/lib/6:/usr/custom_software/lib > for example. > > | Or maybe you could make it an extension of how 'm' worked, so you could make > | directories accept an optional set of colon-separated paths that they serve > | as aliases for: > | > | ldconfig -os=6 -m /usr/local/lib/6:/usr/local/lib:/usr/lib > > I don't really get how this is solving the LD_LIBRARY_PATH/LD_PRELOAD since > "-m" is solving the general case with the hints file which I did first. It is just an alternate suggestion to the -p thing above. Having a libmap.conf option to establish the mappings would be fine with me as well. I would just like to have the library paths be something the user explicitly configures rather than having rtld perform black-box path manipulations itself. > | > Final, is that prior binaries built on FreeBSD i386 but run on FreeBSD amd64 > | > that set LD_* environemnt variables would fail on FreeBSD amd64 as is > | > since it didn't set the equivalent LD32_*. To address this for COMPAT_32BIT > | > I have rtld look for LD32_* first and then check for the LD_* second. > | > This way legacy applications work fine. > | > | Hmm, so Yahoo! had some patches to handle this as well. I think Yahoo's > | patches were different though. They actually patched the 32-bit libc to > | capture attempts to get or set LD_* and convert them to actually get/set > | LD32_* instead. I'm not sure which approach is best, but it might be worth > | asking Peter why Yahoo! did it that way and if there were reasons for that > | approach vs. doing it in rtld. I think the primary reason was so that you > | could set LD_LIBRARY_PATH or LD_PRELOAD to reference 64-bit libraries and > | not have it break 32-bit apps, but if a 32-bit app tried to set a variable > | before launching another app it would still DTRT. > > This means you would have to have a modified libc in these environment > and it wouldn't help in the static binary case. I think we are safe > in the case LD_ for a 32 bit binary effecting a 64 bit since rtld should > not allow loading a the wrong type of lib. into a binary. I can do some > more testing around that area. I was trying to keep all changes in the > host environment so we can run unchanged binaries from other vendors. Yes, I'm not quite convinced which approach is best, mostly I just wanted to point out the alternative approach so both could be considered. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Sat Mar 24 20:00:31 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3EFEB1065687; Sat, 24 Mar 2012 20:00:31 +0000 (UTC) (envelope-from rmh.aybabtu@gmail.com) Received: from mail-wg0-f50.google.com (mail-wg0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 5F40A8FC12; Sat, 24 Mar 2012 20:00:25 +0000 (UTC) Received: by wgbds12 with SMTP id ds12so2877975wgb.31 for ; Sat, 24 Mar 2012 13:00:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=KSKpqJz0XY1g9K4duKm7F2v8b6kNIapZ8bFHmY9oMO0=; b=ujxbeebWoRT4rpj1FvFI4KivXndTQEdNfKpItOc5UI2bm/m8lZhUQ1CwMx6x06fM7j ukGCBc8Zl5WB4d5X7zriaIfEFMNY3A5/NCHehry5wG0FJBidrxy85gjU8UjbY7uZFLkv IcgZzrvA9V3cjkakOguCWkjKsNi7KWnwKcBverHLPAzBLbOU0m1JaOcfLJ0aWbg7Wb1H iwF8bhrS3r8VJCdkwIpMLcQTjAvqCmgNEn0wYRB61ccT8y03cR7r3Sivi9+e9HVjaR3Q 8gLEnrdm2jVEVa8BUNnMXSh8kIY17ZkaqOZpfnynfvZnuAfzBtxYB1TamZsokGGjFCc4 4skQ== Received: by 10.216.139.79 with SMTP id b57mr8920259wej.37.1332619218330; Sat, 24 Mar 2012 13:00:18 -0700 (PDT) Received: from thorin (249.Red-81-33-111.dynamicIP.rima-tde.net. [81.33.111.249]) by mx.google.com with ESMTPS id fl2sm41933904wib.4.2012.03.24.13.00.16 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 24 Mar 2012 13:00:17 -0700 (PDT) Sender: Robert Millan Received: from rmh by thorin with local (Exim 4.72) (envelope-from ) id 1SBX8I-0000MX-Hq; Sat, 24 Mar 2012 21:00:14 +0100 Date: Sat, 24 Mar 2012 21:00:14 +0100 From: Robert Millan To: freebsd-arch@freebsd.org Message-ID: <20120324200014.GA91966@thorin> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MGYHOYXEY6WxJCY8" Content-Disposition: inline User-Agent: Mutt/1.5.20 (2009-06-14) Cc: davidxu@freebsd.org Subject: [PATCH] add SIGSERVICE to sys/signal.h X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Mar 2012 20:00:31 -0000 --MGYHOYXEY6WxJCY8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, As SIGSERVICE is de-facto reserved by librt, shouldn't it be defined (or at least listed as such) in , just like SIGTHR/SIGLWP ? See attached patch. -- Robert Millan --MGYHOYXEY6WxJCY8 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="sigservice.diff" Index: lib/librt/sigev_thread.h =================================================================== --- lib/librt/sigev_thread.h (revision 233095) +++ lib/librt/sigev_thread.h (working copy) @@ -67,8 +67,6 @@ #define SNF_REMOVED 0x02 #define SNF_SYNC 0x04 -#define SIGSERVICE (SIGTHR+1) - int __sigev_check_init(); struct sigev_node *__sigev_alloc(int, const struct sigevent *, struct sigev_node *, int); Index: sys/sys/signal.h =================================================================== --- sys/sys/signal.h (revision 233095) +++ sys/sys/signal.h (working copy) @@ -111,6 +111,7 @@ #if __BSD_VISIBLE #define SIGTHR 32 /* reserved by thread library. */ #define SIGLWP SIGTHR +#define SIGSERVICE 33 /* reserved by real-time library. */ #endif #define SIGRTMIN 65 --MGYHOYXEY6WxJCY8--