Date: Mon, 19 Mar 2012 23:03:54 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: John Baldwin <jhb@freebsd.org> Cc: freebsd-arch@freebsd.org Subject: Re: rtld enhancement to add osversion, version 2 Message-ID: <20120319210354.GE2358@deviant.kiev.zoral.com.ua> In-Reply-To: <201203191659.34568.jhb@freebsd.org> References: <201203131729.q2DHTH3b066201@ambrisko.com> <201203191659.34568.jhb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--orO6xySwJI16pVnm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 19, 2012 at 04:59:34PM -0400, John Baldwin wrote: > On Tuesday, March 13, 2012 1:29:17 pm Doug Ambrisko wrote: > > This is round 2 of my rtld enhancements. The primary goal it to have > > rtld look in different places for libraries depending on the legacy > > binary that we want to run. This is especially a problem with binaries= =20 > > linked to libraries from ports since the version of a library in=20 > > /usr/local/lib is the same whether it was FreeBSD 6, 7, 8 etc. until th= e=20 > > library itself changes version due to an interface change. At work we= =20 > > need to run 3rd party binaries on our box of which we don't have source. > > Having a FreeBSD 6 binaries load /usr/local/lib/libiconv.so.3 that was= =20 > > built for FreeBSD 9 is not good. It is worse when libiconv.so.3 is > > linked to libc.so.7 when the FreeBSD 6 binary needs libc.so.6. > >=20 > > I solved this by having rtld extract the OSVERSION from the binary > > and then use that to determine what to do. In my prior version, > > I inserted that into the library directory. That meant to pull > > in libc it would look at: > > /lib/603000/libc.so.6 > > /lib/6/libc.so.6 > > /lib/libc.so.6 > > to find libc.so.6. This meant a lot more look ups. Also I found it > > had a problem in that if we had an ambiguous name say > > /usr/local/lib/libcrypto.so.5 on the FreeBSD 6 machine and=20 > /lib/libcrypto.so.5 > > on the new FreeBSD 8 system, just doing the search logic would get > > a hit on /lib/libcrypto.so.5 before it got to /usr/local/lib/libcrypto.= so.5. > > So then it would mean we'd have to put all FreeBSD lib's in /lib/6 to > > beat the search path. This wasn't a good solution. To solve the > > performance and path issue, I now follow the 32 bit hints file name=20 > > model. Now it does a lookup of the hints file based on the osversion > > and major. So now it looks for the hints file as: > > /var/run/ld-elf-603000.so.hints > > /var/run/ld-elf-6.so.hints > > /var/run/ld-elf.so.hints > > This is faster and has more unique paths for FreeBSD 6 libraries since > > the FreeBSD 6 search paths would be in the hints file. I modified > > ldconfig to accept an "-os=3D<version>" option to create this hints fil= e. > > I tweaked /etc/rc* to make this easy to setup like this: > > ldconfig_os_versions=3D"6" > > ldconfig_6_path=3D"/usr/local/lib/compat/6" >=20 > I think this is a definite improvement from before, thanks! >=20 > > This doesn't solve the LD_LIBRARY_PATH or LD_PRELOAD. Solving that > > I still insert and iterate over the osverion, major and none into the > > path to find the library. The reason for this is that a FreeBSD 8=20 > > binary could exec a FreeBSD 6 binary that execs a FreeBSD 7 binary. =20 > > If for the FreeBSD 6 binary we needed to set a custom LD_LIBRARY_PATH > > and the FreeBSD 7 binary find a library via the FreeBSD 6 search path > > then the FreeBSD 7 binary would die. By adding in the osversion search > > path I can put the FreeBSD 6 library into the search path + the directo= ry > > 6. Then only FreeBSD 6 binaries can find it. An example: > > LD_LIBRARY_PATH=3D/usr/custom_software/lib > > /usr/custom_software/lib/6/libfoo.so.6 > > then only the FreeBSD 6 binary could load it. Since the searches > > would be for the FreeBSD 6 binary: > > /usr/custom_software/lib/603000/libfoo.so.6 > > /usr/custom_software/lib/6/libfoo.so.6 > > /usr/custom_software/lib/libfoo.so.6 > > and for FreeBSD 7 binary: > > /usr/custom_software/lib/702000/libfoo.so.6 > > /usr/custom_software/lib/7/libfoo.so.6 > > /usr/custom_software/lib/libfoo.so.6 > > Only the FreeBSD 6 binary would load /usr/custom_software/lib/6/libfoo.= so.6. > > I do the same search with LD_PRELOAD. >=20 > Hmm, I'm still not quite fan of this. Perhaps you could add an extension= to > ldconfig and the hints file to handle this case? That is, a way to store > path mappings so you could do something like: >=20 > ldconfig -os=3D6 -p /usr/local/lib /usr/local/lib/6 >=20 > Or maybe you could make it an extension of how 'm' worked, so you could m= ake > directories accept an optional set of colon-separated paths that they ser= ve > as aliases for: >=20 > ldconfig -os=3D6 -m /usr/local/lib/6:/usr/local/lib:/usr/lib >=20 > (That would even fit into your existing rc.d script changes I believe). = Then > rtld would keep this internal directory mapping and be able to map the > '/usr/local/lib' and '/usr/lib' directories in LD_PRELOAD and LD_LIBRARY_= PATH > to /usr/local/lib/6. The advantage of this is the same as with your prev= ious > change that all the mappings are configurable and not hard-coded into rtld > itself. >=20 > > Final, is that prior binaries built on FreeBSD i386 but run on FreeBSD = amd64 > > that set LD_* environemnt variables would fail on FreeBSD amd64 as is > > since it didn't set the equivalent LD32_*. To address this for COMPAT_= 32BIT > > I have rtld look for LD32_* first and then check for the LD_* second. > > This way legacy applications work fine. >=20 > Hmm, so Yahoo! had some patches to handle this as well. I think Yahoo's > patches were different though. They actually patched the 32-bit libc to > capture attempts to get or set LD_* and convert them to actually get/set > LD32_* instead. I'm not sure which approach is best, but it might be wor= th > asking Peter why Yahoo! did it that way and if there were reasons for that > approach vs. doing it in rtld. I think the primary reason was so that you > could set LD_LIBRARY_PATH or LD_PRELOAD to reference 64-bit libraries and > not have it break 32-bit apps, but if a 32-bit app tried to set a variable > before launching another app it would still DTRT. >=20 > I do think this is definitely a problem worth solving. Just a trivial note: r232831 added note parsing and stores osrel in the Obj_Entry. --orO6xySwJI16pVnm Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk9nnzoACgkQC3+MBN1Mb4idNACePMp9ApNP3rEyZFnp5WL8POIs QOIAoNjapaNtDE0Jig9y0uK9htu9aXHm =8bnF -----END PGP SIGNATURE----- --orO6xySwJI16pVnm--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120319210354.GE2358>