Date: Tue, 13 Mar 2012 10:29:17 -0700 (PDT) From: Doug Ambrisko <ambrisko@ambrisko.com> To: arch@freebsd.org Subject: rtld enhancement to add osversion, version 2 Message-ID: <201203131729.q2DHTH3b066201@ambrisko.com>
next in thread | raw e-mail | index | archive | help
This is round 2 of my rtld enhancements. The primary goal it to have rtld look in different places for libraries depending on the legacy binary that we want to run. This is especially a problem with binaries linked to libraries from ports since the version of a library in /usr/local/lib is the same whether it was FreeBSD 6, 7, 8 etc. until the library itself changes version due to an interface change. At work we need to run 3rd party binaries on our box of which we don't have source. Having a FreeBSD 6 binaries load /usr/local/lib/libiconv.so.3 that was built for FreeBSD 9 is not good. It is worse when libiconv.so.3 is linked to libc.so.7 when the FreeBSD 6 binary needs libc.so.6. I solved this by having rtld extract the OSVERSION from the binary and then use that to determine what to do. In my prior version, I inserted that into the library directory. That meant to pull in libc it would look at: /lib/603000/libc.so.6 /lib/6/libc.so.6 /lib/libc.so.6 to find libc.so.6. This meant a lot more look ups. Also I found it had a problem in that if we had an ambiguous name say /usr/local/lib/libcrypto.so.5 on the FreeBSD 6 machine and /lib/libcrypto.so.5 on the new FreeBSD 8 system, just doing the search logic would get a hit on /lib/libcrypto.so.5 before it got to /usr/local/lib/libcrypto.so.5. So then it would mean we'd have to put all FreeBSD lib's in /lib/6 to beat the search path. This wasn't a good solution. To solve the performance and path issue, I now follow the 32 bit hints file name model. Now it does a lookup of the hints file based on the osversion and major. So now it looks for the hints file as: /var/run/ld-elf-603000.so.hints /var/run/ld-elf-6.so.hints /var/run/ld-elf.so.hints This is faster and has more unique paths for FreeBSD 6 libraries since the FreeBSD 6 search paths would be in the hints file. I modified ldconfig to accept an "-os=<version>" option to create this hints file. I tweaked /etc/rc* to make this easy to setup like this: ldconfig_os_versions="6" ldconfig_6_path="/usr/local/lib/compat/6" This doesn't solve the LD_LIBRARY_PATH or LD_PRELOAD. Solving that I still insert and iterate over the osverion, major and none into the path to find the library. The reason for this is that a FreeBSD 8 binary could exec a FreeBSD 6 binary that execs a FreeBSD 7 binary. If for the FreeBSD 6 binary we needed to set a custom LD_LIBRARY_PATH and the FreeBSD 7 binary find a library via the FreeBSD 6 search path then the FreeBSD 7 binary would die. By adding in the osversion search path I can put the FreeBSD 6 library into the search path + the directory 6. Then only FreeBSD 6 binaries can find it. An example: LD_LIBRARY_PATH=/usr/custom_software/lib /usr/custom_software/lib/6/libfoo.so.6 then only the FreeBSD 6 binary could load it. Since the searches would be for the FreeBSD 6 binary: /usr/custom_software/lib/603000/libfoo.so.6 /usr/custom_software/lib/6/libfoo.so.6 /usr/custom_software/lib/libfoo.so.6 and for FreeBSD 7 binary: /usr/custom_software/lib/702000/libfoo.so.6 /usr/custom_software/lib/7/libfoo.so.6 /usr/custom_software/lib/libfoo.so.6 Only the FreeBSD 6 binary would load /usr/custom_software/lib/6/libfoo.so.6. I do the same search with LD_PRELOAD. Final, is that prior binaries built on FreeBSD i386 but run on FreeBSD amd64 that set LD_* environemnt variables would fail on FreeBSD amd64 as is since it didn't set the equivalent LD32_*. To address this for COMPAT_32BIT I have rtld look for LD32_* first and then check for the LD_* second. This way legacy applications work fine. The patch is at: http://people.freebsd.org/~ambrisko/rtld_hints_osversion.patch and has been tested a fair amount at work. It can make running a different version of a FreeBSD binary very easy without having any cross-linking. There is a bug in FreeBSD 6.0 that didn't record the osversion in the binary so then this doesn't work. Try to avoid those binaries! We have a local hack that says if there is no osversion then assume it is a 6 binary. Again, this is to solve running legacy binaries on a new version of FreeBSD of which we don't have source to recompile with new ld features of -z origin. Comments? Thanks, Doug A.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201203131729.q2DHTH3b066201>