Date: Mon, 10 Jul 2006 15:07:22 +0200 From: "no@spam@mgedv.net" <nospam@mgedv.net> To: "'Giorgos Keramidas'" <keramida@ceid.upatras.gr> Cc: freebsd-questions@freebsd.org Subject: RE: shared library loader configuration Message-ID: <000501c6a421$c8c3a9a0$01010101@avalon.lan> In-Reply-To: <20060707134616.GA3807@gothmog.pc>
next in thread | previous in thread | raw e-mail | index | archive | help
> -----Original Message----- > From: owner-freebsd-questions@freebsd.org > [mailto:owner-freebsd-questions@freebsd.org] On Behalf Of > Giorgos Keramidas > Sent: Friday, July 07, 2006 3:46 PM > To: nospam@mgedv.net > Cc: freebsd-questions@freebsd.org > Subject: Re: shared library loader configuration > > On 2006-07-07 14:58, "no@spam@mgedv.net" <nospam@mgedv.net> wrote: > >Giorgos Keramidas wrote: > >>On 2006-07-07 14:22, "no@spam@mgedv.net" <nospam@mgedv.net> wrote: > >>> dunno, if it's a misunderstanding, but my only question > "how to tell > >>> the system where to load libraries and in which order to prefer > >>> paths" seems to be still open. > >>> > >>> anyway, thx for the reply ;-) > >>> > >>> ps: i already RdTFM ;-) > >> > >> You don't. Unless you modify the /etc/rc.d/ldconfig > script manually, > >> /lib and /usr/lib will always be the first to search. > >> > >> I'm still not convinced that "telling the system where to load > >> libraries from" is the solution to you problem, but I > don't know what > >> the problem is. You have to describe first *WHAT* the real problem > >> is and *WHY* you think modifying the library path is a solution. > > > > i found the ldconfig rc-script but i thought there might be a > > "cleaner" way of telling the system where the shared > libraries are to > > be found. > > > > any way to tell the system: take /usr/local/lib first w.o. changing > > the ldconfig rc-scripts or developing own startup scripts > that achieve > > that? no way of changing some default configuration file that is > > avail. for that purpose? > > No. For a very good reason too. If you change the default > loader path > to use /usr/local/lib first then *ALL* the programs are > affected. Even > those that are part of the base system. This is, in general, > a very bad > idea as the base-system programs may depend on particular versions of > the libraries to work correctly and those libraries are in > /lib:/usr/lib. > > > some additional thoughts (a little bit of phil.): > > > > i wonder, that anybody scripts such hardcoded stuff into a script > > because the environment /etc/ld*conf* exists, and at least > for a clear > > and proper way for the admin to define what to load from where it > > should be possible, to override a default configuration via the > > config-files, and not with modifications to rc-scripts > which are gone > > by default after each upgrade. > > There's a reason why /lib:/usr/lib take precedence over /usr/local. > > Don't change that, or be prepared to debug all the potential bugs that > may start appearing. > > > to satisfy your couriosity :-) > > This description is much better. > > > i'd like to compile openssl 0.9.8 and a newer zlib for testing some > > software that does crypto & compression using these libs. and i > > wanted to keep the servers as clean as possible from changing > > rc-scripts, etc... to ensure we're able to transfer the outcoming > > piece of program to other boxes w'out much effort. i know > it's inside > > the ports but the problem is, we'd like to tes some sort of code > > that's not enabled by default in the ports. > > You can always install your own version of openssl and zlib and use an > explicit -R option to tell the linker where to find libraries. There > are two steps involved in linking and distributing an executable: > > * Build-time linking, where the compiler/linker has to be > explicitly > pointed at teh right place with -L/foo/lib paths. > > * Runtime linking, where the runtime linker-loader has to > lookup and > locate the library. > > You can affect the first one with explicit -L/foo/lib > options. You can > also hardcode one or more paths to the binary -- solving the second > problem too -- if you use -R/distrib/lib options while building. > > Your problems are only a matter of using the proper build options. > > See for example the ldd output near the end of the following log: > > # root@gothmog:/home/giorgos/tmp/foo# make cleandir > # ===> lib (cleandir) > # ===> lib/libz (cleandir) > # rm -f a.out foo.o foo.o.tmp > # rm -f foo.po foo.po.tmp > # rm -f foo.So foo.so foo.So.tmp > # rm -f libz.so > # rm -f libz.so.* libz.so > # rm -f libz.a libz_p.a libz.so.1 > # rm -f .depend GPATH GRTAGS GSYMS GTAGS > # ===> bin (cleandir) > # ===> bin/foo (cleandir) > # rm -f foo foo.o > # rm -f .depend GPATH GRTAGS GSYMS GTAGS > # root@gothmog:/home/giorgos/tmp/foo# make cleandir > # ===> lib (cleandir) > # ===> lib/libz (cleandir) > # rm -f a.out foo.o foo.o.tmp > # rm -f foo.po foo.po.tmp > # rm -f foo.So foo.so foo.So.tmp > # rm -f libz.so > # rm -f libz.so.* libz.so > # rm -f libz.a libz_p.a libz.so.1 > # rm -f .depend GPATH GRTAGS GSYMS GTAGS > # ===> bin (cleandir) > # ===> bin/foo (cleandir) > # rm -f foo foo.o > # rm -f .depend GPATH GRTAGS GSYMS GTAGS > # root@gothmog:/home/giorgos/tmp/foo# make obj > # ===> lib (obj) > # ===> lib/libz (obj) > # /home/giorgos/tmp/foo/obj/home/giorgos/tmp/foo/lib/libz > created for /home/giorgos/tmp/foo/lib/libz > # ===> bin (obj) > # ===> bin/foo (obj) > # /home/giorgos/tmp/foo/obj/home/giorgos/tmp/foo/bin/foo > created for /home/giorgos/tmp/foo/bin/foo > # root@gothmog:/home/giorgos/tmp/foo# make > # ===> lib (all) > # ===> lib/libz (all) > # cc -O2 -fno-strict-aliasing -pipe > -I/home/giorgos/tmp/foo/lib/libz -g -Wsystem-headers -Werror > -Wall -Wno-format-y2k -Wno-uninitialized -c > /home/giorgos/tmp/foo/lib/libz/foo.c > # building static z library > # ranlib libz.a > # cc -pg -O2 -fno-strict-aliasing -pipe > -I/home/giorgos/tmp/foo/lib/libz -g -Wsystem-headers -Werror > -Wall -Wno-format-y2k -Wno-uninitialized -c > /home/giorgos/tmp/foo/lib/libz/foo.c -o foo.po > # building profiled z library > # ranlib libz_p.a > # cc -fpic -DPIC -O2 -fno-strict-aliasing -pipe > -I/home/giorgos/tmp/foo/lib/libz -g -Wsystem-headers -Werror > -Wall -Wno-format-y2k -Wno-uninitialized -c > /home/giorgos/tmp/foo/lib/libz/foo.c -o foo.So > # building shared library libz.so.1 > # ===> bin (all) > # ===> bin/foo (all) > # cc -O2 -fno-strict-aliasing -pipe > -I/home/giorgos/tmp/foo/bin/foo/../../lib/libz -g -c > /home/giorgos/tmp/foo/bin/foo/foo.c > # cc -O2 -fno-strict-aliasing -pipe > -I/home/giorgos/tmp/foo/bin/foo/../../lib/libz -g -L- > -L/home/giorgos/tmp/foo/bin/foo/../../lib/libz > -L/home/giorgos/tmp/foo/obj/home/giorgos/tmp/foo/bin/foo/../.. > /lib/libz -L/usr/lib -R/tmp/lib -o foo foo.o -lz > # root@gothmog:/home/giorgos/tmp/foo# make install > # ===> lib (install) > # ===> lib/libz (install) > # install -C -o root -g wheel -m 444 libz.a /tmp/lib > # install -C -o root -g wheel -m 444 libz_p.a /tmp/lib > # install -o root -g wheel -m 444 libz.so.1 /tmp/lib > # ln -fs libz.so.1 /tmp/lib/libz.so > # ===> bin (install) > # ===> bin/foo (install) > # install -o root -g wheel -m 555 foo /tmp/bin > # root@gothmog:/home/giorgos/tmp/foo# ldd /tmp/bin/foo > # /tmp/bin/foo: > # libz.so.1 => /tmp/lib/libz.so.1 (0x2807c000) > # libc.so.7 => /lib/libc.so.7 (0x2807e000) > # root@gothmog:/home/giorgos/tmp/foo# > > Note how the /tmp/bin/foo executable depends on `libz' but it knows > where to pick it up from. No conflict with the system version of > libz.so and certainly no problems when the utility runs: > > # root@gothmog:/home/giorgos/tmp/foo# /tmp/bin/foo > # 1 > # 0 > # 1 > # 0 > # 1 > # 0 > # 1 > # 0 > # 1 > # 0 > # root@gothmog:/home/giorgos/tmp/foo# > > The Makefile of this `foo' program and the library it uses > can be found > online at: > > http://people.freebsd.org/~keramida/files/foo-rtld.tgz > > What you can do to override on a _per_program_ basis the location of > libraries is to use: > > * Proper -L/build/path/lib options when compiling to let the > compiler file the build-time version of the library. > > * Proper -R/runtime/path/lib options when compiling to add hints > about the location of the libraries in the binary. > This way, the > runtime linker will know where to look for shared libs. > > > my (very subjective) point of view currently is, that the dynamic > > loading environment could rely on just one (or 2 if you care for > > elf/aout anymore) configuration file, which is being taken > care of the > > libexec-stuff. > > It does. These paths are /lib and /usr/lib, and there is no > good reason > to change them to satisfy just one program. > > > and assume the following: > > you're compiling ~10 libraries in series (all go to /usr/local) and > > they're some sort of connected to each other, you'll always have to > > exec ldconfig -blabla... to ensure, the loader knows about the new > > lib? > > Nope. The libraries should include -R options that point to the right > place for their dependencies. > > It's the responsibility of the builder & packager to make sure these > thigns are done Right(TM), so you won't have to fiddle with tons of > conflicting options and LD_LIBRARY_PATH trickery to have > something that > sort of works. > > Just think about it... > > [case 1] > > Let's say that today you install something that requires > /usr/local/lib to be the first, just because the guy who > compiled libfoo.so thought this was a good idea. > > Then you have to install something else, which depends on > /usr/lib being first because another developer thought this > was a good idea too (or simpe because his program has been > built with the system version of the same library). > > Suddenly, you can only have *one* of these working. > > [case 2] > > You install FreeBSD, which comes with a few dozen tools that > use Gzip compression. FreeBSD comes bundled with a carefully > tuned, debugged, tested and QA'd version of libz.so -- a > library which works 100% as expected with the system > binaries. > > Then one day, you install software package Frobnic Express, > from a company who thought it is a marvelous idea to depend > on /usr/local being the first library lookup path. You > switch, and suddenly nothing from the base system works > correctly. Programs crash, create bogus output files, or > randomly misbehave. After a week of effort, you find that > this is because Frobnic Express installed an incompatible > libz.so in /usr/local/lib which the system binaries can never > work well with. > > It's just not a good idea. Avoid it. At all costs. The hell of > maintenance effort you will have to go through when things go wrong is > not nice. Trust me when I say you won't like it at all :-) > > > hey, i'm not a real developer (as you can see) and i really > > am sure that there was a reason for developing that way. > > i'm also sure, that you can explain good reasons for doing > > it that way (and i wouldn't ever say "i don't care 'bout that!") > > > > but from a more or less "user's" point of view, it's admin- > > overhead which could (theoretically) be avoided. > > > > i'll STFU know - things are going too theoretically ;-) > > This may sound a bit harsh, but if you are not a developer, > why are you > compiling things? > > On the other hand, look at the sample `foo-rtld.tgz' tarball > I created. > See how the fake `libz' and the `foo' utility that depend on it work. > See how they are compiled, and try to use a similar scheme :) > > I hope this clarifies a bit why I think you are headed down a > very wrong > path by trying to override /lib and /usr/lib's precedence... > > Regards, > Giorgos > 1st of all, thx for the efforts to clarify things. not everybody would take this amount of time for a lazy bugger like me who doesn't like to change env-vars and compiler- args. as mentioned, i preferred a simple "switch" that renders the whole compile thing as a "change it, forget about it" thing. that this will break up thinks, is obvious. and i already thought about it (additionally, i thought it might be good for the os to e.g. use openssl-0.9.8b instead of 0.9.7e, where many improvements (security and speed) have been made. but as far as i could figure out, most things are linked against the specific version and changing ld-paths doesn't affect anything. ok dude, you convinced me a little bit and if it were more effort than changing centralized scripts written by us, i'd still wanted to change ldconfig ;-) but now, i use -L and -R, this works fine (until now) and maybe it's not necessary anymore, to workaround ldconfig. regards ;-)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?000501c6a421$c8c3a9a0$01010101>