Date: Fri, 10 May 2002 13:35:56 -0700 From: Terry Lambert <tlambert2@mindspring.com> To: Mikhail Teterin <mi+mx@aldan.algebra.com> Cc: Mikhail Teterin <mi@corbulon.video-collage.com>, current@FreeBSD.org Subject: Re: does the order of .a files matter? Message-ID: <3CDC2F2C.2CB9C818@mindspring.com> References: <200205101233.g4ACXctb041093@corbulon.video-collage.com> <3CDBFA7E.A2A1F09F@mindspring.com> <200205101343.01636.mi%2Bmx@aldan.algebra.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Mikhail Teterin wrote:
> = For my information: Why didn't you take John De Bowsky's advice to:
> =
> = ld $objlist `lorder $liblist | tsort -q`
>
> I tried that before I asked on the mailing list the first time. It
> did reduce the number of the undefined symbols, but not to zero.
It's possible that the symbols are truly undefined (e.g. "stat64"),
but I think that is unlikely.
Here is what I think:
Your proximal problem is that your libraries are badly organized, and
therefore certain object files in them are not being pulled into the
linking process, because your order of operation on the objects is not
in dependency order, because of the improper organization.
> It would probably be quite beneficial if you dropped this paternalistic
> attitude. "Pain high enough"... Please...
If I sounded paternalistic in my answer, it's because the question
being asked is really a "newby" question about how linkers work, and
really doesn't belong on the -current list.
Here is a more comprehensive answer, which does not leave the solution
as "an exercise for the student":
Most linkers don't do what you want, which is make up for programmer
incompetence by doing an automatic topological sort on all symbol
dependencies, regardless of where or in what type of file the symbol
is defined, because most linkers treat archives and libraries very
differently than lists of object files.
This is the technically correct thing for them to do.
The topological sorting interior to archives ("libraries") is
supposed to be accomplished via "ranlib":
ranlib generates an index to the contents of an archive,
and stores it in the archive. The index lists each symbol
defined by a member of an archive that is a relocatable
object file.
You may use `nm -s' or `nm --print-armap' to list this in-
dex.
An archive with such an index speeds up linking to the li-
brary, and allows routines in the library to call each
***************************************************
other without regard to their placement in the archive.
******************************************************
This only works intra-archive, not inter-archive. For inter-archive,
you are expected to order the archives in the proper order, and the
breakup of objects into archives is assumed to have been arranged by
the original programmer such that they are a directed acyclic graph.
That is, it is expected that if you have three libraries, then they
define an order set of symbol dependencies, such that *no two of the
following are simultaneously true*:
a<-b<-c, a<-c<-b, b<-c<-a, b<-a<-c, c<-a<-b, c<-b<-a
Any place the original programmer has violated this assumption means
linking the library more than once, e.g. if both are true:
a<-b<-c, b<-c<-a
Then you need:
$(OBJS) -la -lb -lc -la
Such code is, by definition, broken. Relinking the "a" library because
the "c" library consumes symbols defined in "a" but not consumed by
"$(OBJS)" is *an incredibly ugly workaround* to the fact that the
object in the libraries are not order in DAG order, and then linked in
exterior edge-of-DAG order.
In fact, the link order above indicates that the dependency order is
cyclic rather than acyclic ("a depends on b depends on c depends on a ...
infinity").
> Tsort is ALREADY incorporated into ld in some shape, because object
> files on command line or within one .a CAN be misordered.
Within one .a, they are only permitted to be misordered if "ranlib"
has been run on the archive (see the quotation of the ranlib manual
page, above).
Within multiple .a's, they are handled differently, because linking
against a .a file does not necessarily pull in all of the object
files in the archive. *This is intentional; it is by design*.
> All I ask, is why the collections of object files provided by different
> static libs are/can not be treated as one big collection.
This is a conflicting requirement.
You can't have it both ways.
It's my understanding that you are making libraries in the first
place in order to get around command line length limitations, and
have settled upon archives, rather than incremental linking using
"ld -r -o A.o ${A_OBJS}".
If this is the case, it would probably be a good idea to choose
which objects go into which library carefully, to avoid ending
up with undefined symbols.
Alternately, if you insist on using ".a" files directly, as if they
were normal object files, someone has already posted that you should
probably use the "--whole-archive" argument, so that the archive
contents aren't pulled in *only* if they define symbols which are
in the current undefined symbol list, but to pull them in unconditionally
(i.e. "treat them as a list of object files instead of an archive").
> = I have to say that, given a choice between "make world" taking several
> = minutes longer and you not having to reorganize your code into logical
> = component units, vs. it taking less time to do a make world, and one
> = programmer having to *fix* their code, I have to pick you fixing your
> = code.
>
> Of course. Does not seem like it will come to this, however.
Yes.
Because I and amost guarantee you that the linker will not be changed
to do what you request, by default, because it is the wrong way to
fix a problem that shouldn't exist in the first place. The FreeBSD
and the compiler folks will both agree on this, and neither one will
change the linker behaviour for you. In the end, you will have to
solve your problem a different way... one of the three ways you
have already been told to solve it, in this thread, and which I have
laid out, in no uncertain terms, in this email.
> = Also, this is a tools problem, and the tools provide a way (even if
> = it's ugly) to get the behaviour you want, with a single option before
> = your objects, and another one after.
>
> Hmm, no. The only reliable option is to either build a "library of all
> libraries" or link the the object files into the executable directly.
>
> This, BTW, shows how inconsistent the current situation is -- the linker
> does not mind misordered object files, but is very picky about the order
> of static libraries (shared ones are can be misordered too, AFAIK). I
> don't see how one can sincerely defend its lack of what still appears
> to be a missing feature.
Libraries are not object files, and they are not lists of object files.
You can create a list of object files by:
ld -r -o list_x.o ${X_OBJS}
and then linking:
cc -o foo ${OBJS} list_x.o
Instead of:
cc -o foo ${OBJS} list_x.a
An archive of object files is very different than a list of object
files (unless you specify the "--whole-archive" command line argument);
an archive of object files attempts to minimize the inclusion of objects
in the archive that have no relevence to the resulting executable. It
does this by only pulling in objects in the archive if they satisfy some
preexisting dependency, and *leaving out objects which do not meet this
criteria*.
*PLEASE* either reorganize your libraries, or use incremental
linking, *not libraries*, to make it clear to the people who follow
you that what there are is collections of object files, and *not*
really true libraries. A true library implies that you have done
the dependency ordering necessary to avoid the problems you are
seeing because you are using libraries and *have not done that*.
In other words, *do not use "--whole-archive", please*.
-- Terry
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3CDC2F2C.2CB9C818>