From owner-freebsd-ports@FreeBSD.ORG Fri Jun 6 19:45:50 2014 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75998CF5; Fri, 6 Jun 2014 19:45:50 +0000 (UTC) Received: from mailrelay007.isp.belgacom.be (mailrelay007.isp.belgacom.be [195.238.6.173]) by mx1.freebsd.org (Postfix) with ESMTP id 8D7972785; Fri, 6 Jun 2014 19:45:48 +0000 (UTC) X-Belgacom-Dynamic: yes X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlgGABoZklNbs4fn/2dsb2JhbABYAYMNUkzEFAGBBhd1hAMBAQQBOhwjBQsLDgoJEAITDyoeBhMbiB8MAc1XF45pBycBhBkBA5ocgUOSAoM+Ow Received: from 231.135-179-91.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([91.179.135.231]) by relay.skynet.be with ESMTP; 06 Jun 2014 21:44:38 +0200 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.14.8/8.14.8) with ESMTP id s56JibTx006279; Fri, 6 Jun 2014 21:44:37 +0200 (CEST) (envelope-from tijl@FreeBSD.org) Date: Fri, 6 Jun 2014 21:44:37 +0200 From: Tijl Coosemans To: Bryan Drewery Subject: Re: To all port maintainers: libtool Message-ID: <20140606214437.4e9d60b0@kalimero.tijl.coosemans.org> In-Reply-To: <5391D62D.205@FreeBSD.org> References: <20140508002420.5d37e7f6@kalimero.tijl.coosemans.org> <20140508212756.00000df3@Leidinger.net> <20140509001641.63310821@kalimero.tijl.coosemans.org> <53908119.5040505@FreeBSD.org> <20140605185303.474063c6@kalimero.tijl.coosemans.org> <20140606150224.Horde.DQm9mw37wR_4m0sWWrikjA1@webmail.leidinger.net> <20140606162736.205806af@kalimero.tijl.coosemans.org> <5391D62D.205@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Alexander Leidinger , freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jun 2014 19:45:50 -0000 On Fri, 06 Jun 2014 09:54:37 -0500 Bryan Drewery wrote: > On 6/6/14, 9:27 AM, Tijl Coosemans wrote: >> On Fri, 06 Jun 2014 15:02:24 +0200 Alexander Leidinger wrote: >>> Quoting Tijl Coosemans (from Thu, 5 Jun 2014 >>> 18:53:03 +0200): >>>> On Thu, 05 Jun 2014 09:39:21 -0500 Bryan Drewery wrote: >>>>> I don't know what .la files are used for and have no time currently to >>>>> research it. >>>>> >>>>> What is the impact to non-ports consumers of removing .la files? Do they >>>>> also need patches to make them build? >>>> >>>> Removing a .la file is somewhat like a library version bump. Anything >>>> that depends on it needs to be recompiled. >>> >>> I remember from tests waaaaay in the past that not all programs will >>> be happy when the .la files are not there. I remember that I once >>> tried to remove the .la files but it didn't work as the program wanted >>> to open the .la files (after recompile). Maybe libltdl is openening >>> them? Did you make some checks/tests in this regard? >> >> Essentially .la files are small shell scripts that set some variables so >> in theory they can be used in all kinds of places, but this seems of >> little practical value. Libltdl can open and parse .la files (to find >> the name of the .so file it can dlopen) but it can also work directly >> with .so files. >> >> If a program uses .la files directly then the port can't delete them of >> course, but so far I haven't encountered such programs. > > My main question was non-ports consumers though. What is the impact on > them? We're talking a lot about bumping revisions and forcing rebuilds. > > Will non-ports consumers require rebuild or changes if .la files are > missing? If they have a .la file that links to the .la file that went missing then their .la file needs to be rebuilt. Like I said before, this is very similar to .so version bumps. If they have a program (or library) that links to another library that goes from .so.X to .so.(X+1) then their program (or library) needs to be rebuilt too. This is nothing out of the ordinary. If you update a dependency that exists in the ports tree you might have to rebuild stuff that exists outside the tree. This is something you always have to keep in mind. > Would these non-ports consumers be able to properly build and link > without .la files and without modifications? These .la files seem > similar to .pc files which are often critical to out-of-tree consumers > that assume Linux /usr paths instead of /usr/local paths. You specifically asked about the impact of the removal of .la files so I restricted my answer to just that, but now I suspect you really want to know about all impacts of USES=libtool. So here it goes. A .la file operates one level below a .pc file so to speak. A libfoo.pc file gives you information about a libfoo package (what compiler and linker flags to use to link with libfoo), while a libfoo.la file will give you information about the dependencies of libfoo. Currently all forms of USES=libtool erase this dependency information or erase the .la file altogether. Suppose you have a program (or library) X outside the ports tree which links to libfoo inside the tree which in turn links to libbar. The command to link X looks roughly like this: libtool --mode=link cc -o X X1.o X2.o -L/usr/local/lib -lfoo Without USES=libtool in the libfoo port, the libtool script will turn that cc command into "cc -o X X1.o X2.o -L/usr/local/lib -lfoo -lbar" and the resulting binary X will link to both libfoo.so and libbar.so. If X is a library, libtool will also generate X.la which lists libfoo.la and libbar.la as dependencies. Note that these dependencies accumulate. A binary Y that links to library X ends up linking to X.so, libfoo.so and libbar.so. That's what we want to get rid of. With USES=libtool (any form) in the libfoo port, there's no dependency information in libfoo.la so the above libtool command does not add -lbar to the cc command line. X only links with libfoo.so and X.la only lists libfoo.la as a dependency (if it exists, otherwise just -lfoo). This may fail if X uses libbar API directly. The modification you have to make then is to add libbar as a dependency of X in your makefile. USES=libtool exposes these kinds of hidden dependencies. It forces direct dependencies to be listed explicitly. So far the case of libfoo being a shared library. Things may break if it's a static library. If a program X uses functions from a file in the static libfoo.a archive and that file requires functions from libbar then you must also specify -lbar on the command line so it was very convenient for libtool to add -lbar automatically. With USES=libtool (any form) in the libfoo port you must now add libbar as a dependency of X in your makefile even if X doesn't use libbar API directly. We may always revisit this of course but at the time of the creation of USES=libtool this was considered an acceptable loss for the following reasons: 1) Static linking with libtool outside the ports tree is assumed to be used rarely and if it's used it won't be with any complicated dependency chains. 2) The accumulated overlinking of shared libraries is too bad to ignore. 3) All major Linux distributions do this too. 4) There's an alternative with pkg-config. It can add -lbar automatically too. If the libfoo port installs a .pc file "pkg-config --libs --static libfoo" will expand to "-L/usr/local/lib -lfoo -lbar". So to summarise, when a port adds some form of USES=libtool, code outside the tree may see these two errors: 1) Linking X fails due to a missing .la file. Solution: find the .la file that still contains a reference to the missing .la file (simple grep will do) and rebuild the library it corresponds to. 2) Linking X fails due to unresolved symbols. Solution: add the libraries that provide those symbols to the X linker command line in your makefile (or use "pkg-config --libs --static" instead of "pkg-config --libs").