From owner-freebsd-ports-bugs@FreeBSD.ORG Thu Mar 23 07:01:39 2006 Return-Path: X-Original-To: freebsd-ports-bugs@FreeBSD.org Delivered-To: freebsd-ports-bugs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E0D0D16A420; Thu, 23 Mar 2006 07:01:38 +0000 (UTC) (envelope-from Ralf.Wildenhues@gmx.de) Received: from smtp3.netcologne.de (smtp3.netcologne.de [194.8.194.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3828A43D4C; Thu, 23 Mar 2006 07:01:38 +0000 (GMT) (envelope-from Ralf.Wildenhues@gmx.de) Received: from localhost.localdomain (xdsl-84-44-128-151.netcologne.de [84.44.128.151]) by smtp3.netcologne.de (Postfix) with ESMTP id B054F673F9; Thu, 23 Mar 2006 08:01:36 +0100 (CET) Received: from ralf by localhost.localdomain with local (Exim 4.50) id 1FMJp4-0001vV-OH; Thu, 23 Mar 2006 08:01:30 +0100 Date: Thu, 23 Mar 2006 08:01:30 +0100 From: Ralf Wildenhues To: Peter Jeremy Message-ID: <20060323070130.GD5787@iam.uni-bonn.de> Mail-Followup-To: Peter Jeremy , bug-followup@FreeBSD.org, bug-libtool@gnu.org, magick-bugs@imagemagick.org, FreeBSD-gnats-submit@FreeBSD.org, freebsd-ports-bugs@FreeBSD.org References: <200603220827.k2M8RVE0076913@server.vk2pj.dyndns.org> <20060322162811.GB7573@iam.uni-bonn.de> <20060322184233.GC688@turion.vk2pj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060322184233.GC688@turion.vk2pj.dyndns.org> Organization: Department of Numerical Simulation, University of Bonn User-Agent: Mutt/1.5.9i Cc: freebsd-ports-bugs@FreeBSD.org, FreeBSD-gnats-submit@FreeBSD.org, magick-bugs@imagemagick.org, bug-libtool@gnu.org, bug-followup@FreeBSD.org Subject: Re: ports/94826: [patch] Very slow startup for libtool ltdl clients (eg gnucash) X-BeenThere: freebsd-ports-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Ports bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Mar 2006 07:01:39 -0000 * Peter Jeremy wrote on Wed, Mar 22, 2006 at 07:42:33PM CET: > I should acknowledge that the problem and fix were pointed out by > joerg@britannica.bec.de Thanks, I'll mention Joerg. > On Wed, 2006-Mar-22 17:28:11 +0100, Ralf Wildenhues wrote: > >I'll put adding such a test on the TODO list. I'll also put reworking > >the manual loading algorithm on the TODO list, it shouldn't scale > >nonlinearly in the number of loaded deplibs, however many modules we > >open. > > I've still got a trace of all calls to lt_dlopenext() during gnucash > startup but I've cleaned up the more detailed debugging output that > I collected (though I could probably re-create it). > > I suspect it's a combination of: > - both dlopen() and lt_dlopen() are recursively attempting to load > dependencies Right. > - The .la file can contain duplicate dependencies (which don't appear > to be stripped out) This is "normal": there are systems where you actually need to link each library against all its deplibs. So we store all of them. The linker and runtime linker on FreeBSD both follow DT_NEEDED entries and look in DT_RPATH entries inside libraries for finding and loading indirect library dependencies, though (I think). So it should be possible to improve quite a bit upon your 10 seconds startup time, by changing Libtool to not link in indirect deplibs on FreeBSD (and DragonFly). This would be similar to the changes that Debian currently has in their Libtool package. We plan to fix need_all_deplibs=no (it will have a different name then, and it has a couple of known issues at the moment), and then make it available for all systems that support this. (You'd have to relink all libraries once for this, though.) > - It looks like duplicate entries in the library search path > (inherited from LD_LIBRARY_PATH and added via .la) aren't deleted Ah, ok. This is another limitation in libltdl. > - Tracking lt_dlopen/lt_dlclose shows that libraries are being opened > and closed multiple times during startup. Hmm. Surely for FreeBSD this should be a non-issue now; but for systems where we _do_ need to load all deplibs, I'm not sure whether we can get around unloading them in the case that the module itself could not be opened. After all, those deplibs could have been the wrong ones to use in the first place, but we're not able to know until we've tried loading the module itself. Luckily this issue is important on very few systems only, today. > Before joerg pointed me in the right direction, I was looking at > adding caching to lt_dlopenext() or lt_dlopen() but hadn't worked out > how to correctly track reference counts across open/close calls. I think that should be possible, but then again the details may depend on the system in question. I'll try to test on Tru64 when I have some more time at hand. Cheers, Ralf