Date: Tue, 21 Mar 2006 05:24:15 +1100 From: Peter Jeremy <peterjeremy@optushome.com.au> To: Eric Schuele <e.schuele@computer.org> Cc: freebsd-hackers@freebsd.org Subject: Re: repeatedly opening the same .so(s) is slow? Message-ID: <20060320182415.GD747@turion.vk2pj.dyndns.org> In-Reply-To: <441EDD35.3080105@computer.org> References: <441EDD35.3080105@computer.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 2006-Mar-20 10:49:57 -0600, Eric Schuele wrote: >I have a port (gnucash) which takes 3-4 minutes to open on a 2.6GHz >machine. It used to take 15-20 seconds till all of the libtool changes. It takes 15 minutes on a -current Athlon XP-1800 and about 2 minutes on a 2.2GHz AMD64 running -stable. >I have no idea if the symptom is related to libtool or not. I initially thought it was libtool related but now I'm uncertain. I didn't just upgrade libtool, I upgraded quite a few other ports at the same time. On the not-libtool side, ade@ says that he hasn't seen this behaviour with other libtool/libltdl ports and I've found that guile include it's own libltdl code (based on libtool). I'm not sure if this is gnucash specific or affects other guile applications. >Using truss, I can see that gnucash/guile is trying to open a dozen or >two files, repeatedly. It fails attempting to open it the first few >times everytime it tries to access it, because it is traversing the >LD_LIBRARY_PATH: Worse than that, it's expanding LD_LIBRARY_PATH using additional paths embedded in the .la files that it's opening. >Now I said a dozen or two files repeatedly. It is 12-20 files maybe... >but it is attempting to open them *hundreds of thousands of times*! It >goes on and on and on... I took a complete ktrace of the startup and there are 24e6 NAMI events with the top files tested 2,000,000 times. > I have >thought of placing symlinks in the folder(s) where it first looks for >any given file, to make sure it finds the file... but this does not seem >quite right either. It's definitely a hack. I tried something like this and it didn't help much. The code still wants to open libraries multiple times. I've been looking at adding caching to lt_dlopenext() and my first attempt went much faster but blew up because I wasn't correctly handling open/close/open sequences (libm is opened and then closed 42,000 times). I think this is the way forward but need to find the time to understand ltdl.[ch] (~4800 lines). >What I'm wondering is.... what is the lists opinion on how to best fix >this type of a problem. Is this even the cause of my long startup? Any system calls involving opening pathnames are expensive, even with the namei cache. Having 4 orders of magnitude too many is a destinct problem. >I have spoken with one or two of the gnucash devs, they seem to think >this is unique to FreeBSD, meaning they have not seen this problem on >any other platform. They said it might have to do with how FreeBSD >handles how files are opened up many times recursively!? Possibly Linux can more efficiently handle opening a non-existent file but the underlying problem is that there are far too many system calls being executed during the gnucash startup. It would be interesting to get a truss of gnuash starting on another OS (unfortunately, I don't have access to any Linux systems) and/or some other guile applications. >If there is a more appropriate list, please let me know. -ports may be better. -- Peter Jeremy
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060320182415.GD747>