Date: Thu, 23 Apr 1998 01:58:31 -0500 (EST) From: "John S. Dyson" <toor@dyson.iquest.net> To: peter.jeremy@alcatel.com.au (Peter Jeremy) Cc: freebsd-security@FreeBSD.ORG Subject: Re: Static vs. dynamic linking Message-ID: <199804230658.BAA08577@dyson.iquest.net> In-Reply-To: <199804230345.NAA20055@gsms01.alcatel.com.au> from Peter Jeremy at "Apr 23, 98 01:45:19 pm"
next in thread | previous in thread | raw e-mail | index | archive | help
> On Wed, 22 Apr 1998 18:01:46 -0500 (EST), "John S. Dyson" <dyson@FreeBSD.ORG> wrote: > >I haven't been watching this discussion carefully, but here > >are the costs of shared libs: > > > > fork() performance is slower. > This is virtually solely due to the cost of duplicating the larger > page tables. In reality (and given the COW semantics, which delay > the actual page table duplication), how much slower is forking a > dynamic executable together with its shared libraries than forking > a static executable (where much of the library code is embedded in > the executable anyway)? > All I can say, is that it is enough slower to make it undesirable if you can avoid shared libs. I have made measurements, and the problems are real. Alot of time has been spent to make our kernel performance as good as it is (which is really good.) As I remember, it seems like forks for dynamic libs is about 2X slower or so. > > > Shared libs often require additional page table pages > > to map them. > I'm not familiar with the low-level VM, so the answer to this might > be obvious: Why can't the page tables associated with shared libraries > be shared between processes? I know they won't get inherited, but > can't mmap() share page tables as well as the underlying objects? > We use different locations in different images. There are numerous problems with what you suggest, and I have worked on it before. It is possible, but the symbol resolution still kills you. About 1/2+ of the overhead is in user mode, and the other 1/2- is in the kernel. Of course, the page table page sharing would be mostly only effective on .text segments, due to the need for seperate .data mappings in every program image. It seems that the mgmt overhead to maintain consistancy would be nearly break-even with our extremely quick mapping mechanisms. We have some very aggressive and efficient optimizations in the kernel for both the fork and exec cases. Actually, there is little more overhead than is measured on lmbench. We don't play the "defer, and hide from lmbench" game. We incur mapping cost up front on our executables, since our mapping during exec is about 100X faster than when faulting. It is tricky to get it right so it is faster, but it really does make a system wide difference. The exec overhead is relatively more huge than the fork overhead. > > > > Shared libs don't help .text sharing, > I don't quite follow this one. Isn't being able to share a single > copy of libc.so across (almost) every process on the system better > than sharing the part of libc.a compiled into /bin/sh across several > dozen sh processes, with a second copy of much the same code being > shared across several dozen csh processes (for example)? > If you have multiple invocations of the same process, we still share text. The problem is that .text is sparse and diffuse in the shared libs, and there is alot of cache footprint overhead, and additional dynamic page usage due to the sparseness. Shared libs really don't help in the case of multiple (>3-4) static and dynamic process instances, and mostly hurt because of this. > > >Programs where it is likely slightly advantageous to link shared: > > cc1, cc1plus, > Actually, for programs where the text+data is very large compared to > the libraries (the above, plus emacs and maybe perl(*)) the space > savings incurred using shared libraries are probably outweighed by the > performance gains through making them static. > I typically like the toolchain to be static, but that is my personal preference. On a single user system, it is probably a wash, but on a multi-user server, you are probably very right. Since we are talking about servers -- the various large (non-dlopen) items should probably be static. > > Also, I think John has ignored some of the advantages of shared > libraries: > > - Fixing bugs in libraries is much easier - just replace a single > libc.so and the bug is fixed in all the programs that use it. This > mightn't be much of an issue for the hackers amongst us (who > regularly rebuild their entire systems), but will be an issue > as we try to expand our user base to less knowledgable people > (and people who don't want to have to do a `make world' every > time a CERT advisory comes out). > - Run-time control (and extendability) of configuration. Examples > are Sun's name service switch and volume management, as well as > the idea of plug-in authentication modules for login (where this > thread started). > > (*) In reality, perl5 needs access to the dynamic loader anyway and > so isn't as good a choice. > We are a server OS, and our system *is* benchmarked. Lets not kill our already good (hard to achieve) performance. If we decide to go entirely dynamic, then the job isn't done in the performance area, and alot of work is needed. IMO, The only reasonable decision at this time is to allow those who have special applications rebuild their systems as needed, which they likely do anyway. We need to build for our predominant user base who don't normally need to build-world. I see our (non-make world) market as servers, by far, followed by workstations. Embedded product is mostly a (make world) market. How many of our problems can be fixed in the library(s) alone, anyway? John To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe security" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199804230658.BAA08577>