Date: Mon, 2 Jun 2003 15:24:18 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: Gordon Tetlow <gordont@gnf.org> Cc: Dag-Erling Smorgrav <des@ofug.org> Subject: Re: Making a dynamically-linked root Message-ID: <200306022224.h52MOIwj002729@apollo.backplane.com> References: <20030602171942.GA87863@roark.gnf.org> <xzp4r3844eb.fsf@flood.ping.uio.no> <20030602202947.GE87863@roark.gnf.org> <xzpznl02nry.fsf@flood.ping.uio.no> <200306022125.h52LPhhc002291@apollo.backplane.com> <20030602214956.GG87863@roark.gnf.org>
next in thread | previous in thread | raw e-mail | index | archive | help
:Actually, it was a diskless boot, so it was in the system cache. =3D) I
:know this is a rigged demo, but the point is the same, yes, it's slower,
:but we also have a huge gain from going to a dynamically linked world.
:It would also serve as encouragement to get things like pre-binding and
:caching working.
:
:-gordon
Ah, but you are still waiting on 'disk I/O'... it just happens to be
*network* disk I/O, so it doesn't matter if it's in the server's
cache or not.
A lot of the delay is due to the client program stalling until the page
is faulted in over the network (and not doing any other work in the mean
time), then running for a few cycles and stalling again waiting for the
next random page to be faulted in.
Another big issue with the diskless code is the path cache. Whenever
a shell script runs a program using a relative path (like 'ls' instead of
'/bin/ls'), it tries to stat the program file for each path element in
the path. With a local disk the local system's name cache is coherent
and these operations are nearly instantanious. Over NFS, however, a
lot of retesting of the same paths are done over the network over and
over again, leading to a massive perceived slow down, and even retesting
a good path like /bin/sh often generates NFS traffic looking up "/bin/sh"
over and over again.
For example, if in one window you start a tcpdump and monitor port
2049 (typically nfsd), and in another window you run /bin/sh, you
will see at least 3 NFS lookups. If you exit the shell and run it again
you will see the same 3 NFS lookups again. And again, and again.
This alone is probably responsible for most of the rc script slowdown.
It is probably all the path lookups on the dynamic link libraries at
program startup that is causing the problem, not exec() per-say. If
you think running /bin/sh produces a lot of NFS traffic, try running
'/usr/bin/nm' without any arguments and look at the NFS traffic.
/usr/bin/nm, being a dynamic executable, will do no less then 14
uncacheable synchronous NFS operations just to deal with its shared
libraries.
-Matt
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200306022224.h52MOIwj002729>