From owner-cvs-src@FreeBSD.ORG Wed Jun 18 09:11:40 2008 Return-Path: Delivered-To: cvs-src@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2FF81065687 for ; Wed, 18 Jun 2008 09:11:40 +0000 (UTC) (envelope-from peter@wemm.org) Received: from hu-out-0506.google.com (hu-out-0506.google.com [72.14.214.233]) by mx1.freebsd.org (Postfix) with ESMTP id 0A6418FC36 for ; Wed, 18 Jun 2008 09:11:39 +0000 (UTC) (envelope-from peter@wemm.org) Received: by hu-out-0506.google.com with SMTP id 34so7922623hue.8 for ; Wed, 18 Jun 2008 02:11:38 -0700 (PDT) Received: by 10.82.150.15 with SMTP id x15mr21273bud.23.1213780298278; Wed, 18 Jun 2008 02:11:38 -0700 (PDT) Received: by 10.82.162.18 with HTTP; Wed, 18 Jun 2008 02:11:38 -0700 (PDT) Message-ID: Date: Wed, 18 Jun 2008 02:11:38 -0700 From: "Peter Wemm" To: "Robert Watson" In-Reply-To: <20080618091320.F18654@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200806170633.m5H6XMJH084600@repoman.freebsd.org> <20080617134828.GA30076@zim.MIT.EDU> <20080617140600.GE1176@hoeg.nl> <4857D508.8070907@FreeBSD.org> <20080617170755.GA30958@zim.MIT.EDU> <20080618091320.F18654@fledge.watson.org> Cc: Maxim Sobolev , src-committers@freebsd.org, Ed Schouten , cvs-src@freebsd.org, cvs-all@freebsd.org, David Xu , David Schultz Subject: Re: cvs commit: src/include Makefile spawn.h unistd.h src/lib/libc/gen Makefile.inc Symbol.map exec.3 exec.c posix_spawn.c X-BeenThere: cvs-src@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2008 09:11:40 -0000 On Wed, Jun 18, 2008 at 1:16 AM, Robert Watson wrote: > On Tue, 17 Jun 2008, David Schultz wrote: > >> On Tue, Jun 17, 2008, Maxim Sobolev wrote: >>> >>> Ed Schouten wrote: >>>> >>>> * David Schultz wrote: >>>>> >>>>> I have no objections to this, but doesn't it defeat the whole purpose >>>>> to implement posix_spawn() as a library function that just calls fork/exec? >>>> >>>> When (if?) applications start to use posix_spawn() we may decide to move >>>> it into the kernel at any time. It should be okay for now. >>> >>> Are there any benefits of doing it in the kernel vs. doing it via >>> fork+exec? >> >> The only reason spawn exists is to better support platforms where fork is >> slow, so implementing it in terms of fork/exec defeats the purpose and >> potentially tricks configure scripts into making incorrect assumptions about >> performance tradeoffs. However, maybe spawn would still be useful if >> misguided application writers used it for other reasons (e.g., to make it >> easier to port Win32 apps), and I'm guessing that's why it was added. >> Implementing it in the kernel has disadvantages, too; in particular, it >> would add a lot of complexity for gains that are likely to be minimal in >> FreeBSD. > > Apple's experience has been somewhat to the contrary -- while the > architecture varies some by OS release, one of the persisting performance > problems they were seeing was the cost of fork()+execve() from applications > with very large numbers of shared libraries, plugins, memory mappings, etc. > Currently, they address this by having a process launch applications "by > proxy" as a result of IPC requests instead of forking and execing, but you > might reasonably argue that the problem is with the fork()+execve() model. It gets significant for FreeBSD too. Here's some food for thought. Imagine % cc -static -o forkspeed forkspeed.c % time ./forkspeed 10000 0.020u 0.592s 0:00.55 110.9% 170+2652k 0+0io 0pf+0w % cc -o forkspeed forkspeed.c % time ./forkspeed 10000 0.090u 1.239s 0:01.25 105.6% 5+196k 0+0io 0pf+0w % cc -o forkspeed forkspeed.c -lncurses -lutil -lcrypto -lssl % time ./forkspeed 10000 0.070u 1.785s 0:02.09 88.5% 4+154k 0+0io 0pf+0w -static vs dynamic: 2.27 times slower. -static vs extra dynamic libs: 3.80 times slower. % cat forkspeed.c #include #include #include #include int main(int ac, char *av[]) { int n = atoi(av[1]); int i; for (i = 0; i < n; i++) { if (fork() == 0) _exit(0); wait(0); } return (0); } No exec(), just measuring the overhead of fork(). Just for amusement, the same with vfork(): peter@overcee[1:48am]~-130> cc -static -o forkspeed forkspeed.c peter@overcee[1:48am]~-131> time ./forkspeed 10000 0.007u 0.097s 0:00.11 81.8% 161+2503k 0+0io 0pf+0w peter@overcee[1:48am]~-132> cc -o forkspeed forkspeed.c peter@overcee[1:49am]~-133> time ./forkspeed 10000 0.008u 0.170s 0:00.11 100.0% 4+139k 0+0io 0pf+0w peter@overcee[1:49am]~-134> cc -o forkspeed forkspeed.c -lncurses -lutil -lcrypto -lssl peter@overcee[1:49am]~-135> time ./forkspeed 10000 0.000u 0.106s 0:00.10 100.0% 4+158k 0+0io 0pf+0w Essentially the same regardless of libraries. vfork is 5 times faster for -static, 11 times faster for regular dynamic, and 20 times faster for extra libraries. So.. if something auto-detects posix_spawn(), which uses vfork(), it would be a win compared to the usual fork()/exec(). A small win, but still a win. It would have to do a lot of iterations to add up. Incidently, this is why /usr/bin/make and /usr/bin/gcc are statically linked. /bin/sh used to be, but isn't so that ~user can use nsswitch. For amusement, think of kde and gnome with all their libraries. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell