From owner-freebsd-stable@FreeBSD.ORG Sun Sep 18 11:25:34 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93FE6106566C; Sun, 18 Sep 2011 11:25:34 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from fep16.mx.upcmail.net (fep16.mx.upcmail.net [62.179.121.36]) by mx1.freebsd.org (Postfix) with ESMTP id 63E788FC13; Sun, 18 Sep 2011 11:25:33 +0000 (UTC) Received: from edge04.upcmail.net ([192.168.13.239]) by viefep16-int.chello.at (InterMail vM.8.01.02.02 201-2260-120-106-20100312) with ESMTP id <20110918112531.NXDX1962.viefep16-int.chello.at@edge04.upcmail.net>; Sun, 18 Sep 2011 13:25:31 +0200 Received: from pinky ([213.93.232.119]) by edge04.upcmail.net with edge id aBRS1h03h2bDWHx04BRUjZ; Sun, 18 Sep 2011 13:25:31 +0200 X-SourceIP: 213.93.232.119 Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: "Mikolaj Golub" References: <20110918045413.GA63773@DataIX.net> <20110918053901.GA31617@icarus.home.lan> <86d3eydsmf.fsf@kopusha.home.net> Date: Sun, 18 Sep 2011 13:25:26 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Ronald Klop" Message-ID: In-Reply-To: <86d3eydsmf.fsf@kopusha.home.net> User-Agent: Opera Mail/11.51 (Win32) X-Cloudmark-Analysis: v=1.1 cv=jqrf5qXv5urL2URZ3a9OklEvUdrM81ZtB3GWlHLPCVk= c=1 sm=0 a=lm4eusMn8AwA:10 a=4xaea3bvHo0A:10 a=bgpUlknNv7MA:10 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=QycZ5dHgAAAA:8 a=a14uRV0-AAAA:8 a=JPUQt-R5AAAA:8 a=iSxftX8V9Fd4APHUQKkA:9 a=_UVwWIs6cbXYAebWmmUA:7 a=CjuIK1q_8ugA:10 a=Ea5LKd3xFwUA:10 a=QrMgEKT2GQAA:10 a=SV7veod9ZcQA:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 Cc: "freebsd-stable@freebsd.org" , Jeremy Chadwick , cperciva@freebsd.org Subject: Re: /usr/bin/script eating 100% cpu with portupgrade and xargs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Sep 2011 11:25:34 -0000 On Sun, 18 Sep 2011 12:58:32 +0200, Mikolaj Golub wrote: > > On Sun, 18 Sep 2011 08:47:13 +0200 Ronald Klop wrote: > > RK> On Sun, 18 Sep 2011 07:39:01 +0200, Jeremy Chadwick > RK> wrote: > > >> On Sun, Sep 18, 2011 at 12:54:13AM -0400, Jason Hellenthal wrote: > >>> On Sun, Sep 18, 2011 at 01:49:15AM +0200, Ronald Klop wrote: > >>> > Hi, > >>> > > >>> > I'm running portupgrade in screen to update all the ports for > >>> > 9-BETA2/9-CURRENT on amd64. While doing this script eats 100% cpu. > >>> > Because portupgrade -fa crashed I'm running this command to > update the > >>> > remaining non-updates ports. > >>> > find /var/db/pkg -name +DESC -mtime +2 |cut -d / -f 5 | xargs > >>> time nice -n > >>> > 20 portupgrade -f > >>> > > >>> > The output of truss -p `pgrep script` is this: > >>> > clock_gettime(13,{1316301104.000000000 }) = 0 (0x0) > >>> > select(5,{0 4},0x0,0x0,{30.000000 }) = 1 (0x1) > >>> > read(0,0x7fffffffcdf0,1024) = 0 (0x0) > >>> > write(4,0x7fffffffcdf0,0) = 0 (0x0) > >>> > clock_gettime(13,{1316301104.000000000 }) = 0 (0x0) > >>> > select(5,{0 4},0x0,0x0,{30.000000 }) = 1 (0x1) > >>> > read(0,0x7fffffffcdf0,1024) = 0 (0x0) > >>> > write(4,0x7fffffffcdf0,0) = 0 (0x0) > >>> > clock_gettime(13,{1316301104.000000000 }) = 0 (0x0) > >>> > select(5,{0 4},0x0,0x0,{30.000000 }) = 1 (0x1) > >>> > read(0,0x7fffffffcdf0,1024) = 0 (0x0) > >>> > write(4,0x7fffffffcdf0,0) = 0 (0x0) > >>> > clock_gettime(13,{1316301104.000000000 }) = 0 (0x0) > >>> > select(5,{0 4},0x0,0x0,{30.000000 }) = 1 (0x1) > >>> > read(0,0x7fffffffcdf0,1024) = 0 (0x0) > >>> > write(4,0x7fffffffcdf0,0) = 0 (0x0) > >>> > > >>> > So it is really fast in reading and writing 0 bytes most of the > time. > >>> > > >>> > I also found > >>> http://web.archiveorange.com/archive/v/6ETvLvjo60Gj9geAUAb6 > >>> > and I think I am better of by rewriting my command so > stdin/stdout is > >>> > still the terminal. Although the link is a couple of years old. > >>> > > >>> > Is this known? Can somebody explain me why my xargs command is > >>> not working > >>> > well? > >>> > > >>> > >>> Are you absolutely sure that its script(1) causing this ? 100% CPU > usage > >>> has been a known side effect of screen(1) for quite some time. > Rebuild > >>> it and try again. > >> > >> Jason's referring to this, I believe: > >> > http://www.freebsd.org/cgi/cvsweb.cgi/ports/sysutils/screen/Makefile#rev1.55 > >> > >> To clarify the what the commit message means: it does not mean "when > the > >> package is installed the installation takes up 100% CPU". It means > >> "once the package is installed and screen is used, screen takes up > 100% > >> CPU". I know because I've seen this behaviour in the past (one of > the > >> many, many reasons I build ports from source). > >> > >> However: > >> > http://www.freebsd.org/cgi/cvsweb.cgi/ports/sysutils/screen/Makefile#rev1.78 > >> > >> So: If a binary package is being installed through your above > >> portupgrade command, and you're seeing this problem, then it sounds > to > >> me like commit revision 1.78 is a regression and NO_PACKAGE should be > >> put back into place + packages removed from all mirrors. > >> > >> There are many reasons to not use GNU screen at all, or if you must > have > >> something like it, use tmux. I recently had to provide an analysis > of > >> how GNU screen destroys one's terminal[1]; so if the above problem > turns > >> out to be caused by GNU screen as well, I'll just add it to my > >> ever-growing list of reasons the software should be nuked from orbit. > >> > >> Otherwise, if this turns out to be a problem with portupgrade (which > you > >> found some evidence supporting such), then the solution is simple: > stop > >> using portupgrade, use portmaster (if it lacks things you need ask > Doug > >> Barton, he's incredibly receptive to adding new features/fixing > things). > >> Two databases that aren't compatible, ruby shims, and other crap = > not > >> worth it. Think the database ordeal is long over > with/fixed/whatever? > >> It isn't[2]. > >> > >> [1]: > >> > http://lists.freebsd.org/pipermail/freebsd-stable/2011-June/063052.html > >> [2]: > >> > http://www.dslreports.com/forum/r26304856-FreeBSD-defining-portmaster-alias > >> > > RK> I have a repeatable test. Run top in a window and this command in > another. > RK> $ echo test | script /tmp/script-test sleep 1000 > RK> Script started, output file is /tmp/script-test > RK> test > > RK> PID USERNAME THR PRI NICE SIZE RES STATE C TIME > RK> CPU COMMAND > RK> 29656 ronald 1 103 0 12324K 1244K CPU4 4 1:03 > RK> 100.00% script > > RK> So it has nothing to do with portupgrade or screen. The output of > RK> truss -p29656 is the same as posted previously. > > I believe the behaviour is after this commit: > > http://svnweb.freebsd.org/base?view=revision&revision=125848 > > I think we should skip select on STDIN after reading EOF from it, like > in the > patch below. > It is a while since I programmed C, but why will writing 0 bytes give the reader an end-of-file? Shouldn't the fd be closed to indicate end-of-file? Ronald.