From owner-freebsd-stable@FreeBSD.ORG Tue Oct 4 18:20:44 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 625F2106564A; Tue, 4 Oct 2011 18:20:44 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from smtp-out2.tiscali.nl (smtp-out2.tiscali.nl [195.241.79.177]) by mx1.freebsd.org (Postfix) with ESMTP id AC3D18FC0A; Tue, 4 Oct 2011 18:20:43 +0000 (UTC) Received: from [212.182.167.131] (helo=sjakie.klop.ws) by smtp-out2.tiscali.nl with esmtp (Exim) (envelope-from ) id 1RB9be-0002p3-Nr; Tue, 04 Oct 2011 20:20:42 +0200 Received: from 212-182-167-131.ip.telfort.nl (localhost [127.0.0.1]) by sjakie.klop.ws (Postfix) with ESMTP id 48EE7C4; Tue, 4 Oct 2011 20:20:34 +0200 (CEST) Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: "Mikolaj Golub" References: <20110918045413.GA63773@DataIX.net> <20110918053901.GA31617@icarus.home.lan> <86d3eydsmf.fsf@kopusha.home.net> Date: Tue, 04 Oct 2011 20:20:33 +0200 MIME-Version: 1.0 From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/11.51 (FreeBSD) Content-Transfer-Encoding: quoted-printable Cc: Chris Torek , "freebsd-stable@freebsd.org" , cperciva@freebsd.org, Jeremy Chadwick Subject: Re: /usr/bin/script eating 100% cpu with portupgrade and xargs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2011 18:20:44 -0000 On Tue, 04 Oct 2011 13:15:24 +0200, Mikolaj Golub =20 wrote: > On Sun, Sep 18, 2011 at 1:58 PM, Mikolaj Golub =20 > wrote: >> >> On Sun, 18 Sep 2011 08:47:13 +0200 Ronald Klop wrote: >> >> RK> On Sun, 18 Sep 2011 07:39:01 +0200, Jeremy Chadwick >> RK> wrote: >> >> >> On Sun, Sep 18, 2011 at 12:54:13AM -0400, Jason Hellenthal wrote: >> >>> On Sun, Sep 18, 2011 at 01:49:15AM +0200, Ronald Klop wrote: >> >>> > Hi, >> >>> > >> >>> > I'm running portupgrade in screen to update all the ports for >> >>> > 9-BETA2/9-CURRENT on amd64. While doing this script eats 100% =20 >> cpu. >> >>> > Because portupgrade -fa crashed I'm running this command to =20 >> update the >> >>> > remaining non-updates ports. >> >>> > find /var/db/pkg -name +DESC -mtime +2 |cut -d / -f 5 | xargs >> >>> time nice -n >> >>> > 20 portupgrade -f >> >>> > >> >>> > The output of truss -p `pgrep script` is this: >> >>> > clock_gettime(13,{1316301104.000000000 }) =3D 0 (0x0) >> >>> > select(5,{0 4},0x0,0x0,{30.000000 }) =3D 1 (0x1) >> >>> > read(0,0x7fffffffcdf0,1024) =3D 0 (0x0) >> >>> > write(4,0x7fffffffcdf0,0) =3D 0 (0x0) >> >>> > clock_gettime(13,{1316301104.000000000 }) =3D 0 (0x0) >> >>> > select(5,{0 4},0x0,0x0,{30.000000 }) =3D 1 (0x1) >> >>> > read(0,0x7fffffffcdf0,1024) =3D 0 (0x0) >> >>> > write(4,0x7fffffffcdf0,0) =3D 0 (0x0) >> >>> > clock_gettime(13,{1316301104.000000000 }) =3D 0 (0x0) >> >>> > select(5,{0 4},0x0,0x0,{30.000000 }) =3D 1 (0x1) >> >>> > read(0,0x7fffffffcdf0,1024) =3D 0 (0x0) >> >>> > write(4,0x7fffffffcdf0,0) =3D 0 (0x0) >> >>> > clock_gettime(13,{1316301104.000000000 }) =3D 0 (0x0) >> >>> > select(5,{0 4},0x0,0x0,{30.000000 }) =3D 1 (0x1) >> >>> > read(0,0x7fffffffcdf0,1024) =3D 0 (0x0) >> >>> > write(4,0x7fffffffcdf0,0) =3D 0 (0x0) >> >>> > >> >>> > So it is really fast in reading and writing 0 bytes most of the= =20 >> time. >> >>> > >> >>> > I also found >> >>> http://web.archiveorange.com/archive/v/6ETvLvjo60Gj9geAUAb6 >> >>> > and I think I am better of by rewriting my command so =20 >> stdin/stdout is >> >>> > still the terminal. Although the link is a couple of years old. >> >>> > >> >>> > Is this known? Can somebody explain me why my xargs command is >> >>> not working >> >>> > well? >> >>> > >> >>> >> >>> Are you absolutely sure that its script(1) causing this ? 100% CP= U =20 >> usage >> >>> has been a known side effect of screen(1) for quite some time. =20 >> Rebuild >> >>> it and try again. >> >> >> >> Jason's referring to this, I believe: >> >> =20 >> http://www.freebsd.org/cgi/cvsweb.cgi/ports/sysutils/screen/Makefile#r= ev1.55 >> >> >> >> To clarify the what the commit message means: it does not mean =20 >> "when the >> >> package is installed the installation takes up 100% CPU". It mean= s >> >> "once the package is installed and screen is used, screen takes up= =20 >> 100% >> >> CPU". I know because I've seen this behaviour in the past (one of= =20 >> the >> >> many, many reasons I build ports from source). >> >> >> >> However: >> >> =20 >> http://www.freebsd.org/cgi/cvsweb.cgi/ports/sysutils/screen/Makefile#r= ev1.78 >> >> >> >> So: If a binary package is being installed through your above >> >> portupgrade command, and you're seeing this problem, then it sound= s =20 >> to >> >> me like commit revision 1.78 is a regression and NO_PACKAGE should= =20 >> be >> >> put back into place + packages removed from all mirrors. >> >> >> >> There are many reasons to not use GNU screen at all, or if you mus= t =20 >> have >> >> something like it, use tmux. I recently had to provide an analysi= s =20 >> of >> >> how GNU screen destroys one's terminal[1]; so if the above problem= =20 >> turns >> >> out to be caused by GNU screen as well, I'll just add it to my >> >> ever-growing list of reasons the software should be nuked from =20 >> orbit. >> >> >> >> Otherwise, if this turns out to be a problem with portupgrade =20 >> (which you >> >> found some evidence supporting such), then the solution is simple:= =20 >> stop >> >> using portupgrade, use portmaster (if it lacks things you need ask= =20 >> Doug >> >> Barton, he's incredibly receptive to adding new features/fixing =20 >> things). >> >> Two databases that aren't compatible, ruby shims, and other crap =3D= =20 >> not >> >> worth it. Think the database ordeal is long over =20 >> with/fixed/whatever? >> >> It isn't[2]. >> >> >> >> [1]: >> >> =20 >> http://lists.freebsd.org/pipermail/freebsd-stable/2011-June/063052.htm= l >> >> [2]: >> >> =20 >> http://www.dslreports.com/forum/r26304856-FreeBSD-defining-portmaster-= alias >> >> >> >> RK> I have a repeatable test. Run top in a window and this command in= =20 >> another. >> RK> $ echo test | script /tmp/script-test sleep 1000 >> RK> Script started, output file is /tmp/script-test >> RK> test >> >> RK> PID USERNAME THR PRI NICE SIZE RES STATE C TIME >> RK> CPU COMMAND >> RK> 29656 ronald 1 103 0 12324K 1244K CPU4 4 1:03 >> RK> 100.00% script >> >> RK> So it has nothing to do with portupgrade or screen. The output of >> RK> truss -p29656 is the same as posted previously. >> >> I believe the behaviour is after this commit: >> >> http://svnweb.freebsd.org/base?view=3Drevision&revision=3D125848 >> >> I think we should skip select on STDIN after reading EOF from it, like= =20 >> in the >> patch below. > > For the record. The issue has been fixed in CURRENT and the fix has > been merged to STABLE. > > Thanks Kostik and Chris for their comments and suggestions. > I saw the commits. Thanks a lot for the quick response. Ronald.