Date: Sun, 18 Sep 2011 13:58:32 +0300 From: Mikolaj Golub <trociny@freebsd.org> To: "Ronald Klop" <ronald-freebsd8@klop.yi.org> Cc: "freebsd-stable@freebsd.org" <freebsd-stable@freebsd.org>, Jeremy Chadwick <freebsd@jdc.parodius.com>, cperciva@freebsd.org Subject: Re: /usr/bin/script eating 100% cpu with portupgrade and xargs Message-ID: <86d3eydsmf.fsf@kopusha.home.net> In-Reply-To: <op.v1zrszht8527sy@pinky> (Ronald Klop's message of "Sun, 18 Sep 2011 08:47:13 %2B0200") References: <op.v1y8gdtf8527sy@pinky> <20110918045413.GA63773@DataIX.net> <20110918053901.GA31617@icarus.home.lan> <op.v1zrszht8527sy@pinky>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
RK> On Sun, 18 Sep 2011 07:39:01 +0200, Jeremy Chadwick
RK> <freebsd@jdc.parodius.com> wrote:
>> On Sun, Sep 18, 2011 at 12:54:13AM -0400, Jason Hellenthal wrote:
>>> On Sun, Sep 18, 2011 at 01:49:15AM +0200, Ronald Klop wrote:
>>> > Hi,
>>> >
>>> > I'm running portupgrade in screen to update all the ports for
>>> > 9-BETA2/9-CURRENT on amd64. While doing this script eats 100% cpu.
>>> > Because portupgrade -fa crashed I'm running this command to update the
>>> > remaining non-updates ports.
>>> > find /var/db/pkg -name +DESC -mtime +2 |cut -d / -f 5 | xargs
>>> time nice -n
>>> > 20 portupgrade -f
>>> >
>>> > The output of truss -p `pgrep script` is this:
>>> > clock_gettime(13,{1316301104.000000000 }) = 0 (0x0)
>>> > select(5,{0 4},0x0,0x0,{30.000000 }) = 1 (0x1)
>>> > read(0,0x7fffffffcdf0,1024) = 0 (0x0)
>>> > write(4,0x7fffffffcdf0,0) = 0 (0x0)
>>> > clock_gettime(13,{1316301104.000000000 }) = 0 (0x0)
>>> > select(5,{0 4},0x0,0x0,{30.000000 }) = 1 (0x1)
>>> > read(0,0x7fffffffcdf0,1024) = 0 (0x0)
>>> > write(4,0x7fffffffcdf0,0) = 0 (0x0)
>>> > clock_gettime(13,{1316301104.000000000 }) = 0 (0x0)
>>> > select(5,{0 4},0x0,0x0,{30.000000 }) = 1 (0x1)
>>> > read(0,0x7fffffffcdf0,1024) = 0 (0x0)
>>> > write(4,0x7fffffffcdf0,0) = 0 (0x0)
>>> > clock_gettime(13,{1316301104.000000000 }) = 0 (0x0)
>>> > select(5,{0 4},0x0,0x0,{30.000000 }) = 1 (0x1)
>>> > read(0,0x7fffffffcdf0,1024) = 0 (0x0)
>>> > write(4,0x7fffffffcdf0,0) = 0 (0x0)
>>> >
>>> > So it is really fast in reading and writing 0 bytes most of the time.
>>> >
>>> > I also found
>>> http://web.archiveorange.com/archive/v/6ETvLvjo60Gj9geAUAb6
>>> > and I think I am better of by rewriting my command so stdin/stdout is
>>> > still the terminal. Although the link is a couple of years old.
>>> >
>>> > Is this known? Can somebody explain me why my xargs command is
>>> not working
>>> > well?
>>> >
>>>
>>> Are you absolutely sure that its script(1) causing this ? 100% CPU usage
>>> has been a known side effect of screen(1) for quite some time. Rebuild
>>> it and try again.
>>
>> Jason's referring to this, I believe:
>> http://www.freebsd.org/cgi/cvsweb.cgi/ports/sysutils/screen/Makefile#rev1.55
>>
>> To clarify the what the commit message means: it does not mean "when the
>> package is installed the installation takes up 100% CPU". It means
>> "once the package is installed and screen is used, screen takes up 100%
>> CPU". I know because I've seen this behaviour in the past (one of the
>> many, many reasons I build ports from source).
>>
>> However:
>> http://www.freebsd.org/cgi/cvsweb.cgi/ports/sysutils/screen/Makefile#rev1.78
>>
>> So: If a binary package is being installed through your above
>> portupgrade command, and you're seeing this problem, then it sounds to
>> me like commit revision 1.78 is a regression and NO_PACKAGE should be
>> put back into place + packages removed from all mirrors.
>>
>> There are many reasons to not use GNU screen at all, or if you must have
>> something like it, use tmux. I recently had to provide an analysis of
>> how GNU screen destroys one's terminal[1]; so if the above problem turns
>> out to be caused by GNU screen as well, I'll just add it to my
>> ever-growing list of reasons the software should be nuked from orbit.
>>
>> Otherwise, if this turns out to be a problem with portupgrade (which you
>> found some evidence supporting such), then the solution is simple: stop
>> using portupgrade, use portmaster (if it lacks things you need ask Doug
>> Barton, he's incredibly receptive to adding new features/fixing things).
>> Two databases that aren't compatible, ruby shims, and other crap = not
>> worth it. Think the database ordeal is long over with/fixed/whatever?
>> It isn't[2].
>>
>> [1]:
>> http://lists.freebsd.org/pipermail/freebsd-stable/2011-June/063052.html
>> [2]:
>> http://www.dslreports.com/forum/r26304856-FreeBSD-defining-portmaster-alias
>>
RK> I have a repeatable test. Run top in a window and this command in another.
RK> $ echo test | script /tmp/script-test sleep 1000
RK> Script started, output file is /tmp/script-test
RK> test
RK> PID USERNAME THR PRI NICE SIZE RES STATE C TIME
RK> CPU COMMAND
RK> 29656 ronald 1 103 0 12324K 1244K CPU4 4 1:03
RK> 100.00% script
RK> So it has nothing to do with portupgrade or screen. The output of
RK> truss -p29656 is the same as posted previously.
I believe the behaviour is after this commit:
http://svnweb.freebsd.org/base?view=revision&revision=125848
I think we should skip select on STDIN after reading EOF from it, like in the
patch below.
--
Mikolaj Golub
[-- Attachment #2 --]
Index: usr.bin/script/script.c
===================================================================
--- usr.bin/script/script.c (revision 225648)
+++ usr.bin/script/script.c (working copy)
@@ -53,6 +53,7 @@ static const char sccsid[] = "@(#)script.c 8.1 (Be
#include <libutil.h>
#include <paths.h>
#include <signal.h>
+#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
@@ -86,6 +87,7 @@ main(int argc, char *argv[])
char ibuf[BUFSIZ];
fd_set rfd;
int flushtime = 30;
+ bool stdin_closed;
aflg = kflg = 0;
while ((ch = getopt(argc, argv, "aqkt:")) != -1)
@@ -161,10 +163,12 @@ main(int argc, char *argv[])
tvp = NULL;
start = time(0);
- FD_ZERO(&rfd);
+ stdin_closed = false;
for (;;) {
+ FD_ZERO(&rfd);
FD_SET(master, &rfd);
- FD_SET(STDIN_FILENO, &rfd);
+ if (!stdin_closed)
+ FD_SET(STDIN_FILENO, &rfd);
if (flushtime > 0) {
tv.tv_sec = flushtime;
tv.tv_usec = 0;
@@ -176,8 +180,13 @@ main(int argc, char *argv[])
cc = read(STDIN_FILENO, ibuf, BUFSIZ);
if (cc < 0)
break;
- if (cc == 0)
+ if (cc == 0) {
+ /*
+ * Send EOF to the program being scripted.
+ */
(void)write(master, ibuf, 0);
+ stdin_closed = true;
+ }
if (cc > 0) {
(void)write(master, ibuf, cc);
if (kflg && tcgetattr(master, &stt) >= 0 &&
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?86d3eydsmf.fsf>