From owner-freebsd-stable@FreeBSD.ORG Tue Nov 11 14:36:38 2008 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 27332106567E for ; Tue, 11 Nov 2008 14:36:38 +0000 (UTC) (envelope-from ken73.chen@gmail.com) Received: from rn-out-0910.google.com (rn-out-0910.google.com [64.233.170.185]) by mx1.freebsd.org (Postfix) with ESMTP id AE8FB8FC2F for ; Tue, 11 Nov 2008 14:36:37 +0000 (UTC) (envelope-from ken73.chen@gmail.com) Received: by rn-out-0910.google.com with SMTP id j71so2184197rne.12 for ; Tue, 11 Nov 2008 06:36:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:in-reply-to:mime-version:content-type:references; bh=Z4QYby6jixbMjAjjHqejrpQWva7ZBfK0gKxaPKZDLnk=; b=mFcHILl909ARLe+aVe7VsThpDpTFZrzghoxDhqQzW7z+08fFw+yHEnNBNbeNJLQEjy H22WRifB1KSVLGUMw7H15Y1FG98vB2aBGf9kJRhkLLtTCvEQWEX9P6DfvHaScfgw53GA gQ/x9qYiRHa/QjIAuo5GkksQcrgKqA3K0DKWQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version :content-type:references; b=SvHewqAwjGMJoZ+3MCzdii6RNcJ3lQ7SKCcPjRdHaDPB+BmlRfnAAhcDjA24pCsxpk xnhg/wlOLrJi9Ri5yaxd+Sz6C459PVucvlfXv5hjNgFeVzKYKAS48WV6lZ7uq00BKRKh pCxXFCAotLwBVdg3lH0/hptI0AeWwtHNp+Ync= Received: by 10.142.251.15 with SMTP id y15mr2986750wfh.126.1226412357409; Tue, 11 Nov 2008 06:05:57 -0800 (PST) Received: by 10.142.232.19 with HTTP; Tue, 11 Nov 2008 06:05:57 -0800 (PST) Message-ID: Date: Tue, 11 Nov 2008 22:05:57 +0800 From: "Ken Chen" To: stable@freebsd.org In-Reply-To: <491726F4.4040808@lozenetz.org> MIME-Version: 1.0 References: <491726F4.4040808@lozenetz.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: php-cgi frozen with sbwait when SMP enable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Nov 2008 14:36:38 -0000 The report from lighttpd looks fine: cache.cached-itmes: 98293 cache.hitrate(%): 96 cache.memory-inuse(KB): 6143 fastcgi.active-requests: 16 fastcgi.backend.0.0.connected: 419008 fastcgi.backend.0.0.died: 0 fastcgi.backend.0.0.disabled: 0 fastcgi.backend.0.0.load: 16 fastcgi.backend.0.0.overloaded: 0 fastcgi.backend.0.load: 17 fastcgi.requests: 419008 But at this moment: web4# ps alx | grep php-cgi | grep -v grep | awk '{print $9}' | sort | uniq -c | sort -n 1 biord 1 wait 2 - 16 sbwait 29 accept web4# ps alx | grep php-cgi | grep -v grep | grep sbwait 65534 61392 61384 1 4 -15 182328 69312 sbwait I< ?? 0:53.39 /usr/local/bin/php-cgi 65534 61399 61384 0 4 -15 182328 71112 sbwait I< ?? 1:09.60 /usr/local/bin/php-cgi 65534 61409 61384 0 4 -15 182328 72812 sbwait I< ?? 1:39.40 /usr/local/bin/php-cgi 65534 61411 61384 0 4 -15 183352 74536 sbwait I< ?? 1:49.08 /usr/local/bin/php-cgi 65534 61412 61384 0 4 -15 183352 74508 sbwait I< ?? 1:33.31 /usr/local/bin/php-cgi 65534 61414 61384 0 4 -15 182328 62860 sbwait I< ?? 0:28.81 /usr/local/bin/php-cgi 65534 61418 61384 0 4 -15 182328 71448 sbwait I< ?? 1:17.56 /usr/local/bin/php-cgi 65534 61426 61384 0 4 -15 183352 60456 sbwait I< ?? 0:22.16 /usr/local/bin/php-cgi 65534 71529 61384 0 4 -15 182328 74144 sbwait I< ?? 0:36.87 /usr/local/bin/php-cgi 65534 71555 61384 0 4 -15 182328 72820 sbwait I< ?? 0:19.19 /usr/local/bin/php-cgi 65534 71556 61384 0 4 -15 182328 74452 sbwait I< ?? 0:38.27 /usr/local/bin/php-cgi 65534 71590 61384 0 4 -15 182328 76828 sbwait I< ?? 0:57.42 /usr/local/bin/php-cgi 65534 71594 61384 0 4 -15 182328 75576 sbwait I< ?? 0:46.50 /usr/local/bin/php-cgi 65534 71595 61384 0 4 -15 182328 84048 sbwait I< ?? 1:52.15 /usr/local/bin/php-cgi 65534 77285 61384 0 4 -15 182328 88280 sbwait S< ?? 0:15.22 /usr/local/bin/php-cgi 65534 77288 61384 3 4 -15 182328 88808 sbwait S< ?? 0:14.43 /usr/local/bin/php-cgi 65534 77317 61384 0 4 -15 182328 88912 sbwait S< ?? 0:12.79 /usr/local/bin/php-cgi 65534 77323 61384 0 4 -15 182328 89140 sbwait S< ?? 0:13.51 /usr/local/bin/php-cgi 65534 77359 61384 6 4 -15 182328 88200 sbwait S< ?? 0:13.04 /usr/local/bin/php-cgi 65534 77372 61384 2 4 -15 182328 89200 sbwait S< ?? 0:12.16 /usr/local/bin/php-cgi 65534 77392 61384 1 4 -15 181304 87200 sbwait S< ?? 0:11.02 /usr/local/bin/php-cgi 65534 77401 61384 1 4 -15 182328 88800 sbwait S< ?? 0:12.49 /usr/local/bin/php-cgi The PID of php-cgi which are less than 71595 are frozen. 2008/11/10 Anton - Valqk > You can try taking look to lighttpd status and fcgi processes status > like this: > server.modules += ( "mod_status" ) > status.status-url = "/server-status" > status.statistics-url = "/sstatus1" > > status.statistics-url gives info for each fastcgi like this: > > fastcgi.active-requests: 0 > fastcgi.backend.backend1.0.connected: 12493970 > fastcgi.backend.backend1.0.died: 0 > fastcgi.backend.backend1.0.disabled: 0 > fastcgi.backend.backend1.0.load: 0 > fastcgi.backend.backend1.0.overloaded: 0 > fastcgi.backend.backend1.load: 1 > fastcgi.requests: 19479062 > > > etc... read what each means on lighttpd site... > pls tell what caused this, it'd be very interesting to me! > > cheers, > valqk. > > Ken Chen wrote: > > I capture something. > > > > Please check the PID 57776. It's CPU time never change since my previous > > mail here. > > > > web4# ps alx | grep php-cgi | grep -v grep | grep sbwait > > 65534 57776 47240 0 4 0 182328 84984 sbwait I ?? 2:02.12 > > /usr/local/bin/php-cgi > > 65534 57801 47240 0 4 0 182328 82408 sbwait I ?? 0:19.97 > > /usr/local/bin/php-cgi > > 65534 57809 47240 0 4 0 182328 84096 sbwait I ?? 1:12.03 > > /usr/local/bin/php-cgi > > 65534 57823 47240 0 4 0 182328 84492 sbwait I ?? 2:04.21 > > /usr/local/bin/php-cgi > > 65534 57833 47240 0 4 0 183352 83316 sbwait I ?? 0:28.62 > > /usr/local/bin/php-cgi > > 65534 57866 47240 0 4 0 182328 79952 sbwait I ?? 0:05.92 > > /usr/local/bin/php-cgi > > 65534 57870 47240 0 4 0 182328 83184 sbwait I ?? 0:56.83 > > /usr/local/bin/php-cgi > > 65534 57871 47240 0 4 0 182328 83388 sbwait I ?? 0:54.96 > > /usr/local/bin/php-cgi > > 65534 57891 47240 0 4 0 182328 84436 sbwait I ?? 1:58.32 > > /usr/local/bin/php-cgi > > 65534 57925 47240 0 4 0 182328 84380 sbwait I ?? 2:03.53 > > /usr/local/bin/php-cgi > > 65534 65944 47240 0 4 0 182328 84184 sbwait I ?? 0:39.97 > > /usr/local/bin/php-cgi > > 65534 65952 47240 0 4 0 182328 84408 sbwait I ?? 0:21.37 > > /usr/local/bin/php-cgi > > 65534 66007 47240 0 4 0 183352 90960 sbwait I ?? 1:16.81 > > /usr/local/bin/php-cgi > > 65534 66014 47240 5 4 0 182328 92748 sbwait S ?? 1:41.23 > > /usr/local/bin/php-cgi > > 65534 66038 47240 1 4 0 182328 91900 sbwait I ?? 1:38.04 > > /usr/local/bin/php-cgi > > 65534 66060 47240 0 4 0 182328 90048 sbwait I ?? 1:15.46 > > /usr/local/bin/php-cgi > > 65534 66078 47240 3 4 0 182328 92224 sbwait S ?? 1:39.66 > > /usr/local/bin/php-cgi > > web4# top -b > > last pid: 70768; load averages: 1.62, 1.65, 1.43 up 4+15:56:06 > > 22:53:48 > > 85 processes: 1 running, 84 sleeping > > > > Mem: 492M Active, 1204M Inact, 218M Wired, 60M Cache, 112M Buf, 27M Free > > Swap: 2019M Total, 20K Used, 2019M Free > > > > > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > > 69544 nobody 1 8 0 203M 38500K nanslp 1 6:31 11.33% php > > 47290 nobody 1 4 0 101M 98M kqread 1 30:42 2.98% > lighttpd > > 66526 nobody 1 4 0 178M 92796K accept 1 1:40 1.12% php-cgi > > 66077 nobody 1 4 0 178M 92512K accept 0 1:49 1.07% php-cgi > > 65921 nobody 1 4 0 178M 92696K accept 0 1:43 0.98% php-cgi > > 65968 nobody 1 4 0 178M 92484K accept 0 1:43 0.93% php-cgi > > 66017 nobody 1 4 0 178M 92444K accept 0 1:50 0.88% php-cgi > > 65979 nobody 1 4 0 178M 92676K accept 1 1:44 0.88% php-cgi > > 66424 nobody 1 4 0 178M 92928K accept 1 1:36 0.88% php-cgi > > 65938 nobody 1 4 0 178M 92336K accept 1 1:52 0.73% php-cgi > > 65951 nobody 1 4 0 178M 92704K accept 0 1:48 0.73% php-cgi > > 66016 nobody 1 4 0 178M 92232K accept 1 1:41 0.73% php-cgi > > 65950 nobody 1 4 0 178M 93192K accept 0 1:51 0.68% php-cgi > > 65999 nobody 1 4 0 178M 92940K accept 1 1:46 0.63% php-cgi > > 66008 nobody 1 4 0 178M 93000K accept 1 1:46 0.63% php-cgi > > 69286 nobody 1 4 0 178M 92208K accept 1 0:37 0.63% php-cgi > > 47289 nobody 1 4 0 73400K 70640K kqread 1 12:02 0.59% > lighttpd > > 65980 nobody 1 4 0 178M 93156K accept 1 1:51 0.59% php-cgi > > 2008/11/7 Ivan Voras > > > > > >> Ken Chen wrote: > >> > >>> Oh.. sorry, I forgot to provide the information of my environment. > >>> > >>> web4# php-cgi -v > >>> PHP 5.2.6 (cgi-fcgi) (built: Nov 2 2008 11:16:30) > >>> Copyright (c) 1997-2008 The PHP Group > >>> Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies > >>> with XCache v1.2.2, Copyright (c) 2005-2007, by mOo > >>> web4# /usr/local/lighttpd/sbin/lighttpd -v > >>> lighttpd-1.4.19 - a light and fast webserver > >>> Build-Date: Sep 1 2008 16:58:51 > >>> web4# uname -a > >>> FreeBSD web4.xxxx.com 7.0-RELEASE-p5 FreeBSD 7.0-RELEASE-p5 #11: Mon > Nov > >>> > >> 3 > >> > >>> 01:10:36 CST 2008 root@web4.xxxx.com:/usr/obj/usr/src/sys/WEB4 > i386 > >>> web4# ps alx | grep php-cgi | grep -v grep | grep sbwait > >>> 65534 57776 47240 0 4 0 182328 84984 sbwait I ?? 2:02.12 > >>> /usr/local/bin/php-cgi > >>> 65534 57801 47240 0 4 0 182328 82408 sbwait I ?? 0:19.97 > >>> /usr/local/bin/php-cgi > >>> 65534 57809 47240 0 4 0 182328 84096 sbwait I ?? 1:12.03 > >>> /usr/local/bin/php-cgi > >>> 65534 57823 47240 0 4 0 182328 84492 sbwait I ?? 2:04.21 > >>> /usr/local/bin/php-cgi > >>> 65534 57833 47240 0 4 0 183352 83316 sbwait I ?? 0:28.62 > >>> /usr/local/bin/php-cgi > >>> 65534 57866 47240 0 4 0 182328 79952 sbwait I ?? 0:05.92 > >>> /usr/local/bin/php-cgi > >>> 65534 57870 47240 0 4 0 182328 83184 sbwait I ?? 0:56.83 > >>> /usr/local/bin/php-cgi > >>> 65534 57871 47240 0 4 0 182328 83388 sbwait I ?? 0:54.96 > >>> /usr/local/bin/php-cgi > >>> 65534 57891 47240 0 4 0 182328 84436 sbwait I ?? 1:58.32 > >>> /usr/local/bin/php-cgi > >>> 65534 57925 47240 0 4 0 182328 84380 sbwait I ?? 2:03.53 > >>> /usr/local/bin/php-cgi > >>> 65534 65944 47240 0 4 0 182328 84184 sbwait I ?? 0:39.97 > >>> /usr/local/bin/php-cgi > >>> 65534 65952 47240 0 4 0 182328 84408 sbwait I ?? 0:21.37 > >>> /usr/local/bin/php-cgi > >>> > >> This does seem a bit unusual, but seeing that your execution times are > >> not null it might that the PHP servers are actually doing some useful > >> work. You should have a mixture of various states in PHP - do they show > >> up in top? > >> > >> My own example is: > >> > >> last pid: 77421; load averages: 2.82, 2.59, 2.13 > >> up > >> 55+16:58:49 15:48:16 > >> 209 processes: 2 running, 206 sleeping, 1 zombie > >> CPU: 49.8% user, 0.0% nice, 2.8% system, 0.0% interrupt, 47.4% idle > >> Mem: 1493M Active, 1583M Inact, 278M Wired, 139M Cache, 112M Buf, 505M > Free > >> Swap: 4500M Total, 416M Used, 4084M Free, 9% Inuse > >> > >> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > >> 75863 www 1 4 0 162M 50020K sbwait 3 2:54 36.77% > php-cgi > >> 76830 www 1 103 0 156M 41556K CPU2 3 1:28 36.77% > php-cgi > >> 76834 www 1 4 0 163M 56628K sbwait 0 2:23 33.59% > php-cgi > >> 76019 www 1 4 0 150M 38948K accept 3 3:12 20.56% > php-cgi > >> 76825 www 1 4 0 158M 42912K accept 2 1:21 18.16% > php-cgi > >> 76846 www 1 4 0 162M 42600K sbwait 1 1:07 14.36% > php-cgi > >> 76835 www 1 4 0 151M 39948K accept 2 1:28 12.60% > php-cgi > >> 76829 www 1 4 0 150M 36564K sbwait 2 1:46 2.98% > php-cgi > >> > >> This is unusually high load, a spike, for this server but it has many > >> cores and it's stable. It's also running 7.1-PRERELEASE. > >> > >> > >> > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org > " > > > > > >