From owner-freebsd-stable@FreeBSD.ORG Thu Jun 12 08:18:34 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E5D7106567E; Thu, 12 Jun 2008 08:18:34 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.freebsd.org (Postfix) with ESMTP id A28F98FC22; Thu, 12 Jun 2008 08:18:33 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from inchoate.gsoft.com.au (inchoate.gsoft.com.au [203.31.81.30]) (authenticated bits=0) by cain.gsoft.com.au (8.13.8/8.13.8) with ESMTP id m5C8IUiR096022 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 12 Jun 2008 17:48:31 +0930 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: Jeremy Chadwick Date: Thu, 12 Jun 2008 17:48:26 +0930 User-Agent: KMail/1.9.7 References: <4846B64F.4090700@minibofh.org> <200806121635.36998.doconnor@gsoft.com.au> <20080612072812.GA35851@eos.sc1.parodius.com> In-Reply-To: <20080612072812.GA35851@eos.sc1.parodius.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2561461.soqbAt2eX5"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200806121748.28133.doconnor@gsoft.com.au> X-Spam-Score: -3.977 () ALL_TRUSTED,BAYES_00 X-Scanned-By: MIMEDefang 2.63 on 203.31.81.10 Cc: freebsd-apache@freebsd.org, freebsd-stable@freebsd.org Subject: Re: apachectl gracefult causes Signal 11 crash after 6.3 to 7.0 upgrade [SOLVED] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2008 08:18:34 -0000 --nextPart2561461.soqbAt2eX5 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Thu, 12 Jun 2008, Jeremy Chadwick wrote: > > Yeah.. I have rebuilt lots of stuff, although not libc. > > Huh? Sorry, I meant that I have to explicitly rebuilt it since I did a=20 buildworld to make sure it wasn't fubar'd somehow. I haven't done that mainly because I find it extremely unlikely it would=20 only break Apache in this manner but nothing else. I might rebuild it to get debug symbols though.. > > This machine has build world, kernel, KDE, etc.. I am pretty sure > > the hardware is OK as none of the builds had an issue. > > libc is part of world. *Every* program relies (is linked with) on > libc. Yes, sorry for my confusing turn of phrase! :) > > #10 0x08060c1f in main (argc=3D-716130182, argv=3D0x1) at > > http_main.c:5883 > > I can't say much about this, but I'm willing to bet it's the result > of some Apache + PHP weirdness. I've never known gdb on FreeBSD to > be as reliable/useful as, say, on Linux or Solaris. Always > odd/strange things happening with gdb on FreeBSD. Yeah :( > > I tried truss and it seemed to be taking a long time (5-10 minutes) > > and generating a lot of seemingly identical logging :( > > Okay, let's backtrack here. > > The OP states that he can induce a segfault of httpd when doing > "apachectl graceful". Is that the exact problem you're seeing, or > are you seeing problems where PHP/Apache segfaults during operation?=20 > I just want to be clear. > > If the latter, then truss "generating lots of seemingly identical > logging" is probably expected. I'm guessing it's select() or poll() > or something related to kqueue/kevent, as it'd be waiting for I/O on > the HTTP socket. You'd have to submit the HTTP request to the PHP > script to get it to crash. > > In either case, you may have to resort to using ktrace + kdump, which > may or may not help narrow this down. > > Use "ktrace -i -t+ httpd -X" (I hope that'll work; I'm not sure if > ktrace allows you to pass arguments to a command), which will start > populating a file called ktrace.out. You should then do the > "apachectl graceful" in another window (or if the latter, submit the > HTTP request), and ktrace may exit when the segfault happens (I'm not > sure about this; it may sit there indefinitely). > > In the case it doesn't exit, and you've confirmed the core happened > (check "dmesg"), you should ^C the ktrace and then do "ktrace -C" > just to be sure nothing got wedged. > > You'll then have to use kdump to decode the contents of ktrace.out. =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart2561461.soqbAt2eX5 Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.8 (FreeBSD) iD8DBQBIUNvU5ZPcIHs/zowRAnXaAKCTsHVjsPNFNTxTVR1R/nQ+CU2eUQCeIfzn kNajhS04+47Gqfl3OK80WkU= =hN6U -----END PGP SIGNATURE----- --nextPart2561461.soqbAt2eX5--