From owner-freebsd-current@FreeBSD.ORG Thu Jul 9 08:52:48 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C353B106564A for ; Thu, 9 Jul 2009 08:52:48 +0000 (UTC) (envelope-from john.marshall@riverwillow.com.au) Received: from mail1.riverwillow.net.au (mail1.riverwillow.net.au [203.58.93.36]) by mx1.freebsd.org (Postfix) with ESMTP id 547FE8FC0C for ; Thu, 9 Jul 2009 08:52:48 +0000 (UTC) (envelope-from john.marshall@riverwillow.com.au) Received: from rwpc12.mby.riverwillow.net.au (rwpc12.mby.riverwillow.net.au [172.25.24.168]) (authenticated bits=0) by mail1.riverwillow.net.au (8.14.3/8.14.3) with ESMTP id n698qgbE023332 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Thu, 9 Jul 2009 18:52:43 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=riverwillow.com.au; s=m1001; t=1247129563; bh=i3i5GfEJGeByclh9XTWq/dV2tiK4MZW3umxUj8y3NeE=; h=Date:From:To:Subject:Message-ID:References:Mime-Version: Content-Type:In-Reply-To; b=M6/9zSXB5UoUVPukJdAODhGwE7U+tHG0OuxOgjDtBvoy+hnjUvJArYWHatkwdJyU8 0P8MMujXhFXS+Q9stIO2KtFr4XIqrwZKlkFOtEqisJObAGWT+EPArZbuw5v9Yp6mF2 QHLuqNcFbUSLx4Bv+/ZTSS04KaOycO2XYKeY1pks= Received: from rwpc12.mby.riverwillow.net.au (localhost [127.0.0.1]) by rwpc12.mby.riverwillow.net.au (8.14.3/8.14.3) with ESMTP id n698qgKC027993 for ; Thu, 9 Jul 2009 18:52:42 +1000 (AEST) (envelope-from john.marshall@riverwillow.com.au) Received: (from john@localhost) by rwpc12.mby.riverwillow.net.au (8.14.3/8.14.3/Submit) id n698qgwY027992 for freebsd-current@freebsd.org; Thu, 9 Jul 2009 18:52:42 +1000 (AEST) (envelope-from john) Date: Thu, 9 Jul 2009 18:52:42 +1000 From: John Marshall To: freebsd-current@freebsd.org Message-ID: <20090709085242.GC27373@rwpc12.mby.riverwillow.net.au> Mail-Followup-To: freebsd-current@freebsd.org References: <20090709060556.GA27373@rwpc12.mby.riverwillow.net.au> <20090709073054.GB27373@rwpc12.mby.riverwillow.net.au> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oTHb8nViIGeoXxdp" Content-Disposition: inline In-Reply-To: <20090709073054.GB27373@rwpc12.mby.riverwillow.net.au> User-Agent: Mutt/1.4.2.3i OpenPGP: id=A29A84A2; url=http://pki.riverwillow.net.au/pgp/johnmarshall.asc Subject: Re: Process stuck in vmmaps on 8.0-BETA1 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jul 2009 08:52:49 -0000 --oTHb8nViIGeoXxdp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, 09 Jul 2009, 17:30 +1000, John Marshall wrote: > On Thu, 09 Jul 2009, 10:42 +0400, pluknet wrote: > > 2009/7/9 John Marshall : > > > After upgrading... > > > - boot new kernel to single-user > > > - make installworld > > > - make delete-old > > > - make delete-old-libs > > > - mergemaster > > > - reboot > > > > > > I re-built a few of my applications. I noticed a problem with ntpd > > > 4.2.4p7. The build was fine, it started fine, but got stuck in vmmaps > > > and I couldn't kill it. Stopping the operating system appears to be = the > > > only remedy. I have re-built a few times (starting with 'make > > > distclean') just to make sure. > > > > > > UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME CO= MMAND > > > 0 791 1 0 44 0 4944 4920 vmmaps Ds ?? 0:00.01 nt= pd > > > > >=20 > > Can you place here 'procstat -k 791', where 791 is pid of ntpd? > > It'd be nice also if you go through all ddb steps described in > > Debugging Deadlocks chapter of FreeBSD Developers' Handbook. >=20 > Here is some procstat output. I'm just rebuilding the kernel with the > debugging options enabled - not something I've ever done before. >=20 > rwsrv05# procstat 2788 > PID PPID PGID SID TSID THR LOGIN WCHAN EMUL COMM = =20 > 2788 1 2788 2788 0 1 john vmmaps FreeBSD ELF32 ntpd = =20 > rwsrv05# procstat -k 2788 > PID TID COMM TDNAME KSTACK = =20 > 2788 100164 ntpd - mi_switch sleepq_switch sl= eepq_wait _sleep vm_map_unlock_and_wait vm_map_delete vm_map_fixed vm_mmap = mmap syscall Xint0x80_syscall=20 > rwsrv05# procstat -v 2788 > PID START END PRT RES PRES REF SHD FL TP PATH > 2788 0x8048000 0x807e000 r-x 54 60 2 1 CN vn /usr/local/bin/nt= pd > 2788 0x807e000 0x8080000 rw- 2 0 1 0 C- vn /usr/local/bin/nt= pd > 2788 0x8080000 0x8100000 rw- 128 0 1 0 C- df=20 > 2788 0x2807e000 0x280ab000 r-x 45 0 171 75 CN vn /libexec/ld-elf.s= o.1 > 2788 0x280ab000 0x280ad000 rw- 2 0 1 0 C- vn /libexec/ld-elf.s= o.1 > 2788 0x280ad000 0x280c0000 rw- 19 0 1 0 C- df=20 > 2788 0x280c0000 0x280d7000 r-x 23 0 1 0 CN vn /lib/libm.so.5 > 2788 0x280d7000 0x280d8000 r-x 1 0 1 0 CN vn /lib/libm.so.5 > 2788 0x280d8000 0x280d9000 rw- 1 0 1 0 C- vn /lib/libm.so.5 > 2788 0x280d9000 0x28211000 r-x 312 0 1 0 CN vn /lib/libcrypto.so= .5 > 2788 0x28211000 0x28212000 r-x 1 0 1 0 CN vn /lib/libcrypto.so= .5 > 2788 0x28212000 0x2822a000 rw- 24 0 1 0 C- vn /lib/libcrypto.so= .5 > 2788 0x2822a000 0x2822c000 rw- 2 0 1 0 C- df=20 > 2788 0x2822c000 0x28232000 r-x 6 0 1 0 CN vn /lib/libkvm.so.4 > 2788 0x28232000 0x28233000 r-x 1 0 1 0 CN vn /lib/libkvm.so.4 > 2788 0x28233000 0x28234000 rw- 1 0 1 0 C- vn /lib/libkvm.so.4 > 2788 0x28234000 0x2824c000 r-x 24 0 1 0 CN vn /usr/lib/libelf.s= o.1 > 2788 0x2824c000 0x2824d000 r-x 1 0 1 0 CN vn /usr/lib/libelf.s= o.1 > 2788 0x2824d000 0x2824e000 rw- 1 0 1 0 C- vn /usr/lib/libelf.s= o.1 > 2788 0x2824e000 0x28251000 r-x 3 0 15 10 CN vn /usr/lib/librt.so= .1 > 2788 0x28251000 0x28252000 r-x 1 0 1 0 CN vn /usr/lib/librt.so= .1 > 2788 0x28252000 0x28253000 rw- 1 0 1 0 C- vn /usr/lib/librt.so= .1 > 2788 0x28253000 0x28260000 r-x 13 0 1 0 CN vn /lib/libmd.so.4 > 2788 0x28260000 0x28261000 r-x 1 0 1 0 CN vn /lib/libmd.so.4 > 2788 0x28261000 0x28262000 rw- 1 0 1 0 C- vn /lib/libmd.so.4 > 2788 0x28262000 0x28351000 r-x 239 0 1 0 CN vn /lib/libc.so.7 > 2788 0x28351000 0x28352000 r-x 1 0 1 0 CN vn /lib/libc.so.7 > 2788 0x28352000 0x28358000 rw- 6 0 1 0 C- vn /lib/libc.so.7 > 2788 0x28358000 0x2836e000 rw- 22 0 1 0 C- df=20 > 2788 0x2836e000 0x2837a000 --- 0 0 0 0 -- --=20 > 2788 0x28400000 0x28500000 rw- 256 0 1 0 C- df=20 > 2788 0xbfbe0000 0xbfc00000 rwx 32 0 1 0 C- df=20 > rwsrv05#=20 OK, now that I've rebuilt the kernel with the debugging options not commented out, I'm getting a number of 'lock order reversal' messages printed on the console: is that normal? =46rom the Debugging Deadlocks chapter to which I was referred by pluknet (above) it appears that I need to enter 'sysctl debug.kdb.enter=3D1' or 'sysctl debug.kdb.panic=3D1' after I get the process into the desired 'stuck' state. If I enter either of those commands, the system reboots. Now *I'm* stuck. --=20 John Marshall --oTHb8nViIGeoXxdp Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.11 (FreeBSD) iEYEARECAAYFAkpVr9oACgkQw/tAaKKahKIVIQCfflqAk86596cLM/E89flY9Qg5 00kAnRc9WLdLRTa4nhHVLaB4VJC45X7B =aaKn -----END PGP SIGNATURE----- --oTHb8nViIGeoXxdp--