From owner-freebsd-stable@FreeBSD.ORG Fri Feb 5 11:32:07 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 74163106566B for ; Fri, 5 Feb 2010 11:32:07 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from host.omnilan.net (host.omnilan.net [62.245.232.135]) by mx1.freebsd.org (Postfix) with ESMTP id E5D748FC1A for ; Fri, 5 Feb 2010 11:32:06 +0000 (UTC) Received: from titan.flintsbach.schmalzbauer.de (titan.flintsbach.schmalzbauer.de [172.21.1.150]) (authenticated bits=0) by host.omnilan.net (8.13.8/8.13.8) with ESMTP id o15BW2Xr062652 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 5 Feb 2010 12:32:05 +0100 (CET) (envelope-from h.schmalzbauer@omnilan.de) Message-ID: <4B6C01AB.5030500@omnilan.de> Date: Fri, 05 Feb 2010 12:31:55 +0100 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Thunderbird 2.0.0.23 (X11/20090906) MIME-Version: 1.0 To: freebsd-stable X-Enigmail-Version: 0.95.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig120A922E0E2D3AAF348CCE25" Subject: 8.0-RELEASE hangs with lighttpd, unionfs related? Some traces included X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Feb 2010 11:32:07 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig120A922E0E2D3AAF348CCE25 Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: quoted-printable Hello, when I start lighttpd at boot time, the system half-locks in a way, that = any process, which accesses /usr/local/etc stalls. It's also impossible=20 to shut down. /usr/local/etc is unionfs mounted. I compiled a kernel with debug options. When mounting unionfs at boot time, here's the firt LOR with trace: lock order reversal: 1st 0xffffff00018b47f8 unionfs (unionfs) @=20 /usr/src/sys/fs/unionfs/union_subr.c:356 2nd 0xffffff00018d9d80 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2188 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x49 witness_checkorder() at witness_checkorder+0x7ea __lockmgr_args() at __lockmgr_args+0xcbd ffs_lock() at ffs_lock+0x8c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x50 vrele() at vrele+0x120 unionfs_noderem() at unionfs_noderem+0x1c4 unionfs_reclaim() at unionfs_reclaim+0x11 vgonel() at vgonel+0xf1 vrecycle() at vrecycle+0x58 unionfs_inactive() at uniougen2.2: at usbus2 nfs_inactive+ukbd0: 0 on usbus2 x20 vinactive() at vinactive+0x6b vput() at vput+0x216 kern_statatkbd1 at ukbd0 _vnhook() at kern_statat_vnhook+0xe9 kern_statat() at kern_statat+0x15 stat() at stat+0x22 syscall() at syscall+0x1af Xfast_syscall() at Xfast_syscall+0xe1 --- suhid0: y on usbus2 scall (188, FreeBSD ELF64, stat), rip =3D 0x8009a055c, rsp =3D=20 0x7fffffffe5b8, rbp =3D 0x800b312c0 --- KDB: enter: witness_checkorder [thread pid 27 tid 100068 ] Stopped at kdb_enter+0x3d: movq $0,0x4c04dc(%rip) Tracing pid 27 tid 100068 td 0xffffff00016fe720 kdb_enter() at kdb_enter+0x3d witness_checkorder() at witness_checkorder+0x7ea __lockmgr_args() at __lockmgr_args+0xcbd ffs_lock() at ffs_lock+0x8c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x50 vrele() at vrele+0x120 unionfs_noderem() at unionfs_noderem+0x1c4 unionfs_reclaim() at unionfs_reclaim+0x11 vgonel() at vgonel+0xf1 vrecycle() at vrecycle+0x58 unionfs_inactive() at unionfs_inactive+0x20 vinactive() at vinactive+0x6b vput() at vput+0x216 kern_statat_vnhook() at kern_statat_vnhook+0xe9 kern_statat() at kern_statat+0x15 stat() at stat+0x22 syscall() at syscall+0x1af Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (188, FreeBSD ELF64, stat), rip =3D 0x8009a055c, rsp =3D=20 0x7fffffffe5b8, rbp =3D 0x800b312c0 - Like mentioned, there is that strange problem with lighttpd started at=20 boot time. Other /urs/local/etc/rc.d startups don't lead to a=20 /usr/local/etc deadlock. Unfortunately I don't get any panic or anything else when the hang happen= s. How can I aquire more information? It's no problem to log in and to do everything else outside=20 /usr/local/etc... =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Here's a LOR at shutdown with trace: lock order reversal: 1st 0xffffff0001bc2098 ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1200 2nd 0xffffff0001bc1ba8 devfs (devfs) @=20 /usr/src/sys/ufs/ffs/ffs_vfsops.c:1194 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x49 witness_checkorder() at witness_checkorder+0x7ea __lockmgr_args() at __lockmgr_args+0xcbd vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x50 ffs_flushfiles() at ffs_flushfiles+0x93 ffs_unmount() at ffs_unmount+0x48 dounmount() at dounmount+0x2ac vfs_unmountall() at vfs_unmountall+0x54 boot() at boot+0x814 mkdumpheader() at mkdumpheader syscall() at syscall+0x1af Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (55, FreeBSD ELF64, reboot), rip =3D 0x40829c, rsp =3D=20 0x7fffffffe738, rbp =3D 0x402290 --- KDB: enter: witness_checkorder [thread pid 1 tid 100002 ] Stopped at kdb_enter+0x3d: movq $0,0x4c04dc(%rip) Tracing pid 1 tid 100002 td 0xffffff0001310ab0 kdb_enter() at kdb_enter+0x3d witness_checkorder() at witness_checkorder+0x7ea __lockmgr_args() at __lockmgr_args+0xcbd vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x50 ffs_flushfiles() at ffs_flushfiles+0x93 ffs_unmount() at ffs_unmount+0x48 dounmount() at dounmount+0x2ac vfs_unmountall() at vfs_unmountall+0x54 boot() at boot+0x814 mkdumpheader() at mkdumpheader syscall() at syscall+0x1af Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (55, FreeBSD ELF64, reboot), rip =3D 0x40829c, rsp =3D=20 0x7fffffffe738, rbp =3D 0x402290 --- Any Help highly appreciated! Thanks, -Harry --------------enig120A922E0E2D3AAF348CCE25 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iEYEARECAAYFAktsAbIACgkQLDqVQ9VXb8hGswCeJdmgieDOlI8iMCcqfTAye4aD Fe0AoMyVddRUyYig0/nXaQqus4Nl7KUz =dMun -----END PGP SIGNATURE----- --------------enig120A922E0E2D3AAF348CCE25--