From owner-freebsd-stable@FreeBSD.ORG Mon Jan 25 09:24:29 2010 Return-Path: Delivered-To: FreeBSD-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B2461065670 for ; Mon, 25 Jan 2010 09:24:29 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (skuns.zoral.com.ua [91.193.166.194]) by mx1.freebsd.org (Postfix) with ESMTP id E6C0F8FC12 for ; Mon, 25 Jan 2010 09:24:28 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id o0P9OBT3072222 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 25 Jan 2010 11:24:11 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3) with ESMTP id o0P9OBHo018034; Mon, 25 Jan 2010 11:24:11 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.3/8.14.3/Submit) id o0P9OBoP018033; Mon, 25 Jan 2010 11:24:11 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 25 Jan 2010 11:24:11 +0200 From: Kostik Belousov To: Peter Jeremy Message-ID: <20100125092411.GK3877@deviant.kiev.zoral.com.ua> References: <201001242023.o0OKNj5p044592@server.vk2pj.dyndns.org> <20100125080659.GN31243@server.vk2pj.dyndns.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4wkndigzIeYF6Hbg" Content-Disposition: inline In-Reply-To: <20100125080659.GN31243@server.vk2pj.dyndns.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: bzeeb+freebsd+lor@zabbadoz.net, FreeBSD-stable@freebsd.org Subject: Re: New zfs/bufwait LOR X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2010 09:24:29 -0000 --4wkndigzIeYF6Hbg Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 25, 2010 at 07:07:00PM +1100, Peter Jeremy wrote: > I had the following crop up recently in 8-STABLE/amd64 from end of > November. It's been reported as kern/143184. Basically, page containing the buffer for read(2) is swapped out. This causes page fault in copyout(9) and entry into vm subsystem while zfs vnode lock is held. If the buffer is backed by e.g. UFS vnode instead of anonymous memory, you would get UFS/zfs LOR. The problem is generic, I am working on the solution in collaboration with Peter Holm, basing on the Jeff Roberson idea. >=20 > lock order reversal: > 1st 0xffffff002f7fb270 zfs (zfs) @ /usr/src/sys/kern/vfs_vnops.c:533 > 2nd 0xffffff80803a26e0 bufwait (bufwait) @ /usr/src/sys/vm/vm_pager.c:311 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a > _witness_debugger() at _witness_debugger+0x2c > witness_checkorder() at witness_checkorder+0x66f > __lockmgr_args() at __lockmgr_args+0x475 > initpbuf() at initpbuf+0xb9 > getpbuf() at getpbuf+0xdc > swap_pager_getpages() at swap_pager_getpages+0x1aa > vm_fault() at vm_fault+0x5f7 > trap_pfault() at trap_pfault+0x128 > trap() at trap+0x379 > calltrap() at calltrap+0x8 > --- trap 0xc, rip =3D 0xffffffff8049497b, rsp =3D 0xffffff809a427830, rbp= =3D 0xffffff809a4278b0 --- > copyout() at copyout+0x3b > dmu_read_uio() at dmu_read_uio+0x98 > zfs_freebsd_read() at zfs_freebsd_read+0x56f > VOP_READ_APV() at VOP_READ_APV+0x44 > vn_read() at vn_read+0x149 > dofileread() at dofileread+0xa1 > kern_readv() at kern_readv+0x60 > read() at read+0x55 > syscall() at syscall+0x1ac > Xfast_syscall() at Xfast_syscall+0xe1 > --- syscall (3, FreeBSD ELF64, read), rip =3D 0x8008ce86c, rsp =3D 0x7fff= fffeb718, rbp =3D 0x805b41d18 --- >=20 > --=20 > Peter Jeremy --4wkndigzIeYF6Hbg Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAktdYzsACgkQC3+MBN1Mb4g2TgCghDryQ12imDm7FSbLDYBWFJK8 yoMAnRy2gJ2KKVwzXLmUM5i1TDl2dEHJ =xCUI -----END PGP SIGNATURE----- --4wkndigzIeYF6Hbg--