From owner-freebsd-current@freebsd.org Tue Dec 8 19:36:16 2015 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D86C99D5ACA for ; Tue, 8 Dec 2015 19:36:16 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id C5EDF1C1F for ; Tue, 8 Dec 2015 19:36:16 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: by mailman.ysv.freebsd.org (Postfix) id C54539D5AC9; Tue, 8 Dec 2015 19:36:16 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4E049D5AC8 for ; Tue, 8 Dec 2015 19:36:16 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (mail.nomadlogic.org [IPv6:2607:f2f8:a098::4]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B55FB1C1E for ; Tue, 8 Dec 2015 19:36:16 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (localhost [127.0.0.1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.nomadlogic.org (Postfix) with ESMTPS id 06EDE125EE1 for ; Tue, 8 Dec 2015 11:36:10 -0800 (PST) Received: from pop.rubicorp.com (unknown [72.34.113.100]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.nomadlogic.org (Postfix) with ESMTPSA id DC293125EBA for ; Tue, 8 Dec 2015 11:36:09 -0800 (PST) To: current@freebsd.org From: Pete Wright Subject: LOR On AMD64 hosted by KVM hypervisor X-Enigmail-Draft-Status: N1110 Message-ID: <56673129.4090207@nomadlogic.org> Date: Tue, 8 Dec 2015 11:36:09 -0800 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 19:36:16 -0000 Hey All, I am seeing a repeated LOR on r291495 that is pretty reproducible. This happens right after the system boots: lock order reversal: 1st 0xfffffe03e37fa920 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3476 2nd 0xfffff80024c72200 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:281 stack backtrace: #0 0xffffffff80a7b2e0 at witness_debugger+0x70 #1 0xffffffff80a7b1e1 at witness_checkorder+0xe71 #2 0xffffffff80a28ab2 at _sx_xlock+0x72 #3 0xffffffff80cc0a5d at ufsdirhash_add+0x3d #4 0xffffffff80cc390f at ufs_direnter+0x62f #5 0xffffffff80ccccd3 at ufs_makeinode+0x5f3 #6 0xffffffff80cc881d at ufs_create+0x2d #7 0xffffffff80fb2ed1 at VOP_CREATE_APV+0xf1 #8 0xffffffff80ae3568 at vn_open_cred+0x2f8 #9 0xffffffff80adc8ec at kern_openat+0x25c #10 0xffffffff80e6a4fe at amd64_syscall+0x2de #11 0xffffffff80e4972b at Xfast_syscall+0xfb Here is the system info: % uname -ar FreeBSD bsd-current.trp-srd.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r291495: Mon Nov 30 23:14:34 UTC 2015 root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 The server itself is a VM running under the KVM hypervisor. I am currently rebuilding world+kernel now. The base OS image is from the latest 11-CURRENT snapshot, and I have been able to reproduce this on several hypervisors. Has anyone else seen this? Cheers, -pete -- Pete Wright pete@nomadlogic.org twitter => @nomadlogicLA