From owner-freebsd-bugs@FreeBSD.ORG Thu Dec 19 04:50:01 2013
Return-Path: <owner-freebsd-bugs@FreeBSD.ORG>
Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
(No client certificate requested)
by hub.freebsd.org (Postfix) with ESMTPS id 1F63CE04
for <freebsd-bugs@smarthost.ysv.freebsd.org>;
Thu, 19 Dec 2013 04:50:01 +0000 (UTC)
Received: from freefall.freebsd.org (freefall.freebsd.org
[IPv6:2001:1900:2254:206c::16:87])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by mx1.freebsd.org (Postfix) with ESMTPS id EE8421815
for <freebsd-bugs@smarthost.ysv.freebsd.org>;
Thu, 19 Dec 2013 04:50:00 +0000 (UTC)
Received: from freefall.freebsd.org (localhost [127.0.0.1])
by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id rBJ4o0GN058989
for <freebsd-bugs@freefall.freebsd.org>; Thu, 19 Dec 2013 04:50:00 GMT
(envelope-from gnats@freefall.freebsd.org)
Received: (from gnats@localhost)
by freefall.freebsd.org (8.14.7/8.14.7/Submit) id rBJ4o0CI058988;
Thu, 19 Dec 2013 04:50:00 GMT (envelope-from gnats)
Resent-Date: Thu, 19 Dec 2013 04:50:00 GMT
Resent-Message-Id: <201312190450.rBJ4o0CI058988@freefall.freebsd.org>
Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer)
Resent-To: freebsd-bugs@FreeBSD.org
Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Kurt Lidl <lidl@pix.net>
Received: from mx1.freebsd.org (mx1.freebsd.org
[IPv6:2001:1900:2254:206a::19:1])
(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
(No client certificate requested)
by hub.freebsd.org (Postfix) with ESMTPS id 786AFD83
for <freebsd-gnats-submit@FreeBSD.org>; Thu, 19 Dec 2013 04:40:41 +0000 (UTC)
Received: from oldred.freebsd.org (oldred.freebsd.org
[IPv6:2001:1900:2254:206a::50:4])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.freebsd.org (Postfix) with ESMTPS id 5901217C6
for <freebsd-gnats-submit@FreeBSD.org>; Thu, 19 Dec 2013 04:40:41 +0000 (UTC)
Received: from oldred.freebsd.org ([127.0.1.6])
by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id rBJ4ee6c092944
for <freebsd-gnats-submit@FreeBSD.org>; Thu, 19 Dec 2013 04:40:40 GMT
(envelope-from nobody@oldred.freebsd.org)
Received: (from nobody@localhost)
by oldred.freebsd.org (8.14.5/8.14.5/Submit) id rBJ4eeT6092920;
Thu, 19 Dec 2013 04:40:40 GMT (envelope-from nobody)
Message-Id: <201312190440.rBJ4eeT6092920@oldred.freebsd.org>
Date: Thu, 19 Dec 2013 04:40:40 GMT
From: Kurt Lidl <lidl@pix.net>
To: freebsd-gnats-submit@FreeBSD.org
X-Send-Pr-Version: www-3.1
Subject: kern/184985: gmirror panic on deactivated mirror
X-BeenThere: freebsd-bugs@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Bug reports <freebsd-bugs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-bugs>,
<mailto:freebsd-bugs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-bugs/>
List-Post: <mailto:freebsd-bugs@freebsd.org>
List-Help: <mailto:freebsd-bugs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-bugs>,
<mailto:freebsd-bugs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 19 Dec 2013 04:50:01 -0000
>Number: 184985
>Category: kern
>Synopsis: gmirror panic on deactivated mirror
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Thu Dec 19 04:50:00 UTC 2013
>Closed-Date:
>Last-Modified:
>Originator: Kurt Lidl
>Release: 9.2-RELEASE / 10.0-RC2
>Organization:
>Environment:
FreeBSD b524-fbsd10rc2.pix.net 10.0-RC2 FreeBSD 10.0-RC2 #0 r259404: Sun Dec 15 08:18:20 UTC 2013 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD b524-fbsd9_2.pix.net 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep 26 22:50:31 UTC 2013 root@bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
>Description:
I have verified the following problem on both a virtual machine running 9.2-RELEASE,
as well as a virtual machine running 10.0-RC2. (This problem was originally spotted
on a real machine, but it's been narrowed down using just virtual machines.)
The machine panics when 'gmirror status' is run after a gmirror is completely
deactivated.
The machine MUST have >1 CPU for the panic to occur. With only a single processor on
the machine, the panic will not occur.
Panic looks like this under 10.0-RC2:
root@b524-fbsd10rc2:/var/crash # kgdb /boot/kernel/kernel vmcore.0
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...
Unread portion of the kernel message buffer:
GEOM_MIRROR: Device scratch: provider ada0p3 disconnected.
GEOM_MIRROR: Device scratch: provider mirror/scratch destroyed.
GEOM_MIRROR: Device scratch destroyed.
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address = 0x378
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff808b78c6
stack pointer = 0x28:0xfffffe001ddfea10
frame pointer = 0x28:0xfffffe001ddfeab0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 13 (g_event)
trap number = 12
panic: page fault
cpuid = 1
KDB: stack backtrace:
#0 0xffffffff808e7d60 at kdb_backtrace+0x60
#1 0xffffffff808af845 at panic+0x155
#2 0xffffffff80c8e612 at trap_fatal+0x3a2
#3 0xffffffff80c8e8e9 at trap_pfault+0x2c9
#4 0xffffffff80c8e076 at trap+0x5e6
#5 0xffffffff80c75312 at calltrap+0x8
#6 0xffffffff808b7442 at _sx_xlock+0x62
#7 0xffffffff81a371bf at g_mirror_dumpconf+0x12f
#8 0xffffffff8081a35c at g_conf_specific+0x14c
#9 0xffffffff8081acd6 at g_run_events+0x166
#10 0xffffffff8088191a at fork_exit+0x9a
#11 0xffffffff80c7584e at fork_trampoline+0xe
Uptime: 33s
Dumping 78 out of 487 MB:..21%..41%..61%..82%
Reading symbols from /boot/kernel/zfs.ko.symbols...done.
Loaded symbols for /boot/kernel/zfs.ko.symbols
Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
Loaded symbols for /boot/kernel/opensolaris.ko.symbols
Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done.
Loaded symbols for /boot/kernel/geom_mirror.ko.symbols
#0 doadump (textdump=<value optimized out>) at pcpu.h:219
219 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) bt
#0 doadump (textdump=<value optimized out>) at pcpu.h:219
#1 0xffffffff808af4c0 in kern_reboot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:447
#2 0xffffffff808af884 in panic (fmt=<value optimized out>)
at /usr/src/sys/kern/kern_shutdown.c:754
#3 0xffffffff80c8e612 in trap_fatal (frame=<value optimized out>,
eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:882
#4 0xffffffff80c8e8e9 in trap_pfault (frame=0xfffffe001ddfe960, usermode=0)
at /usr/src/sys/amd64/amd64/trap.c:699
#5 0xffffffff80c8e076 in trap (frame=0xfffffe001ddfe960)
at /usr/src/sys/amd64/amd64/trap.c:463
#6 0xffffffff80c75312 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:232
#7 0xffffffff808b78c6 in _sx_xlock_hard (sx=0xfffff80018321240,
tid=18446735277652013056, opts=<value optimized out>,
file=0x2beff0 <Address 0x2beff0 out of bounds>, line=0)
at /usr/src/sys/kern/kern_sx.c:556
#8 0xffffffff808b7442 in _sx_xlock (sx=0x2beff0, opts=0,
file=<value optimized out>, line=2879472) at sx.h:152
#9 0xffffffff81a371bf in g_mirror_dumpconf (sb=0xfffff80018310540,
indent=0xffffffff80ea9f7d "\t", gp=<value optimized out>,
cp=<value optimized out>, pp=<value optimized out>)
at /usr/src/sys/modules/geom/geom_mirror/../../../geom/mirror/g_mirror.c:320---Type <return> to continue, or q <return> to quit---
2
#10 0xffffffff8081a35c in g_conf_specific (sb=0xfffff80018310540, mp=0x0,
gp=0x0, pp=0x0, cp=0x0) at /usr/src/sys/geom/geom_dump.c:238
#11 0xffffffff8081acd6 in g_run_events () at /usr/src/sys/geom/geom_event.c:257
#12 0xffffffff8088191a in fork_exit (
callout=0xffffffff8081c8e0 <g_event_procbody>, arg=0x0,
frame=0xfffffe001ddfec00) at /usr/src/sys/kern/kern_fork.c:995
#13 0xffffffff80c7584e in fork_trampoline ()
at /usr/src/sys/amd64/amd64/exception.S:606
#14 0x0000000000000000 in ?? ()
Current language: auto; currently minimal
(kgdb)
>How-To-Repeat:
Assuming, an unused partition on /dev/ada0p3:
while :
do
gmirror label -v scratch /dev/ada0p3
newfs /dev/mirror/scratch
mount /dev/mirror/scratch /mnt
umount -f /mnt
gmirror deactivate scratch /dev/ada0p3
gmirror status scratch
done
The machine will panic on the final 'gmirror status scratch' command.
This happens 100% of the time on the first run through the loop on 9.2-RELEASE.
On 10.0-RC2, it may take two or three iterations of the loop before the panic.
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted: