From owner-freebsd-current@FreeBSD.ORG Tue Sep 27 20:00:29 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7FC6110656B3 for ; Tue, 27 Sep 2011 20:00:29 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay01.ispgateway.de (smtprelay01.ispgateway.de [80.67.18.13]) by mx1.freebsd.org (Postfix) with ESMTP id 126C08FC16 for ; Tue, 27 Sep 2011 20:00:28 +0000 (UTC) Received: from [78.34.155.34] (helo=fabiankeil.de) by smtprelay01.ispgateway.de with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.68) (envelope-from ) id 1R8dpL-0007pF-KI for freebsd-current@freebsd.org; Tue, 27 Sep 2011 22:00:27 +0200 Date: Tue, 27 Sep 2011 22:00:15 +0200 From: Fabian Keil To: freebsd-current@freebsd.org Message-ID: <20110927220015.375ac343@fabiankeil.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/kTIY0JWgN5S_DzofhXnkCee"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 Subject: Fatal trap 12: page fault while in kernel mode -- Stopped at atomic_subtract_int+0x4 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Sep 2011 20:00:29 -0000 --Sig_/kTIY0JWgN5S_DzofhXnkCee Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable I pretty reproducible get the following (handtranscribed) panic when sending an zfs snapshot to geli provider based on an USB stick that disappears (due to a bug, or because it's unplugged):=20 Fatal trap 12: page fault while in kernel mode cpuid =3D 0: apic id =3D 00 fault virtual address =3D 0x288 fault code =3D supervisor write data, page not present instruction pointer =3D 0x20:0xffffffff808e2984 stack pointer =3D 0x28:0xffffff800023fba0 frame pointer =3D 0x28:0xffffff800023fbb0 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 13 (g_up) [ thread pid 13 tid 100010 ] Stopped at atomic_subtract_int+0x4: lock subl %esi,(%rdi) db> where Tracing pid 13 tid 100010 td 0xfffffe00027998c0 atomic_subtract_int() at atomic_subtract_int+0x4 g_io_schdule_up() at g_io_schedule_up+0xa6 g_up_procbody() at g_up_procbody+0x5c fork_exit() at fork_exit+0x11f fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800023fd00, rbp 0 --- It seems to be important that ZFS is actually writing to the stick. If the stick is unplugged while the operation is stalled for other reasons, this particular panic doesn't seem to occur. While I end up in the debugger, dumping core doesn't work and produces a double fault and a bunch of duplicated messages (again handtranscribed): db> dump Dumping 443 out of 1974 MB: Dumping 443 out of 1974 MB Fatal double fault Fatal double fault rip =3D 0xffffffff8066a9e0 rip =3D 0xffffffff8066a9e0 rsp =3D 0xffffff800023c000 rsp =3D 0xffffff800023c000 rbp =3D 0xffffff800023c040 rbp =3D 0xffffff800023c040 cpuid =3D 0; cpuid =3D 0; apic id =3D 00 apic id =3D 00 panic: double fault panic: double fault cpuid =3D 0 cpuid =3D 0 KDB: stack backtrace: KDB: stack backtrace: db_trac_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x187 dblfault_handler() at dblfault_handler+0xa4 Xdblfault() at Xdblfault+0xa8 --- trap 0x17, rip =3D 0xffffffff8066a9e8, rsp =3D 0xffffffff80e56158, rbp = =3D 0xffffff800023c040 --- mi_switch() at mi_switch+0x270 critical_exit() at critical_exit+0x9b spinlock_exit() at spinlock_exit+0x17 mi_switch() at mi_switch+0x275 critical_exit() at critical_exit+0x9b spinlock_exit() at spinlock_exit+0x17 [several pages of the previous three lines skipped] mi_switch() at mi_switch+0x275 critical_exit() at critical_exit+0x9b spinlock_exit() at spinlock_exit+0x17 intr_even_schedule_thread() at intr_event_schedule_thread+0xbb ahci_end_transaction() at ahci_end_transaction+0x398 ahci_ch_intr() at ahci_ch_intr+0x2b5 ahcipoll() at ahcipoll+0x15 xpt_polled_action() at xpt_polled_action+0xf7 I first noticed the problem with CURRENT from a week ago, but given that USB sticks don't usually disappear for me while sending snapshots to them, the problem might not be new. I'm using amd64, the panic above is from a custom kernel without WITNESS and INVARIANTS, but enabling them doesn't seem to affect the trace before the double fault. I wasn't able to reproduce the panic by unplugging the stick while writing to the pool using dd (but only tried once). Fabian --Sig_/kTIY0JWgN5S_DzofhXnkCee Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk6CK1wACgkQBYqIVf93VJ0enQCfc3CtlF0UCC88e0FhcCcEOc1d MLoAnA6TVWceM22LzNeMhKbBJcL9JmB6 =Cifp -----END PGP SIGNATURE----- --Sig_/kTIY0JWgN5S_DzofhXnkCee--