From owner-svn-src-head@freebsd.org Tue Jun 20 20:37:50 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 554ECDA1DC2; Tue, 20 Jun 2017 20:37:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E7730730BC; Tue, 20 Jun 2017 20:37:48 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA19088; Tue, 20 Jun 2017 23:37:46 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1dNPuE-000O3o-Oe; Tue, 20 Jun 2017 23:37:46 +0300 Subject: Re: svn commit: r320156 - in head: cddl/contrib/opensolaris/cmd/zdb cddl/contrib/opensolaris/cmd/ztest cddl/contrib/opensolaris/lib/libzfs/common sys/cddl/contrib/opensolaris/common/zfs sys/cddl/contri... To: Ken Merry Cc: src-committers@FreeBSD.org, svn-src-all@FreeBSD.org, svn-src-head@FreeBSD.org References: <201706201739.v5KHdPhO051256@repo.freebsd.org> <81F84BCA-E973-4D78-B81C-1D398ADFA47E@freebsd.org> From: Andriy Gapon Message-ID: Date: Tue, 20 Jun 2017 23:37:10 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: <81F84BCA-E973-4D78-B81C-1D398ADFA47E@freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2017 20:37:50 -0000 On 20/06/2017 23:29, Ken Merry wrote: > I don’t know for sure that this commit is the cause, but it (and r320153) are the only ZFS commits between a version of head from June 14th that boots off a ZFS mirror, and one that panics. > > Here’s the stack trace: > > Fatal trap 12: page fault while in kernel mode > cpuid = 22; > > Fatal trap 12: page fault while in kernel mode > cpuid = 9; apic id = 09 > fault virtual address = 0x0 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff81e47f21 > stack pointer = 0x28:0xfffffe08b37f8810 > frame pointer = 0x28:0xfffffe08b37f8860 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 0 (zio_free_issue_0_3) > [ thread pid 0 tid 100478 ] > Stopped at 0xffffffff81e47f21 = zio_vdev_io_start+0x1f1: testb $0x1,(%rax) > db> bt > Tracing pid 0 tid 100478 td 0xfffff80193156000 > zio_vdev_io_start() at 0xffffffff81e47f21 = zio_vdev_io_start+0x1f1/frame 0xfffffe08b37f8860 > zio_execute() at 0xffffffff81e4312c = zio_execute+0x36c/frame 0xfffffe08b37f88b0 > zio_nowait() at 0xffffffff81e422b8 = zio_nowait+0xb8/frame 0xfffffe08b37f88e0 > vdev_mirror_io_start() at 0xffffffff81e224fc = vdev_mirror_io_start+0x38c/frame 0xfffffe08b37f8930 > zio_vdev_io_start() at 0xffffffff81e48030 = zio_vdev_io_start+0x300/frame 0xfffffe08b37f8990 > zio_execute() at 0xffffffff81e4312c = zio_execute+0x36c/frame 0xfffffe08b37f89e0 > taskqueue_run_locked() at 0xffffffff809a9d6d = taskqueue_run_locked+0x13d/frame 0xfffffe08b37f8a40 > taskqueue_thread_loop() at 0xffffffff809aab28 = taskqueue_thread_loop+0x88/frame 0xfffffe08b37f8a70 > fork_exit() at 0xffffffff8091e3e4 = fork_exit+0x84/frame 0xfffffe08b37f8ab0 > fork_trampoline() at 0xffffffff80d930fe = fork_trampoline+0xe/frame 0xfffffe08b37f8ab0 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > db> > > (kgdb) list *(zio_vdev_io_start+0x1f1) > 0xd9f21 is in zio_vdev_io_start (/usr/home/kenm/perforce4/kenm/FreeBSD-test/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:350). > 345 > 346 /* > 347 * Ensure that anyone expecting this zio to contain a linear ABD isn't > 348 * going to get a nasty surprise when they try to access the data. > 349 */ > 350 IMPLY(abd_is_linear(zio->io_abd), abd_is_linear(data)); > 351 > 352 zt->zt_orig_abd = zio->io_abd; > 353 zt->zt_orig_size = zio->io_size; > 354 zt->zt_bufsize = bufsize; > > I’ll try rebooting and see if the problem goes away. If not, I’ll roll back the ABD change and see if the problem goes away. Judging from the thread that panic-ed the problem may have to do with our TRIM support. Unfortunately, I didn't have a chance to test the change on a system with working TRIM and, so, I missed it. I will look into this further, but it's almost obvious that the problem is caused by zio->io_abd being NULL for a zio of type ZIO_TYPE_FREE. -- Andriy Gapon