From owner-freebsd-questions@freebsd.org Tue Mar 22 17:06:43 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9CA82AD978C for ; Tue, 22 Mar 2016 17:06:43 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from mailbox.talkpoint.com (mailbox.talkpoint.com [204.141.10.74]) by mx1.freebsd.org (Postfix) with ESMTP id 61A89FB8 for ; Tue, 22 Mar 2016 17:06:42 +0000 (UTC) (envelope-from nevans@talkpoint.com) Received: from localhost (localhost [127.0.0.1]) by mailbox.talkpoint.com (Postfix) with ESMTP id B6451F66003 for ; Tue, 22 Mar 2016 12:59:28 -0400 (EDT) X-Virus-Scanned: amavisd-new at talkpoint.com Received: from mailbox.talkpoint.com ([127.0.0.1]) by localhost (mailbox.talkpoint.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 94jowmkZVWhv for ; Tue, 22 Mar 2016 12:59:27 -0400 (EDT) Received: from pleiades.nextvenue.com (pleiades.talkpoint.com [204.141.15.194]) by mailbox.talkpoint.com (Postfix) with ESMTP id C1F70F66001 for ; Tue, 22 Mar 2016 12:59:27 -0400 (EDT) Date: Tue, 22 Mar 2016 12:59:35 -0400 From: Nick Evans To: freebsd-questions@freebsd.org Subject: Intel 750 Panic Unmounting During TRIM Message-ID: <20160322125935.6be25e20@pleiades.nextvenue.com> X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.6; amd64-portbld-freebsd9.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Mar 2016 17:06:43 -0000 I've got a test system with a 400GB Intel 750 NVME SSD formatted UFS 1mb/4k aligned http://www.intel.com/content/www/us/en/solid-state-drives/solid-state-drives-750-series.html I moved a 50 gigabyte directory off the drive so I could try another drive configuration and while the drive was trimming, I ran umount on the mounted device. umount sat there without returning to the prompt. On another terminal, mount showed the drive unmounted, but after a minute or so in this state with gstat running to show io, the system panicked. I tried to reproduce this a second time by generating a 100gb file from /dev/zero. Deleted the file ran umount which hung again. This time after a while it returned to the prompt stating the device was busy so I tried to umount again while the drive was still trimming. After a short while the system panicked. It seems the panic happens when the TRIM finishes while umount is trying to do its thing. The system is 10.2-RELEASE-p13, GENERIC kernel, Intel 4790K on a Asus Z97-A motherboard, 32GB ram. I have a second Intel 750 drive in another PCI Express slot but it's unused at the moment. Below are the dumps recorded after the system booted back up. This is a test system so I can bring it up and down as needed to test fixes. The second dump is from my reproduced test: panic: losing buffer 1 cpuid = 2 KDB: stack backtrace: #0 0xffffffff80984ef0 at kdb_backtrace+0x60 #1 0xffffffff80948aa6 at vpanic+0x126 #2 0xffffffff80948973 at panic+0x43 #3 0xffffffff809d241b at brelse+0x70b #4 0xffffffff809e9fa7 at flushbuflist+0x137 #5 0xffffffff809e9c1b at bufobj_invalbuf+0x1cb #6 0xffffffff808acf29 at g_vfs_close+0x29 #7 0xffffffff80ba3487 at ffs_unmount+0x2a7 #8 0xffffffff809e4dbf at dounmount+0x4ef #9 0xffffffff809e48c6 at sys_unmount+0x456 #10 0xffffffff80d4b3e7 at amd64_syscall+0x357 #11 0xffffffff80d30acb at Xfast_syscall+0xfb Uptime: 21m6s Dumping 1426 out of 32426 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% #0 doadump (textdump=) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=) at pcpu.h:219 #1 0xffffffff80948702 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:451 #2 0xffffffff80948ae5 in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:758 #3 0xffffffff80948973 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:687 #4 0xffffffff809d241b in brelse (bp=) at /usr/src/sys/kern/vfs_bio.c:1768 #5 0xffffffff809e9fa7 in flushbuflist (bufv=, flags=, bo=0xfffff80014236db8, slpflag=, slptimeo=0) at /usr/src/sys/kern/vfs_subr.c:1436 #6 0xffffffff809e9c1b in bufobj_invalbuf (bo=0xfffff80014236db8, flags=1, slpflag=0, slptimeo=0) at /usr/src/sys/kern/vfs_subr.c:1312 #7 0xffffffff808acf29 in g_vfs_close (cp=0xfffff80014126e80) at /usr/src/sys/geom/geom_vfs.c:277 #8 0xffffffff80ba3487 in ffs_unmount (mp=0xfffff80014317000, mntflags=) at /usr/src/sys/ufs/ffs/ffs_vfsops.c:1272 #9 0xffffffff809e4dbf in dounmount (mp=0xfffff80014317000, flags=134217728, td=0xfffff800141a4000) at /usr/src/sys/kern/vfs_mount.c:1316 #10 0xffffffff809e48c6 in sys_unmount (td=0xfffff800141a4000, uap=0xfffffe084caffa40) at /usr/src/sys/kern/vfs_mount.c:1208 #11 0xffffffff80d4b3e7 in amd64_syscall (td=0xfffff800141a4000, traced=0) at subr_syscall.c:134 #12 0xffffffff80d30acb in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396 #13 0x000000080089190a in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) panic: losing buffer 1 cpuid = 3 KDB: stack backtrace: #0 0xffffffff80984ef0 at kdb_backtrace+0x60 #1 0xffffffff80948aa6 at vpanic+0x126 #2 0xffffffff80948973 at panic+0x43 #3 0xffffffff809d241b at brelse+0x70b #4 0xffffffff809e9fa7 at flushbuflist+0x137 #5 0xffffffff809e9c1b at bufobj_invalbuf+0x1cb #6 0xffffffff808acf29 at g_vfs_close+0x29 #7 0xffffffff80ba3487 at ffs_unmount+0x2a7 #8 0xffffffff809e4dbf at dounmount+0x4ef #9 0xffffffff809e48c6 at sys_unmount+0x456 #10 0xffffffff80d4b3e7 at amd64_syscall+0x357 #11 0xffffffff80d30acb at Xfast_syscall+0xfb Uptime: 1h29m40s Dumping 2342 out of 32426 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% #0 doadump (textdump=) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=) at pcpu.h:219 #1 0xffffffff80948702 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:451 #2 0xffffffff80948ae5 in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:758 #3 0xffffffff80948973 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:687 #4 0xffffffff809d241b in brelse (bp=) at /usr/src/sys/kern/vfs_bio.c:1768 #5 0xffffffff809e9fa7 in flushbuflist (bufv=, flags=, bo=0xfffff8001428fa08, slpflag=, slptimeo=0) at /usr/src/sys/kern/vfs_subr.c:1436 #6 0xffffffff809e9c1b in bufobj_invalbuf (bo=0xfffff8001428fa08, flags=1, slpflag=0, slptimeo=0) at /usr/src/sys/kern/vfs_subr.c:1312 #7 0xffffffff808acf29 in g_vfs_close (cp=0xfffff80014033700) at /usr/src/sys/geom/geom_vfs.c:277 #8 0xffffffff80ba3487 in ffs_unmount (mp=0xfffff800142b0000, mntflags=) at /usr/src/sys/ufs/ffs/ffs_vfsops.c:1272 #9 0xffffffff809e4dbf in dounmount (mp=0xfffff800142b0000, flags=134217728, td=0xfffff8001434c000) at /usr/src/sys/kern/vfs_mount.c:1316 #10 0xffffffff809e48c6 in sys_unmount (td=0xfffff8001434c000, uap=0xfffffe084cb0aa40) at /usr/src/sys/kern/vfs_mount.c:1208 #11 0xffffffff80d4b3e7 in amd64_syscall (td=0xfffff8001434c000, traced=0) at subr_syscall.c:134 #12 0xffffffff80d30acb in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396 #13 0x000000080089190a in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb)