From owner-freebsd-current@FreeBSD.ORG Wed Jan 28 19:04:54 2015 Return-Path: Delivered-To: current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 14CE1B10; Wed, 28 Jan 2015 19:04:54 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91D15D28; Wed, 28 Jan 2015 19:04:53 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t0SJ4gaI046723 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 28 Jan 2015 21:04:42 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t0SJ4gaI046723 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t0SJ4gaE046722; Wed, 28 Jan 2015 21:04:42 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 28 Jan 2015 21:04:42 +0200 From: Konstantin Belousov To: Gleb Smirnoff Subject: Re: panic in softdep_slowdown() Message-ID: <20150128190441.GO42409@kib.kiev.ua> References: <20150127203103.GZ15484@glebius.int.ru> <20150128104842.GL42409@kib.kiev.ua> <20150128182230.GB15484@glebius.int.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150128182230.GB15484@glebius.int.ru> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: current@FreeBSD.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2015 19:04:54 -0000 On Wed, Jan 28, 2015 at 09:22:30PM +0300, Gleb Smirnoff wrote: > On Wed, Jan 28, 2015 at 12:48:42PM +0200, Konstantin Belousov wrote: > K> > Stopped at softdep_slowdown+0x1d3: idivl %ecx,%eax > K> > db> bt > K> > Tracing pid 49 tid 100045 td 0xfffff800026ee000 > K> > softdep_slowdown() at softdep_slowdown+0x1d3/frame 0xfffffe001eb5f2b0 > K> > ffs_truncate() at ffs_truncate+0x1be/frame 0xfffffe001eb5f640 > K> > ufs_setattr() at ufs_setattr+0x4e5/frame 0xfffffe001eb5f6a0 > K> > VOP_SETATTR_APV() at VOP_SETATTR_APV+0x22a/frame 0xfffffe001eb5f710 > K> > VOP_SETATTR() at VOP_SETATTR+0x45/frame 0xfffffe001eb5f760 > K> > vn_truncate() at vn_truncate+0x196/frame 0xfffffe001eb5f870 > K> > fo_truncate() at fo_truncate+0x41/frame 0xfffffe001eb5f8b0 > K> > kern_ftruncate() at kern_ftruncate+0x16d/frame 0xfffffe001eb5f920 > K> > sys_ftruncate() at sys_ftruncate+0x27/frame 0xfffffe001eb5f940 > K> > syscallenter() at syscallenter+0x46e/frame 0xfffffe001eb5f9b0 > K> > amd64_syscall() at amd64_syscall+0x1f/frame 0xfffffe001eb5fab0 > K> > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe001eb5fab0 > K> > --- syscall (480, FreeBSD ELF64, sys_ftruncate), rip = 0x800b511fa, rsp = 0x7fffffffe998, rbp = 0x7fffffffeb90 --- > K> > db> call doadump > K> > Dumping 60 out of 495 MB:..27%..54%..80% > K> > Dump complete > K> > = 0 > K> > db> > K> > > K> > I've got the core file. > K> > K> At least the source line for the panic is needed. > K> Also, print out the value of stat_flush_threads. > > (kgdb) fr 11 > #11 0xffffffff80895d63 in softdep_slowdown (vp=0xfffff800028011d8) > at /usr/src/ifnet/sys/ufs/ffs/ffs_softdep.c:13055 > 13055 if (dep_current[D_DIRREM] < max_softdeps_hard / 2 && > (kgdb) p dep_current > $1 = {1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, > 0, 0, 0, 0} > (kgdb) p max_softdeps_hard > $2 = 153357 > (kgdb) p *ump > $4 = {um_mountp = 0xfffff80002707330, um_dev = 0xfffff800026cbc00, > um_cp = 0xfffff80002717480, um_bo = 0xfffff8000271edb8, > um_devvp = 0xfffff8000271ece8, um_fstype = 2, um_fs = 0xfffff8000273b000, > um_extattr = {uepm_lock = {lock_object = {lo_name = 0x0, lo_flags = 0, > lo_data = 0, lo_witness = 0x0}, sx_lock = 0}, uepm_list = { > lh_first = 0x0}, uepm_ucred = 0x0, uepm_flags = 0}, um_nindir = 4096, > um_bptrtodb = 3, um_seqinc = 8, um_lock = {lock_object = { > lo_name = 0xffffffff80a53d30 "FFS", lo_flags = 16973824, lo_data = 0, > lo_witness = 0xfffffe00008e3400}, mtx_lock = 4}, um_fsckpid = 0, > um_softdep = 0xfffff800027a0200, um_quotas = {0x0, 0x0}, um_cred = {0x0, > 0x0}, um_btime = {0, 0}, um_itime = {0, 0}, um_qflags = "\000", > um_savedmaxfilesize = 0, um_candelete = 0, um_writesuspended = 0, > um_balloc = 0xffffffff8086eb90 , > um_blkatoff = 0xffffffff808a8170 , > um_truncate = 0xffffffff808717b0 , > um_update = 0xffffffff80871090 , > um_valloc = 0xffffffff808660c0 , > um_vfree = 0xffffffff808677b0 , > um_ifree = 0xffffffff808af420 , > um_rdonly = 0xffffffff808741c0 , > um_snapgone = 0xffffffff80879b70 } > (kgdb) p stat_flush_threads > $5 = 1 > > I can't see where integer divide fault can happen with stat_flush_threads=1 :( Look at the exact asm instruction which faulted, also look at the registers content. It might be hypervisor bug, after all.