From owner-freebsd-current@freebsd.org Tue Dec 8 16:09:34 2015 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE23F9D3DB2 for ; Tue, 8 Dec 2015 16:09:34 +0000 (UTC) (envelope-from 214748mv@gmail.com) Received: from mail-io0-x233.google.com (mail-io0-x233.google.com [IPv6:2607:f8b0:4001:c06::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F0281235 for ; Tue, 8 Dec 2015 16:09:34 +0000 (UTC) (envelope-from 214748mv@gmail.com) Received: by iofh3 with SMTP id h3so28911679iof.3 for ; Tue, 08 Dec 2015 08:09:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=kxs47MfDyllvq0BBuwQt04mVJsr6ykbx1S8Yio2npjY=; b=0as44VphSEJWokmeBcs1MmNW97fyU70ZZguNy673oSdTFGKuXJK+nBab4jrpGn344L qQhIVdLvyOrPp4CL36NoJLbQqUUZUmYa2kukLj6bZ1sjktD0bSRHWmVZyb37gptv0vSZ W60dhxtYzqdOeisRIGI3HoLOqKTRv3MSVHSapNTUl9ORsRUzF0OXsEY82lbO/97qB49O y91bx8r1KR1qtC3rCmkbC5NWGBaC5bvIiT9Gy+8wRfUIA8FcHiWl6U8P/8G2RHv/8fo1 Ur7QlPX4PvaMaLOASQprN9WRvqhRu7+J0/zJPYJxV6pxZqIKXgCWSF7nUlX1mMq52Eu4 4yuQ== MIME-Version: 1.0 X-Received: by 10.107.46.137 with SMTP id u9mr664907iou.136.1449590973992; Tue, 08 Dec 2015 08:09:33 -0800 (PST) Received: by 10.79.111.199 with HTTP; Tue, 8 Dec 2015 08:09:33 -0800 (PST) In-Reply-To: <055E0877-533A-4378-A306-FDE511543243@gmail.com> References: <055E0877-533A-4378-A306-FDE511543243@gmail.com> Date: Tue, 8 Dec 2015 17:09:33 +0100 Message-ID: Subject: Re: Panic at shutdown From: "Ranjan1018 ." <214748mv@gmail.com> To: Garrett Cooper Cc: FreeBSD CURRENT Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 16:09:34 -0000 2015-11-29 0:10 GMT+01:00 Garrett Cooper : > > > On Nov 28, 2015, at 12:32, Ranjan1018 . <214748mv@gmail.com> wrote: > > > > Hi, > > > > sometimes I have the panic in the photo at shutdown: > > > > http://imgur.com/mXrgFLp > > > > Unfortunately this happens randomly. > > > > I am running: > > > > $ uname -a > > > > FreeBSD ativ 11.0-CURRENT FreeBSD 11.0-CURRENT #3 r291160M: Sun Nov 22 > > 17:10:38 CET 2015 root@ativ:/usr/obj/usr/src/sys/GENERIC amd64 > > The panic is in the ZFS code. > > Have you run memtest on the machine recently? > Good suggestion I have run memtest successfully for few hours on my laptop. I have understood the panic cause: is an invalid offset. The original function in /sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c is: boolean_t txg_list_member(txg_list_t *tl, void *p, uint64_t txg) { int t = txg & TXG_MASK; txg_node_t *tn = (txg_node_t *)((char *)p + tl->tl_offset); return (tn->tn_member[t] != 0); } I have modified the function to print an uncommon or invalid tl->tl_offset : boolean_t txg_list_member(txg_list_t *tl, void *p, uint64_t txg) { size_t ofs = tl->tl_offset; { static int cnt=0; if ( (cnt++ % 1000) == 0 || (ofs != 88 && ofs != 984) ) printf("**** %d) tl->tl_offset %zu\n", cnt, ofs); } txg_node_t *tn = (txg_node_t *)((char *)p + ofs); return (tn->tn_member[txg & TXG_MASK] != 0); } I have received the panic again with an invalid tl->tl_offset of 16045693110842147038. In /val/log/messages I have: Dec 8 10:32:42 ativ kernel: Waiting (max 60 seconds) for system process `vnlru' to stop...done Dec 8 10:32:42 ativ kernel: Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Dec 8 10:32:42 ativ kernel: Waiting (max 60 seconds) for system process `syncer' to stop... Dec 8 10:32:42 ativ kernel: Syncing disks, vnodes remaining...0 0 0 done Dec 8 10:32:42 ativ kernel: All buffers synced. Dec 8 10:32:42 ativ kernel: **** 9692) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9693) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9694) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9695) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9708) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9709) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9710) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9711) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9720) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9721) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9722) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: **** 9723) tl->tl_offset 384 Dec 8 10:32:42 ativ kernel: Uptime: 1h57m42s Dec 8 10:32:42 ativ kernel: **** 9736) tl->tl_offset 16045693110842147038 Dec 8 10:32:42 ativ kernel: Dec 8 10:32:42 ativ kernel: Dec 8 10:32:42 ativ kernel: Fatal trap 9: general protection fault while in kernel mode Dec 8 10:32:42 ativ kernel: cpuid = 2; apic id = 02 Dec 8 10:32:42 ativ kernel: instruction pointer = 0x20:0xffffffff8211b1cb Dec 8 10:32:42 ativ kernel: stack pointer = 0x28:0xfffffe0119525990 Dec 8 10:32:42 ativ kernel: frame pointer = 0x28:0xfffffe01195259c0 Dec 8 10:32:42 ativ kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Dec 8 10:32:42 ativ kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Dec 8 10:32:42 ativ kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Dec 8 10:32:42 ativ kernel: current process = 0 (dbu_evict) Probably the panic is caused by some memory already freed, the hex value of 16045693110842147038 is 0xdeadc0dedeadc0de. To solve the panic I need some tips form someone more expert than me in ZFS code. Thanks. -- Maurizio