From owner-freebsd-fs@freebsd.org Sun Sep 16 21:00:26 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59FC610AA571 for ; Sun, 16 Sep 2018 21:00:26 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A0527802FC for ; Sun, 16 Sep 2018 21:00:25 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 650DB10AA56A; Sun, 16 Sep 2018 21:00:25 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 434F810AA569 for ; Sun, 16 Sep 2018 21:00:25 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D9C82802F5 for ; Sun, 16 Sep 2018 21:00:24 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 02D7927D0D for ; Sun, 16 Sep 2018 21:00:24 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8GL0NsH098856 for ; Sun, 16 Sep 2018 21:00:23 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8GL0Nsq098852 for fs@FreeBSD.org; Sun, 16 Sep 2018 21:00:23 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201809162100.w8GL0Nsq098852@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: fs@FreeBSD.org Subject: Problem reports for fs@FreeBSD.org that need special attention Date: Sun, 16 Sep 2018 21:00:23 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Sep 2018 21:00:26 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic New | 217062 | for file systems mounted with -o noexec, exec=off New | 221909 | [ZFS] Add a sysctl to toggle send_corrupt_data Open | 140068 | [smbfs] [patch] smbfs does not allow semicolon in Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 211491 | System hangs after "Uptime" on reboot with ZFS 6 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Mon Sep 17 14:34:15 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2DE05109EC75 for ; Mon, 17 Sep 2018 14:34:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BF8B67E13B for ; Mon, 17 Sep 2018 14:34:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 83DCD109EC74; Mon, 17 Sep 2018 14:34:14 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 727A5109EC73 for ; Mon, 17 Sep 2018 14:34:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 14E537E13A for ; Mon, 17 Sep 2018 14:34:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 4A32A117F5 for ; Mon, 17 Sep 2018 14:34:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8HEYDV2080613 for ; Mon, 17 Sep 2018 14:34:13 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8HEYDoO080612 for fs@FreeBSD.org; Mon, 17 Sep 2018 14:34:13 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 230704] All the memory eaten away by ZFS 'solaris' malloc Date: Mon, 17 Sep 2018 14:34:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: Mark.Martinec@ijs.si X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 14:34:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230704 --- Comment #5 from Mark.Martinec@ijs.si --- Tried it now (based on 11.2-RELEASE-p3). It is a major improvement: the 'vmstat -m' InUse count on a solaris zone came down to 18 (from previous 520 or so) per invocation of a 'zpool list' on a defunct pool: # (while true; do zpool list stuff >/dev/null; vmstat -m | fgrep solaris; s= leep 1; done) | awk '{print $2-a, $3; a=3D$2}' 41167 20851K 18 20854K 18 20856K 18 20859K 18 20861K 18 20863K 18 20866K 18 20868K 18 20870K 18 20873K 18 20875K 18 20878K 18 20880K 18 20882K 18 20885K So instead of four days, this host would now stay up 30 times longer. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Sep 17 15:11:28 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDFA3109FD96 for ; Mon, 17 Sep 2018 15:11:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6A9877FBAF for ; Mon, 17 Sep 2018 15:11:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 2FBCD109FD95; Mon, 17 Sep 2018 15:11:28 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EB29109FD94 for ; Mon, 17 Sep 2018 15:11:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B4E4F7FBAA for ; Mon, 17 Sep 2018 15:11:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id EB10811C57 for ; Mon, 17 Sep 2018 15:11:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8HFBQBd097132 for ; Mon, 17 Sep 2018 15:11:26 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8HFBQsK097123 for fs@FreeBSD.org; Mon, 17 Sep 2018 15:11:26 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 230704] All the memory eaten away by ZFS 'solaris' malloc Date: Mon, 17 Sep 2018 15:11:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: markj@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 15:11:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230704 --- Comment #6 from Mark Johnston --- (In reply to Mark.Martinec from comment #5) This might be the result of zfs_dbgmsg(). If so, the memory usage will stop increasing once zfs_dbgmsg_size hits the 4MB limit. Could you run zpool -Hp for a while and see if the memory usage stops increasing? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Sep 17 15:22:13 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0031410A01F2 for ; Mon, 17 Sep 2018 15:22:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 9153480285 for ; Mon, 17 Sep 2018 15:22:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 561E410A01F1; Mon, 17 Sep 2018 15:22:12 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44CDD10A01F0 for ; Mon, 17 Sep 2018 15:22:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA0CB80281 for ; Mon, 17 Sep 2018 15:22:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 33B8B11DD7 for ; Mon, 17 Sep 2018 15:22:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8HFMBal048509 for ; Mon, 17 Sep 2018 15:22:11 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8HFMBm5048508 for fs@FreeBSD.org; Mon, 17 Sep 2018 15:22:11 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 230704] All the memory eaten away by ZFS 'solaris' malloc Date: Mon, 17 Sep 2018 15:22:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: markj@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: markj@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 15:22:13 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230704 Mark Johnston changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|fs@FreeBSD.org |markj@FreeBSD.org Status|New |In Progress --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Sep 17 20:15:19 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B4F310A6447 for ; Mon, 17 Sep 2018 20:15:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id ED0E28B492 for ; Mon, 17 Sep 2018 20:15:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id B127B10A6446; Mon, 17 Sep 2018 20:15:18 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9FDE610A6445 for ; Mon, 17 Sep 2018 20:15:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4224D8B490 for ; Mon, 17 Sep 2018 20:15:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A0BEB1476D for ; Mon, 17 Sep 2018 20:15:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8HKFHYf012405 for ; Mon, 17 Sep 2018 20:15:17 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8HKFHS9012404 for fs@FreeBSD.org; Mon, 17 Sep 2018 20:15:17 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 229614] ZFS lockup in zil_commit_impl Date: Mon, 17 Sep 2018 20:15:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: trzepka@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 20:15:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229614 Mike Nelson changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |trzepka@gmail.com --- Comment #10 from Mike Nelson --- I have the same problem, does anyone know the solution? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Sep 17 20:34:43 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F53E10A6C66 for ; Mon, 17 Sep 2018 20:34:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 032128C1E3 for ; Mon, 17 Sep 2018 20:34:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id B856910A6C65; Mon, 17 Sep 2018 20:34:42 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6E8910A6C64 for ; Mon, 17 Sep 2018 20:34:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 491528C1E2 for ; Mon, 17 Sep 2018 20:34:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 8BEE114A49 for ; Mon, 17 Sep 2018 20:34:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8HKYfWT051775 for ; Mon, 17 Sep 2018 20:34:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8HKYf5a051772 for fs@FreeBSD.org; Mon, 17 Sep 2018 20:34:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 229614] ZFS lockup in zil_commit_impl Date: Mon, 17 Sep 2018 20:34:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: grembo@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 20:34:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229614 --- Comment #11 from Michael Gmelin --- (In reply to Mike from comment #10) As a workaround, try if setting kern.maxvnodes to a high number (it's usual= ly quite low on small VMs) helps, e.g. sysctl kern.maxvnodes=3D2000000 use sysrc -f /etc/sysctl.conf kern.maxvnodes=3D2000000 to set it permanently. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Sep 17 20:59:39 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 248E910A7777 for ; Mon, 17 Sep 2018 20:59:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B6D6A8CFE0 for ; Mon, 17 Sep 2018 20:59:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 7A9BF10A7776; Mon, 17 Sep 2018 20:59:38 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6926610A7775 for ; Mon, 17 Sep 2018 20:59:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C9048CFDB for ; Mon, 17 Sep 2018 20:59:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 69EF014D0C for ; Mon, 17 Sep 2018 20:59:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8HKxb1g095578 for ; Mon, 17 Sep 2018 20:59:37 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8HKxbRZ095577 for fs@FreeBSD.org; Mon, 17 Sep 2018 20:59:37 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 229614] ZFS lockup in zil_commit_impl Date: Mon, 17 Sep 2018 20:59:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: michal.kulik3@o2.pl X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 20:59:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229614 --- Comment #12 from Mike --- (In reply to Michael Gmelin from comment #11) But I don't use VPS/Cloud/AWS. I have a dedicated server and kern.maxvnodes=3D2215782. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Sep 17 21:07:30 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C58B910A7A57 for ; Mon, 17 Sep 2018 21:07:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 62F418D404 for ; Mon, 17 Sep 2018 21:07:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 23A2110A7A56; Mon, 17 Sep 2018 21:07:30 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1251210A7A54 for ; Mon, 17 Sep 2018 21:07:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AA6758D3FE for ; Mon, 17 Sep 2018 21:07:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id F0F8814E76 for ; Mon, 17 Sep 2018 21:07:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8HL7S3X078589 for ; Mon, 17 Sep 2018 21:07:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8HL7Skd078584 for fs@FreeBSD.org; Mon, 17 Sep 2018 21:07:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 229614] ZFS lockup in zil_commit_impl Date: Mon, 17 Sep 2018 21:07:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: grembo@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 21:07:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229614 --- Comment #13 from Michael Gmelin --- (In reply to Mike from comment #12) Interesting, I haven't seen the problem on bare metal yet (we have 4 million there which is plenty for our workloads). Check sysctl vfs.numvnodes under your typical workloads and set kern.maxvnodes so that it's twice as much(even if it works this is just a workaround, the underlying issue absolutely has to be fixed). Can you reproduce the problem reliably, or does it just happen once in a wh= ile (e.g. during nightly periodic jobs)? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Sep 19 10:23:35 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C33761096516 for ; Wed, 19 Sep 2018 10:23:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 601E57B14D for ; Wed, 19 Sep 2018 10:23:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 24DD91096515; Wed, 19 Sep 2018 10:23:35 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 138AA1096514 for ; Wed, 19 Sep 2018 10:23:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A78837B146 for ; Wed, 19 Sep 2018 10:23:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 1048D26770 for ; Wed, 19 Sep 2018 10:23:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8JANXEp033444 for ; Wed, 19 Sep 2018 10:23:33 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8JANXmt033443 for fs@FreeBSD.org; Wed, 19 Sep 2018 10:23:33 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 230962] Kernel panic when writing extended attributes with soft updates enabled Date: Wed, 19 Sep 2018 10:23:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: panic X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: intellisun@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2018 10:23:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230962 intellisun@gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |intellisun@gmail.com --- Comment #1 from intellisun@gmail.com --- I have the same problem with soft-updates and acls activated on UFS volume. FreeBSD version 11.2-STABLE, r335822. If I disable either soft-updates or a= cls, the system behaves normally. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Sep 19 14:18:41 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ACE50109C75F for ; Wed, 19 Sep 2018 14:18:41 +0000 (UTC) (envelope-from theraven@freebsd.org) Received: from theravensnest.org (theravensnest.org [46.226.110.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "theravensnest.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3857982512 for ; Wed, 19 Sep 2018 14:18:41 +0000 (UTC) (envelope-from theraven@freebsd.org) Received: from [10.164.186.139] ([167.220.196.139]) (authenticated bits=0) by theravensnest.org (8.15.2/8.15.2) with ESMTPSA id w8JEFH3M028492 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 19 Sep 2018 14:15:18 GMT (envelope-from theraven@freebsd.org) X-Authentication-Warning: mail: Host [167.220.196.139] claimed to be [10.164.186.139] To: freebsd-fs@freebsd.org From: David Chisnall Subject: Fixing vdev name Message-ID: Date: Wed, 19 Sep 2018 15:16:16 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2018 14:18:41 -0000 Hello the list, I have a VM that uses ZFS with a pair of striped devices. The first is a partition on the root disk, created by the installer. When this was too small, I added another device (da1) and told the pool to expand to use it (no redundancy, because the underlying storage for the VM images provides that). After a reboot, I can no longer boot the system. Booting from the install CD and attempting to import the pool, it correctly identifies da0p4 as one of the devices, but gives me a long number instead of da1. How do I fix this so that the pool again points to da1? David From owner-freebsd-fs@freebsd.org Wed Sep 19 14:43:59 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3FB72109D850 for ; Wed, 19 Sep 2018 14:43:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CD9D583A1C for ; Wed, 19 Sep 2018 14:43:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 8F877109D84B; Wed, 19 Sep 2018 14:43:58 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7E2F4109D84A for ; Wed, 19 Sep 2018 14:43:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1DA0483A1A for ; Wed, 19 Sep 2018 14:43:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 6C678BD1 for ; Wed, 19 Sep 2018 14:43:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8JEhvhR059552 for ; Wed, 19 Sep 2018 14:43:57 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8JEhvlB059551 for fs@FreeBSD.org; Wed, 19 Sep 2018 14:43:57 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 230962] Kernel panic when writing extended attributes with soft updates enabled Date: Wed, 19 Sep 2018 14:43:54 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: panic X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2018 14:43:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230962 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mckusick@FreeBSD.org --- Comment #2 from Kirk McKusick --- Do either of you have a test case / example that will trigger the panic? We have not been able to reproduce the problem and so have no way to unders= tand what is causing it. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Sep 19 19:05:15 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4FD8A10A41C3 for ; Wed, 19 Sep 2018 19:05:15 +0000 (UTC) (envelope-from Alexander@leidinger.net) Received: from mailgate.Leidinger.net (bastille.leidinger.net [89.238.82.207]) (using TLSv1.2 with cipher DHE-RSA-CAMELLIA128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD6CB8DE83 for ; Wed, 19 Sep 2018 19:05:14 +0000 (UTC) (envelope-from Alexander@leidinger.net) Date: Wed, 19 Sep 2018 21:04:48 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1537383906; bh=+fD7GQnYGJDoXCYdQnWRyIUD73/mSz3pZxlxtcDm7Dk=; h=Date:From:To:Subject:In-Reply-To; b=V5dHdQAAuqayocWo2H6x+pgoiYCxlbDtD+0ipNoEqQ9WJYcN0iQ+yq/xmNINOKbWT iILlVD4qqmrjqwdFjb1S3KNXEovPMWF7cguboF0bghC0fME76pqNu/CfGY5m4VJUfm hGt1cYOwD2kv6LllmiqiNSFEX1b1seCbsIhSwM6B6NnwYS7htawanNIq2aBSWsrqfJ R++t0UbGJ69Jqe1INqDFDgF8/F54BRBzAoE/Rj7/Zdvf4uo1ssr9mhn5nXFrEc0utF t7aKNLzbqOPcObi6cyUK/kW32t926es1OQ1nY9q4PIHb0RRdGtg0d6z24x6tPiIkXS fwxURxoRWkbkA== Message-ID: <20180919210448.Horde.pAvRtBuz4AvR5qdGFrmSjTp@webmail.leidinger.net> From: Alexander Leidinger To: freebsd-fs@freebsd.org Subject: Re: Fixing vdev name In-Reply-To: User-Agent: Horde Application Framework 5 Accept-Language: de,en Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes MIME-Version: 1.0 Content-Disposition: inline X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2018 19:05:15 -0000 Quoting David Chisnall (from Wed, 19 Sep 2018 15:16:16 +0100): > Hello the list, > > I have a VM that uses ZFS with a pair of striped devices. The first > is a partition on the root disk, created by the installer. When > this was too small, I added another device (da1) and told the pool > to expand to use it (no redundancy, because the underlying storage > for the VM images provides that). After a reboot, I can no longer > boot the system. Booting from the install CD and attempting to > import the pool, it correctly identifies da0p4 as one of the > devices, but gives me a long number instead of da1. This means ZFS doesn't see the other device (or more correctly no device with the ZFS meta-data on the device which matches what the pool wants to see as the second vdev). Do you see the second disk from non-ZFS tools? Does the partition info look OK there (if you partitioned it before giving it to ZFS)? Does the geometry/size look correct? > How do I fix this so that the pool again points to da1? As a side note, it doesn't matter if it is da1 or something else (e.g. /dev/gpt/ or whatever), as long as it is a geom provider instead of the uuid of the device like it seems to be the case right now. Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF From owner-freebsd-fs@freebsd.org Wed Sep 19 19:59:48 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11DC910A542C for ; Wed, 19 Sep 2018 19:59:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A3F348FE42 for ; Wed, 19 Sep 2018 19:59:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 64F7B10A542B; Wed, 19 Sep 2018 19:59:47 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5206010A542A for ; Wed, 19 Sep 2018 19:59:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E295B8FE3E for ; Wed, 19 Sep 2018 19:59:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 4CF0138E0 for ; Wed, 19 Sep 2018 19:59:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8JJxkN6025061 for ; Wed, 19 Sep 2018 19:59:46 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8JJxk7g025060 for fs@FreeBSD.org; Wed, 19 Sep 2018 19:59:46 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 229614] ZFS lockup in zil_commit_impl Date: Wed, 19 Sep 2018 19:59:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: michal.kulik3@o2.pl X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2018 19:59:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229614 --- Comment #14 from Mike --- (In reply to Michael Gmelin from comment #13) I have started monitoring vfs.numvnodes and it indeed sometimes almost fill= s up the "2215782" setting of kern.maxvnodes. Do you suggest raising the latter = to 4431564 (twice as much) or would this be too high and could cause potential issues? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Sep 19 22:36:43 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 99ADF10A9175 for ; Wed, 19 Sep 2018 22:36:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2F2CC95A16 for ; Wed, 19 Sep 2018 22:36:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id E358810A9174; Wed, 19 Sep 2018 22:36:42 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1B7810A9173 for ; Wed, 19 Sep 2018 22:36:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6746895A15 for ; Wed, 19 Sep 2018 22:36:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A9B3B51E9 for ; Wed, 19 Sep 2018 22:36:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8JMafa2077180 for ; Wed, 19 Sep 2018 22:36:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8JMafvv077179 for fs@FreeBSD.org; Wed, 19 Sep 2018 22:36:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 229614] ZFS lockup in zil_commit_impl Date: Wed, 19 Sep 2018 22:36:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: grembo@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Sep 2018 22:36:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229614 --- Comment #15 from Michael Gmelin --- (In reply to Mike from comment #14) As long as you're not memory constrained raising kern.maxvnodes shouldn't c= ause any issues. If anything, it could even increase disk i/o performance (we up= ped small vms that would by default have 200k to 1000000 kern.maxvnodes without= any issues). It would be interesting if you could share if the adjustments improved your situation as well as the kind of workloads that cause interlocks in your se= tup right now to help create a test case that reproduces the problem reliably. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Sep 20 07:49:33 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D15B10939B8 for ; Thu, 20 Sep 2018 07:49:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id DEA08813C0 for ; Thu, 20 Sep 2018 07:49:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id A0C7210939B7; Thu, 20 Sep 2018 07:49:32 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F6C610939B6 for ; Thu, 20 Sep 2018 07:49:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 30766813BD for ; Thu, 20 Sep 2018 07:49:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 8F74B12102 for ; Thu, 20 Sep 2018 07:49:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8K7nV29071894 for ; Thu, 20 Sep 2018 07:49:31 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8K7nVlf071893 for fs@FreeBSD.org; Thu, 20 Sep 2018 07:49:31 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 230962] Kernel panic when writing extended attributes with soft updates enabled Date: Thu, 20 Sep 2018 07:49:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: panic X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koro@kzxiv.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2018 07:49:33 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230962 --- Comment #3 from koro@kzxiv.net --- I too am affected by this, it was so bad I had to rollback to 11.1. I've been trying, to no avail, to reproduce it in a VM. It definitely happened on a real machine with a real 8TB drive though. I don't think transferring a whole disk image of this size just for the sak= e of debugging would be feasible, however if there was a way to make a sparse im= age that only contained filesystem metadata (but not file data or unused blocks= ), either with an existing tool or by coding it myself (with some guidance as = to how to obtain said list of blocks), I'd be willing to share (privately) suc= h a disk image. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Sep 20 13:05:56 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13F5C109C2E7 for ; Thu, 20 Sep 2018 13:05:56 +0000 (UTC) (envelope-from theraven@freebsd.org) Received: from theravensnest.org (theravensnest.org [46.226.110.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "theravensnest.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 918DA8C492 for ; Thu, 20 Sep 2018 13:05:55 +0000 (UTC) (envelope-from theraven@freebsd.org) Received: from [10.164.186.139] ([167.220.196.139]) (authenticated bits=0) by theravensnest.org (8.15.2/8.15.2) with ESMTPSA id w8KD4lHE050912 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 20 Sep 2018 13:04:48 GMT (envelope-from theraven@freebsd.org) X-Authentication-Warning: mail: Host [167.220.196.139] claimed to be [10.164.186.139] Subject: Re: Fixing vdev name To: Alexander Leidinger , freebsd-fs@freebsd.org References: From: David Chisnall Message-ID: <58e2716e-5221-c529-d270-ba4e3b322fac@theravensnest.org> Date: Thu, 20 Sep 2018 14:05:46 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2018 13:05:56 -0000 [ Please keep me in the cc line - I'm not subscribed to this list ] >> Hello the list, >> >> I have a VM that uses ZFS with a pair of striped devices. The first >> is a partition on the root disk, created by the installer. When >> this was too small, I added another device (da1) and told the pool >> to expand to use it (no redundancy, because the underlying storage >> for the VM images provides that). After a reboot, I can no longer >> boot the system. Booting from the install CD and attempting to >> import the pool, it correctly identifies da0p4 as one of the >> devices, but gives me a long number instead of da1. > > This means ZFS doesn't see the other device (or more correctly no > device with the ZFS meta-data on the device which matches what the > pool wants to see as the second vdev). So there's been some corruption to the disk? > Do you see the second disk from non-ZFS tools? geom lists da1 as a device of the correct type, but zdb doesn't find any labels for it. > Does the partition info > look OK there (if you partitioned it before giving it to ZFS)? I dind't partition it, I just assigned the whole thing to ZFS. > Does the geometry/size look correct? Yup, in geom list it all looks sensible. > >> How do I fix this so that the pool again points to da1? > > As a side note, it doesn't matter if it is da1 or something else (e.g. > /dev/gpt/ or whatever), as long as it is a geom provider > instead of the uuid of the device like it seems to be the case right > now. So does this mean that something on the (virtual) disk was corrupted (sufficiently to remove both copies of the label and the uberblock)? David From owner-freebsd-fs@freebsd.org Thu Sep 20 15:02:23 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05795109F752 for ; Thu, 20 Sep 2018 15:02:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 957B370E45 for ; Thu, 20 Sep 2018 15:02:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 5AFB9109F74A; Thu, 20 Sep 2018 15:02:22 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49CCB109F749 for ; Thu, 20 Sep 2018 15:02:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DEBB370E40 for ; Thu, 20 Sep 2018 15:02:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 3968315EC4 for ; Thu, 20 Sep 2018 15:02:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8KF2LMK066995 for ; Thu, 20 Sep 2018 15:02:21 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8KF2La7066987 for fs@FreeBSD.org; Thu, 20 Sep 2018 15:02:21 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 230962] Kernel panic when writing extended attributes with soft updates enabled Date: Thu, 20 Sep 2018 15:02:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: panic X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2018 15:02:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230962 --- Comment #4 from Kirk McKusick --- (In reply to koro from comment #3) We think we have a way to reproduce it, but not yet sure. So hold on to that disk image for now. But hopefully we will not need it. Stay tuned. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu Sep 20 19:45:23 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C386F10A7B42 for ; Thu, 20 Sep 2018 19:45:23 +0000 (UTC) (envelope-from Alexander@leidinger.net) Received: from mailgate.Leidinger.net (bastille.leidinger.net [89.238.82.207]) (using TLSv1.2 with cipher DHE-RSA-CAMELLIA128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3E0337D02A; Thu, 20 Sep 2018 19:45:22 +0000 (UTC) (envelope-from Alexander@leidinger.net) Date: Thu, 20 Sep 2018 21:44:52 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leidinger.net; s=outgoing-alex; t=1537472721; bh=kfG106h3bCJ3nTRN/Cg+T44WLA09VSd72C/yvP9CL3c=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=QzKlhEJn13VFZOSxzEbnsuql1zMiaeTDGcO+4lDrJ+a56+gQrd7yF2ju1zBp3RMkC uOCQCkMkCZP3XKMZTWZ1OrZgHIIhcfSioWOHAL0c38wLqFhFe5VUHI770EYnRJDaw2 zUIbS1iS2fuuXzgzanp6E4OjiMdPF+HOc9CdaJ+KHd4ThLTW9yha6lBPPZS7scQd7h Vh65guy8tgLTsumf5E6c5iWKQOOlxtzCCj0b+7SelIUBdiUR642t2V+J5NZDmAeriP WY8JUemrdwOH5dWhoZ5jEke8CeEMz3vfevsOZUun6RrP+YElYAwPU8HASxCubAb6Og cHbdYDsHSpWmw== Message-ID: <20180920214452.Horde.tmI5_tfqjxx2hFC_n4XKZFO@webmail.leidinger.net> From: Alexander Leidinger To: David Chisnall Cc: freebsd-fs@freebsd.org Subject: Re: Fixing vdev name References: <58e2716e-5221-c529-d270-ba4e3b322fac@theravensnest.org> In-Reply-To: <58e2716e-5221-c529-d270-ba4e3b322fac@theravensnest.org> User-Agent: Horde Application Framework 5 Accept-Language: de,en Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes MIME-Version: 1.0 Content-Disposition: inline X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2018 19:45:24 -0000 Quoting David Chisnall (from Thu, 20 Sep 2018 14:05:46 +0100): > [ Please keep me in the cc line - I'm not subscribed to this list ] > >>> Hello the list, >>> >>> I have a VM that uses ZFS with a pair of striped devices. The >>> first is a partition on the root disk, created by the installer. >>> When this was too small, I added another device (da1) and told >>> the pool to expand to use it (no redundancy, because the >>> underlying storage for the VM images provides that). After a >>> reboot, I can no longer boot the system. Booting from the install >>> CD and attempting to import the pool, it correctly identifies >>> da0p4 as one of the devices, but gives me a long number instead >>> of da1. >> >> This means ZFS doesn't see the other device (or more correctly no >> device with the ZFS meta-data on the device which matches what the >> pool wants to see as the second vdev). > > So there's been some corruption to the disk? Maybe. >> Do you see the second disk from non-ZFS tools? > > geom lists da1 as a device of the correct type, but zdb doesn't find > any labels for it. > >> Does the partition info look OK there (if you partitioned it >> before giving it to ZFS)? > > I dind't partition it, I just assigned the whole thing to ZFS. For a raw disk (real hardware) not recommended, but for a LUN from a SAN I see no issue with that. >> Does the geometry/size look correct? > > Yup, in geom list it all looks sensible. > >> >>> How do I fix this so that the pool again points to da1? >> >> As a side note, it doesn't matter if it is da1 or something else >> (e.g. /dev/gpt/ or whatever), as long as it is a geom >> provider instead of the uuid of the device like it seems to be the >> case right now. > > So does this mean that something on the (virtual) disk was corrupted > (sufficiently to remove both copies of the label and the uberblock)? Sounds like. Or the unterlying provider was in some kind of "revert to initial state after power-cycle" mode. Or the backing store was exchanged in between. Or ... If you have the space, make an exact copy of the backing store of both disks, then run "zpool import -F ". This will _try_ to revert back to a pool state which can be imported (before addition of the 2nd vdev). This will then off course be without any data (or chance to recover it) since the addition of the second vdev, but maybe this is good enough for you. You then have off course the possibility to copy both pools back from the "safecopy" you did before and you will be in the same situation as now. I have no other idea right now and don't know if there is some kind of way to let zfs/zdb search harder for ZFS structures on disk (to give you an idea of what may have happened or how big a possible corruption is). Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF From owner-freebsd-fs@freebsd.org Fri Sep 21 00:27:06 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02D3110AD9BE for ; Fri, 21 Sep 2018 00:27:06 +0000 (UTC) (envelope-from jgitlin@goboomtown.com) Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7919E8642A for ; Fri, 21 Sep 2018 00:27:05 +0000 (UTC) (envelope-from jgitlin@goboomtown.com) Received: by mail-io1-xd2a.google.com with SMTP id y3-v6so10458571ioc.5 for ; Thu, 20 Sep 2018 17:27:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=goboomtown-com.20150623.gappssmtp.com; s=20150623; h=from:mime-version:subject:message-id:date:to; bh=b1XR1IgkzmH2EpotrT7rGR0VTn7+lOQ5S4Wr6qguxtA=; b=r6qZeYl6bF5EL6HcpKl34RLs/86kB8goU+EOOcyYnhzkuuw+KqiPAJtmuE/am3FXYY DMFGgsX/asOhXFyy/jtHFTFfOXiiY9411o47fq8iIRFnG69pBgUIJwMwe8nh6pkzUqNc QYFr8Il6NPzpUiriplbytE6XOTjrnLwewOHk0z1uTZcR2WXlPskO9H69zi+lC9knXgYD LmETtPX9afmo9BHeLX9EEyANaMKDkNDFisWku14W7qk8XKjnpWLLm917WEuAbZnNt1CI D3HzFE31plW6eY+IfTuitwGH7nRxr3JPY0I1bXm8fUidp5nxw6lYO2QvAcngQkywJF6i 8XgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:date:to; bh=b1XR1IgkzmH2EpotrT7rGR0VTn7+lOQ5S4Wr6qguxtA=; b=reP8nt5KPJ9wpa+QLKBmvI3ygCVX7ybvpABe7dfTukFBko0vMq04xhKHBmKuswrMR1 64TQK/BCMBoem40WpARPeWbAw8ETnD+hXuwTLtayPf2+6UAI7vgbRUcFXcDkRdXNT+Pb a5Pg7Cw7tKE3RGG76/OtI/P/7NJJ5aK/Iepqn8m22+kxgKK6oKkuFdJmpc3BYfi5G1ET l4MpAP0CbGcxr5WcSGZhhKrTXo4UGMtM4V+XjDJGH6y7uTlBJbFhtE318sD6sbOORtTQ FPZOnwvWJSowwU7WUq+UGO2WmyiWSf26kt+vNAAVnc8ys0HKJQYcX+AZtgahZwSeOPUw fmKQ== X-Gm-Message-State: APzg51B45BZ0pxq0hWa8EoTupMfSvVPChLc/pZIPKalmiMz9Y17nf+u7 Qd9Mn/WFrNw6pzzUCGz7xAhn6NjEQNu2lg== X-Google-Smtp-Source: ANB0VdakMGkQSHmSWH0PQyfDSJt869/vUap2LUNShlUryTlvv7gwf6eRwpBtFcf8bz4N82N5MD2hew== X-Received: by 2002:a6b:9bd1:: with SMTP id d200-v6mr38094388ioe.147.1537489624323; Thu, 20 Sep 2018 17:27:04 -0700 (PDT) Received: from yyz.farcry.sitepalette.com (047-135-146-236.res.spectrum.com. [47.135.146.236]) by smtp.gmail.com with ESMTPSA id w8-v6sm1528977itb.0.2018.09.20.17.27.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Sep 2018 17:27:03 -0700 (PDT) From: Josh Gitlin Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Troubleshooting kernel panic with zfs Message-Id: Date: Thu, 20 Sep 2018 20:27:02 -0400 To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3445.9.1) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2018 00:27:06 -0000 I am working to debug/troubleshoot a kernel panic with a FreeBSD ZFS = iSCSI server, specifically trying to determine if it's a bug or (more = likely) a misconfiguration in our settings. Server is running = 11.2-RELEASE-p2 with 15.6 GiB of RAM and has a single zpool with 4x2 = mirrored vdevs, 2x mirrored zil and 2x l2arc. Server runs pretty much = nothing other than SSH and iSCSI (via ctld) and serves VM virtual disks = to hypervisor servers over 10gbe LAN. The server experienced a kernel panic and we unfortunately did not have = dumpdev set in /etc/rc.conf (we have since corrected this) so the only = info I have is what was on the screen before I rebooted it. (Because = it's a production system I couldn't mess around and had to reboot ASAP) trap number =3D 12 panic: page fault cpuid =3D 6 KDB: stack backtrace: #0 0xffffffff80b3d567 at kdb_backtrace+0x67 #1 0xffffffff80af6b07 at vpanic+0x177 #2 0xffffffff80af6983 at panic+0x43 #3 0xffffffff80f77fcf at trap_fatal+0x35f #4 0xffffffff80f78029 at trap_pfault+0x49 #5 0xffffffff80f777f7 at trap+0x2c7 #6 0xffffffff80f57dac at calltrap+0x8 #7 0xffffffff80dee7e2 at kmem_back+0xf2 #8 0xffffffff80dee6c0 at kmem_malloc+0x60 #9 0xffffffff80de6172 at keg_alloc_slab+0xe2 #10 0xffffffff80de8b7e at keg_fetch_slab+0x14e #11 0xffffffff80de8364 at zone_fetch_slab+0x64 #12 0xffffffff80de848f at zone_import+0x3f #13 0xffffffff80de4b99 at uma_zalloc_arg+0x3d9 #14 0xffffffff826e6ab2 at zio_write_compress+0x1e2 #15 0xffffffff826e574c at zio_execute+0xac #16 0xffffffff80bled74 at taskqueue_run_locked+0x154 #17 0xffffffff80b4fed8 at taskqueue_thread_loop+0x98 Uptime: 18d18h31m6s mpr0: Sending StopUnit: path (xpt0:mpr0:0:10:ffffffff): handle 10=20 mpr0: Incrementing SSU count mpr0: Sending StopUnit: path (xpt0:mpr0:0:13:ffffffff): handle 13=20 mpr0: Incrementing SSU count mpr0: Sending StopUnit: path Ixpt0:mpr0:0:16:ffffffff): handle 16=20 mpr0: Incrementing SSU count My hunch is that, given this was inside kmem_malloc, we were unable to = allocate memory for a zio_write_compress call (the pool does have ZFS = compression on) and hence this is a tuning issue and not a bug... but I = am looking for confirmation and/or suggested changes/troubleshooting = steps. The ZFS tuning configuration has been stable for years, to it may = be a change in behavior or traffic... If this looks like it might be a = bug, I will be able to get more information from a minidump if it = reoccurs and can follow up on this thread. Any advice or suggestions are welcome! [jgitlin@zfs3 ~]$ zpool status pool: srv state: ONLINE scan: scrub repaired 0 in 2h32m with 0 errors on Tue Sep 11 20:32:18 = 2018 config: NAME STATE READ WRITE CKSUM srv ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/s5 ONLINE 0 0 0 gpt/s9 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 gpt/s6 ONLINE 0 0 0 gpt/s10 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 gpt/s7 ONLINE 0 0 0 gpt/s11 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 gpt/s8 ONLINE 0 0 0 gpt/s12 ONLINE 0 0 0 logs mirror-4 ONLINE 0 0 0 gpt/s2-zil ONLINE 0 0 0 gpt/s3-zil ONLINE 0 0 0 cache gpt/s2-cache ONLINE 0 0 0 gpt/s3-cache ONLINE 0 0 0 errors: No known data errors ZFS tuning: vfs.zfs.delay_min_dirty_percent=3D90 vfs.zfs.dirty_data_max=3D4294967296 vfs.zfs.dirty_data_sync=3D3221225472 vfs.zfs.prefetch_disable=3D1 vfs.zfs.top_maxinflight=3D128 vfs.zfs.trim.txg_delay=3D8 vfs.zfs.txg.timeout=3D20 vfs.zfs.vdev.aggregation_limit=3D524288 vfs.zfs.vdev.scrub_max_active=3D3 vfs.zfs.l2arc_write_boost=3D134217728 vfs.zfs.l2arc_write_max=3D134217728 vfs.zfs.l2arc_feed_min_ms=3D200 vfs.zfs.min_auto_ashift=3D12 -- =09 Josh Gitlin Senior DevOps Engineer (415) 690-1610 x155 Stay up to date and join the conversation in Relay = . From owner-freebsd-fs@freebsd.org Fri Sep 21 12:58:24 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4CE31099358 for ; Fri, 21 Sep 2018 12:58:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7F4407D453 for ; Fri, 21 Sep 2018 12:58:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 440141099352; Fri, 21 Sep 2018 12:58:24 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32F591099351 for ; Fri, 21 Sep 2018 12:58:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9DAF7D447 for ; Fri, 21 Sep 2018 12:58:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 23F21215AA for ; Fri, 21 Sep 2018 12:58:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8LCwNEL099991 for ; Fri, 21 Sep 2018 12:58:23 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8LCwNcI099990 for fs@FreeBSD.org; Fri, 21 Sep 2018 12:58:23 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 231516] [ZFS] Crash upon dectection of failed drive. Date: Fri, 21 Sep 2018 12:58:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2018 12:58:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D231516 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri Sep 21 13:04:41 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A34F910999CD for ; Fri, 21 Sep 2018 13:04:41 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smarthost2.sentex.ca", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1610F7DBED for ; Fri, 21 Sep 2018 13:04:40 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5:0:0:0:11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id w8LD4eJA020248 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 21 Sep 2018 09:04:40 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.net [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id w8LD4cRt031176; Fri, 21 Sep 2018 09:04:38 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: Troubleshooting kernel panic with zfs To: Josh Gitlin , freebsd-fs@freebsd.org References: From: Mike Tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; prefer-encrypt=mutual; keydata= xsBNBEzcA24BCACpwI/iqOrs0GfQSfhA1v6Z8AcXVeGsRyKEKUpxoOYxXWc2z3vndbYlIP6E YJeifzKhS/9E+VjhhICaepLHfw865TDTUPr5D0Ed+edSsKjlnDtb6hfNJC00P7eoiuvi85TW F/gAxRY269A5d856bYrzLbkWp2lKUR3Bg6NnORtflGzx9ZWAltZbjYjjRqegPv0EQNYcHqWo eRpXilEo1ahT6nmOU8V7yEvT2j4wlLcQ6qg7w+N/vcBvyd/weiwHU+vTQ9mT61x5/wUrQhdw 2gJHeQXeDGMJV49RT2EEz+QVxaf477eyWsdQzPVjAKRMT3BVdK8WvpYAEfBAbXmkboOxABEB AAHNHG1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5jYT7CwHgEEwECACIFAkzcA24CGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJXHwM2kc8rX+sMH/2V6pTBKsQ5mpWWLgs6wVP2k BC+6r/YKNXv9Rw/PrC6+9hTbgA+sSjJ+8gxsCbJsOQXZrxF0x3l9oYdYfuKcwdwXFX1/FS8p HfBeDkmlH+dI709xT9wgrR4dS5aMmKp0scPrXPIAKiYVOHjOlNItcLYTEEWEFBepheEVsgmk GrNbcrHwOx/u4igUQ8vcpyXPyUki+BsftPw8ZQvBU887igh0OxaCR8AurJppQ5UQd63r81cX E1ZjoFoWCaGK/SjPb/OhpYpu5swoZIhOxQbn7OtakYPsDd5t2A5KhvjI8BMTnd5Go+2xsCmr jlIEq8Bi29gCcfQUvNiClevi13ifmnnOwE0ETNwDbgEIALWGNJHRAhpd0A4vtd3G0oRqMBcM FGThQr3qORmEBTPPEomTdBaHcn+Xl+3YUvTBD/67/mutWBwgp2R5gQOSqcM7axvgMSHbKqBL 9sd1LsLw0UT2O5AYxv3EwzhG84pwRg3XcUqvWA4lA8tIj/1q4Jzi5qOkg1zxq4W9qr9oiYK5 bBR638JUvr3eHMaz/Nz+sDVFgwHmXZj3M6aE5Ce9reCGbvrae7H5D5PPvtT3r22X8SqfVAiO TFKedCf/6jbSOedPN931FJQYopj9P6b3m0nI3ZiCDVSqeyOAIBLzm+RBUIU3brzoxDhYR8pz CJc2sK8l6YjqivPakrD86bFDff8AEQEAAcLAXwQYAQIACQUCTNwDbgIbDAAKCRCVx8DNpHPK 1+iQB/99aqNtez9ZTBWELj269La8ntuRx6gCpzfPXfn6SDIfTItDxTh1hrdRVP5QNGGF5wus N4EMwXouskva1hbFX3Pv72csYSxxEJXjW16oV8WK4KjKXoskLg2RyRP4uXqL7Mp2ezNtVY5F 9nu3fj4ydpHCSaqKy5xd70A8D50PfZsFgkrsa5gdQhPiGGEdxhq/XSeAAnZ4uVLJKarH+mj5 MEhgZPEBWkGrbDZpezl9qbFcUem/uT9x8FYT/JIztMVh9qDcdP5tzANW5J7nvgXjska+VFGY ryZK4SPDczh74mn6GI/+RBi7OUzXXPgpPBrhS5FByjwCqjjsSpTjTds+NGIY Organization: Sentex Communications Message-ID: Date: Fri, 21 Sep 2018 09:04:38 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.83 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2018 13:04:41 -0000 On 9/20/2018 8:27 PM, Josh Gitlin wrote: > > My hunch is that, given this was inside kmem_malloc, we were unable to allocate memory for a zio_write_compress call (the pool does have ZFS compression on) and hence this is a tuning issue and not a bug... but I am looking for confirmation and/or suggested changes/troubleshooting steps. The ZFS tuning configuration has been stable for years, to it may be a change in behavior or traffic... If this looks like it might be a bug, I will be able to get more information from a minidump if it reoccurs and can follow up on this thread. Given all the issues with memory pressure perhaps ARC is eating too much, I would try artificially limiting it so it does not eat up too much RAM. We had a similar purposed server but with 32G RAM and I found one day the system was randomly shooting processes to reclaim RAM eg pid xxx (yyy), uid zzzz, was killed: out of swap space Perhaps sysctl -w vfs.zfs.arc_max=6000000000 eg. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229764 https://lists.freebsd.org/pipermail/freebsd-questions/2018-July/282261.html ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada From owner-freebsd-fs@freebsd.org Fri Sep 21 15:17:46 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98FB0109C99C for ; Fri, 21 Sep 2018 15:17:46 +0000 (UTC) (envelope-from jgitlin@goboomtown.com) Received: from mail-yb1-xb44.google.com (mail-yb1-xb44.google.com [IPv6:2607:f8b0:4864:20::b44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2EB9A830F7 for ; Fri, 21 Sep 2018 15:17:45 +0000 (UTC) (envelope-from jgitlin@goboomtown.com) Received: by mail-yb1-xb44.google.com with SMTP id p74-v6so4973564ybc.9 for ; Fri, 21 Sep 2018 08:17:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=goboomtown-com.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=651tmGGKmscJw5Rgfa5760NHBUTnJikWU7cZLDOXBx4=; b=N8kc+HGZgYbRi0X+uVvERkmWadtdvX/c8Z7j2FeRjrMpJxJBPyFYgKYbNW//aL7oxX VJ+cHGEL6wrKEMbDhsxvCbTgmPQ5M+478DOWum1G0/OT3ebXuJTxWhwIEF+NLnphiap3 KolDR32p27QsqCi+vJz54hbucw2j9kqlLgfezz1eLcvTndJ6H0fm3nRYdT2YpITy0aGU ZWNgQN2IhncW5PbHZfn7rQwqTd2n6V4QKr2wHLzyEmP3aan0Mq5aNcNI49DIXgBPNo2l pWJNE6g2gDP94knhmDvf/c4tdZmW8InU4DwfRk4iO5SdQCtfMQ0pSkXWhHnX7/NS+kwo FoAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=651tmGGKmscJw5Rgfa5760NHBUTnJikWU7cZLDOXBx4=; b=ZN3+ciY1k2JnBFQHnw87PtRW2+egbVd0fGOD+Gik2tR4eriU8A06UztGxXbhcsosLM e6VpveFE7YovWoemRB7xC8KivUXlmeBULl6li2adPmMFRQoIZGaUUCtIz252J/4zHJph dssEQfs2nHaDsBMLDlxfAWDI0WLtAUq/T1422Y89YmVsTxSUl0b7CJY4HAIU/vbGChOZ Ay7J/uDdVjfnE1hWzQHn6d0Uat7hrjZXiHILMD4iMiQDldb0MoBYQOictchDH/NkE+Sf S11/8fB0mqFFbXrfMiu2xfE2g5ai2vN/Br1LM8QLs6+aXbPO3GUhPc9mPYyyqEuR0lrl +4JQ== X-Gm-Message-State: APzg51BoORlc6sgdHymNXqQKV9yJIlsH5zNVaofm15lleoLL9P0h06OX k7zYEIndljLwiBs0k/1hyYezgjfpLdAzPQ== X-Google-Smtp-Source: ANB0Vdbgx/6uPPfA+XxnN9v/48AAu9bfL9Aos7gM9SCjuRGDgCJjZtFN/bmw0/g6eccPthBMX8xb/Q== X-Received: by 2002:a25:e043:: with SMTP id x64-v6mr11492526ybg.520.1537543065192; Fri, 21 Sep 2018 08:17:45 -0700 (PDT) Received: from yyz.farcry.sitepalette.com (047-135-146-236.res.spectrum.com. [47.135.146.236]) by smtp.gmail.com with ESMTPSA id h10-v6sm9540557ywa.35.2018.09.21.08.17.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Sep 2018 08:17:44 -0700 (PDT) From: Josh Gitlin Message-Id: <373102CE-B53A-4547-8665-DF5EC74C23DD@goboomtown.com> Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: Troubleshooting kernel panic with zfs Date: Fri, 21 Sep 2018 11:17:43 -0400 In-Reply-To: To: freebsd-fs@freebsd.org References: X-Mailer: Apple Mail (2.3445.9.1) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2018 15:17:46 -0000 Thanks for the help Mike! In reviewing our settings I also found that = we're setting vm.kmem_size to 1.5x physical RAM installed, based on a = tuning guide someone found. I personally believe that might be misguided = and may be related to the panic we saw, as this causes zfs-stats -a to = show the kernel memory map being larger than system RAM: Kernel Memory Map: 24.00 GiB Size: 21.72% 5.21 GiB Free: 78.28% 18.79 GiB I'll try tuning these settings down and see if the problem happens = again. Appreciate the support! -- =09 Josh Gitlin Senior DevOps Engineer (415) 690-1610 x155 Stay up to date and join the conversation in Relay = . > On Sep 21, 2018, at 9:04 AM, Mike Tancsa wrote: >=20 > On 9/20/2018 8:27 PM, Josh Gitlin wrote: >>=20 >> My hunch is that, given this was inside kmem_malloc, we were unable = to allocate memory for a zio_write_compress call (the pool does have ZFS = compression on) and hence this is a tuning issue and not a bug... but I = am looking for confirmation and/or suggested changes/troubleshooting = steps. The ZFS tuning configuration has been stable for years, to it may = be a change in behavior or traffic... If this looks like it might be a = bug, I will be able to get more information from a minidump if it = reoccurs and can follow up on this thread. >=20 > Given all the issues with memory pressure perhaps ARC is eating too > much, I would try artificially limiting it so it does not eat up too > much RAM. We had a similar purposed server but with 32G RAM and I = found > one day the system was randomly shooting processes to reclaim RAM >=20 > eg > pid xxx (yyy), uid zzzz, was killed: out of swap space >=20 >=20 > Perhaps >=20 > sysctl -w vfs.zfs.arc_max=3D6000000000 >=20 > eg. > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229764 >=20 > = https://lists.freebsd.org/pipermail/freebsd-questions/2018-July/282261.htm= l >=20 > ---Mike >=20 >=20 >=20 >=20 > --=20 > ------------------- > Mike Tancsa, tel +1 519 651 3400 x203 > Sentex Communications, mike@sentex.net > Providing Internet services since 1994 www.sentex.net > Cambridge, Ontario Canada From owner-freebsd-fs@freebsd.org Fri Sep 21 15:50:41 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB32F109D42C for ; Fri, 21 Sep 2018 15:50:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 43412842D1 for ; Fri, 21 Sep 2018 15:50:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 049C2109D42B; Fri, 21 Sep 2018 15:50:41 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE8ED109D42A for ; Fri, 21 Sep 2018 15:50:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 60471842BF for ; Fri, 21 Sep 2018 15:50:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id BF69722D4D for ; Fri, 21 Sep 2018 15:50:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8LFodP9087316 for ; Fri, 21 Sep 2018 15:50:39 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8LFodWM087315 for fs@FreeBSD.org; Fri, 21 Sep 2018 15:50:39 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 231516] [ZFS] Crash upon dectection of failed drive. Date: Fri, 21 Sep 2018 15:50:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: andyf@andyit.com.au X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Sep 2018 15:50:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D231516 Andy Farkas changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |andyf@andyit.com.au --- Comment #3 from Andy Farkas --- Recently saw a ZFS crash due to 2 dying/dead disks (of 4) in a RAIDZ1: FreeBSD 11.1-STABLE #0 r331113: Sun Mar 18 19:57:45 AEST 2018 panic: solaris assert: remove_reference(hdr, ((void *)0), tag) =3D=3D 0 (0x= d =3D=3D 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c, line: 3463 cpuid =3D 7 KDB: stack backtrace: #0 0xffffffff805cf167 at kdb_backtrace+0x67 #1 0xffffffff8058cc46 at vpanic+0x186 #2 0xffffffff8058cab3 at panic+0x43 #3 0xffffffff815cb22c at assfail3+0x2c #4 0xffffffff8124cb50 at arc_buf_destroy+0x1a0 #5 0xffffffff8125c56a at dbuf_read_done+0x4a #6 0xffffffff8124f236 at arc_read_done+0x296 #7 0xffffffff812e357e at zio_done+0x80e #8 0xffffffff812df05c at zio_execute+0xac #9 0xffffffff805e0714 at taskqueue_run_locked+0x154 #10 0xffffffff805e1858 at taskqueue_thread_loop+0x98 #11 0xffffffff805596a2 at fork_exit+0x82 #12 0xffffffff8074a13e at fork_trampoline+0xe Uptime: 53s The culprits: (da2:mps0:0:5:0): SCSI sense: MEDIUM ERROR asc:11,0 (Unrecovered read error) (da2:mps0:0:5:0): Error 5, Unretryable error (da2:mps0:0:5:0): READ(16). CDB: 88 00 00 00 00 01 cf 2f 5c dc 00 00 00 55 = 00 00=20 (da2:mps0:0:5:0): CAM status: SCSI Status Error (da2:mps0:0:5:0): SCSI status: Check Condition (da2:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or = bus device reset occurred) (da2:mps0:0:5:0): Error 6, Retries exhausted (da2:mps0:0:5:0): Invalidating pack ... Sep 21 11:50:16 drunkfish smartd[1060]: Device: /dev/da3 [SAT= ], 39538 Currently unreadable (pending) sectors --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Sep 22 02:53:19 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B295C10ADCA5 for ; Sat, 22 Sep 2018 02:53:19 +0000 (UTC) (envelope-from FreeBSD@shaneware.biz) Received: from ipmail06.adl6.internode.on.net (ipmail06.adl6.internode.on.net [150.101.137.145]) by mx1.freebsd.org (Postfix) with ESMTP id ECB427C806 for ; Sat, 22 Sep 2018 02:53:18 +0000 (UTC) (envelope-from FreeBSD@shaneware.biz) Received: from ppp118-210-165-15.bras2.adl6.internode.on.net (HELO leader.local) ([118.210.165.15]) by ipmail06.adl6.internode.on.net with ESMTP; 22 Sep 2018 12:23:09 +0930 Subject: Re: Troubleshooting kernel panic with zfs To: Mike Tancsa , Josh Gitlin , freebsd-fs@freebsd.org References: From: Shane Ambler Message-ID: <41d177bf-117b-10c3-06a0-7d74d298fede@ShaneWare.Biz> Date: Sat, 22 Sep 2018 12:23:06 +0930 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-AU Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2018 02:53:19 -0000 On 21/9/18 10:34 pm, Mike Tancsa wrote: > > Given all the issues with memory pressure perhaps ARC is eating too > much, I would try artificially limiting it so it does not eat up too > much RAM. We had a similar purposed server but with 32G RAM and I found > one day the system was randomly shooting processes to reclaim RAM > > eg > pid xxx (yyy), uid zzzz, was killed: out of swap space > I have seen these errors, for me they came when >90% ram is wired. This is something I reported in bug 229764. While ARC is wired, there is also a max_wired that limits other wired allocations, these two are independent and can total more than physical ram. max_wired defaults to 30% ram, so arc_max should be kept less than 70% I also had positive results applying review D7538 -- FreeBSD - the place to B...Storing Data Shane Ambler From owner-freebsd-fs@freebsd.org Sat Sep 22 06:07:59 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04FB910B279C for ; Sat, 22 Sep 2018 06:07:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 96F328366F for ; Sat, 22 Sep 2018 06:07:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 58DEB10B2799; Sat, 22 Sep 2018 06:07:58 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47C7510B2798 for ; Sat, 22 Sep 2018 06:07:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DDB158366D for ; Sat, 22 Sep 2018 06:07:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 37C78259F for ; Sat, 22 Sep 2018 06:07:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w8M67vad097189 for ; Sat, 22 Sep 2018 06:07:57 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w8M67vUv097183 for fs@FreeBSD.org; Sat, 22 Sep 2018 06:07:57 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 231569] ZFS per-disk status not showing "resilvering" during resilver Date: Sat, 22 Sep 2018 06:07:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Sep 2018 06:07:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D231569 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.=