From owner-freebsd-fs@FreeBSD.ORG Sun Jun 14 13:26:24 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A0A29A for ; Sun, 14 Jun 2015 13:26:24 +0000 (UTC) (envelope-from juergen.gotteswinter@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id D022A39D for ; Sun, 14 Jun 2015 13:26:23 +0000 (UTC) (envelope-from juergen.gotteswinter@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 261821472003; Sun, 14 Jun 2015 15:26:20 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8t9kScVikemo; Sun, 14 Jun 2015 15:26:17 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 8C3481472002; Sun, 14 Jun 2015 15:26:17 +0200 (CEST) Message-ID: <557D80F8.9000505@internetx.com> Date: Sun, 14 Jun 2015 15:26:16 +0200 From: InterNetX - Juergen Gotteswinter Reply-To: jg@internetx.com Organization: InterNetX GmbH User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: =?UTF-8?B?S2FybGkgU2rDtmJlcmc=?= , Andreas Nilsson , "freebsd-fs@freebsd.org" Subject: Re: [Fwd: Strange networking behaviour in storage server] References: <1433146506.14998.177.camel@data-b104.adm.slu.se> <1433149349.14998.181.camel@data-b104.adm.slu.se> <20150613093117.GB37870@brick.home> <557D8092.7050301@internetx.com> In-Reply-To: <557D8092.7050301@internetx.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jun 2015 13:26:24 -0000 Am 14.06.2015 um 15:24 schrieb InterNetX - Juergen Gotteswinter: > > > Am 13.06.2015 um 11:31 schrieb Edward Tomasz Napierała: >> On 0601T0902, Karli Sjöberg wrote: >>> mån 2015-06-01 klockan 10:33 +0200 skrev Andreas Nilsson: >>>> >>>> >>>> On Mon, Jun 1, 2015 at 10:14 AM, Karli Sjöberg >>>> wrote: >>>> -------- Vidarebefordrat meddelande -------- >>>> > Från: Karli Sjöberg >>>> > Till: freebsd-fs@freebsd.org >>>> > Ämne: Strange networking behaviour in storage server >>>> > Datum: Mon, 1 Jun 2015 07:49:56 +0000 >>>> > >>>> > Hey! >>>> > >>>> > So we have this ZFS storage server upgraded from 9.3-RELEASE >>>> to >>>> > 10.1-STABLE to overcome not being able to 1) use SSD drives >>>> as >>>> > L2ARC[1] >>>> > and 2) not being able to hotswap SATA drives[2]. >>>> > >>>> > After the upgrade we´ve noticed a very odd networking >>>> behaviour, it >>>> > sends/receives full speed for a while, then there is a >>>> couple of >>>> > minutes >>>> > of complete silence where even terminal commands like an >>>> "ls" just >>>> > waits >>>> > until they are executed and then it starts sending full >>>> speed again. I >>>> > ´ve linked to a screenshot showing this send and pause >>>> behaviour. The >>>> > blue line is the total, green is SMB and turquoise is NFS >>>> over jumbo >>>> > frames. It behaves this way regardless of the protocol. >>>> > >>>> > http://oi62.tinypic.com/33xvjb6.jpg >>>> > >>>> > The problem is that these pauses can sometimes be so long >>>> that >>>> > connections drop. Like someone is copying files over SMB or >>>> iSCSI and >>>> > suddenly they get an error message saying that the transfer >>>> failed and >>>> > they have to start over with the file(s). That´s horrible! >>>> > >>>> > So far NFS has proven to be the most resillient, it´s stupid >>>> simple >>>> > nature just waits and resumes transfer when pause is over. >>>> Kudus for >>>> > that. >>>> > >>>> > The server is driven by a Supermicro X9SRL-F, a Xeon 1620v2 >>>> and 64GB >>>> > ECC >>>> > RAM. The hardware has been ruled out, we happened to have a >>>> identical >>>> > MB >>>> > and CPU lying around and that didn´t improve things. We have >>>> also >>>> > installed a Intel PRO 100/1000 Quad-port ethernet adapter to >>>> test if >>>> > that would change things, but it hasn´t, it still behaves >>>> this way. >>>> > >>>> > The two built-in NIC's are Intel 82574L and the Quad-port >>>> NIC's are >>>> > Intel 82571EB, so both em(4) driven. I happen to know that >>>> the em >>>> > driver >>>> > has updated between 9.3 and 10.1. Perhaps that is to blame, >>>> but I have >>>> > no idea. >>>> > >>>> > Is there anyone that can make sense of this? >>>> > >>>> > [1]: >>>> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197164 >>>> > >>>> > [2]: >>>> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191348 >>>> > >>>> > /K >>>> > >>>> > >>>> >>>> >>>> Another observation I´ve made is that during these pauses, the >>>> entire >>>> system is put on hold, even ZFS scrub stops and then resumes >>>> after a >>>> while. Looking in top, the system is completly idle. >>>> >>>> Normally during scrub, the kernel eats 20-30% CPU, but during >>>> a pause, >>>> even the [kernel] goes down to 0.00%. Makes me think the >>>> networking has >>>> nothing to do with it. >>>> >>>> What´s then to blame? ZFS? >>>> >>>> /K >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to >>>> "freebsd-fs-unsubscribe@freebsd.org" >>>> >>>> >>>> Hello, >>>> >>>> >>>> does this happen when clients are only reading from server? >>> >>> Yes it happens when clients are only reading from the server. >>> >>>> Otherwise I would suspect that it could be caused by ZFS writing out a >>>> large chunck of data sitting in its caches, and until that is complete >>>> I/O is stalled. >>> >>> That´s what so strange, we have three more systems set up about the same >>> size and none of others are acting this way. >>> >>> The only thing I can think of that differs that we haven´t tested ruling >>> out yet is ctld, the other systems are still running istgt as their >>> iSCSI daemon. >> >> So, were you able to rule out ctld? >> >> Do you have local, or terminal, access to the machine? When the problem >> manifests, do local commands work? In other words, is the whole machine >> wedged, or just the network? If it's just the network, it might be >> caused by ctld consuming all available mbufs. You could run "netstat -m" >> before and after to check that. >> > > You already checked (doublechecked) HBA Firmware etc? Cabling is fine? > > I expect you already disabled tso, gro, rxcsum, txcsum on your NIC(s). I > had similar effects, with all those fancy uberfeatures enabled. > > Give it a try... ifconfig foo0 -rxcsum -txcsum -tso -gro > > Capturing a few MB of Traffic before/after could be also very helpful to > see if... > errm, sorry. Forgot something... how does your Network Setup look like? Link Aggregations? Which Switches, which Linespeed, Stacked or not? Any Drops / Errors on your Interfaces? > >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> From owner-freebsd-fs@FreeBSD.ORG Sun Jun 14 13:30:39 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BB91B244 for ; Sun, 14 Jun 2015 13:30:39 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5D02D69B for ; Sun, 14 Jun 2015 13:30:39 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 8EF874C4C870; Sun, 14 Jun 2015 15:24:38 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id o5EbeYbbfe+w; Sun, 14 Jun 2015 15:24:36 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id EB24A4C4C842; Sun, 14 Jun 2015 15:24:35 +0200 (CEST) Message-ID: <557D8092.7050301@internetx.com> Date: Sun, 14 Jun 2015 15:24:34 +0200 From: InterNetX - Juergen Gotteswinter Reply-To: jg@internetx.com User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: =?UTF-8?B?S2FybGkgU2rDtmJlcmc=?= , Andreas Nilsson , "freebsd-fs@freebsd.org" Subject: Re: [Fwd: Strange networking behaviour in storage server] References: <1433146506.14998.177.camel@data-b104.adm.slu.se> <1433149349.14998.181.camel@data-b104.adm.slu.se> <20150613093117.GB37870@brick.home> In-Reply-To: <20150613093117.GB37870@brick.home> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jun 2015 13:30:39 -0000 Am 13.06.2015 um 11:31 schrieb Edward Tomasz Napierała: > On 0601T0902, Karli Sjöberg wrote: >> mån 2015-06-01 klockan 10:33 +0200 skrev Andreas Nilsson: >>> >>> >>> On Mon, Jun 1, 2015 at 10:14 AM, Karli Sjöberg >>> wrote: >>> -------- Vidarebefordrat meddelande -------- >>> > Från: Karli Sjöberg >>> > Till: freebsd-fs@freebsd.org >>> > Ämne: Strange networking behaviour in storage server >>> > Datum: Mon, 1 Jun 2015 07:49:56 +0000 >>> > >>> > Hey! >>> > >>> > So we have this ZFS storage server upgraded from 9.3-RELEASE >>> to >>> > 10.1-STABLE to overcome not being able to 1) use SSD drives >>> as >>> > L2ARC[1] >>> > and 2) not being able to hotswap SATA drives[2]. >>> > >>> > After the upgrade we´ve noticed a very odd networking >>> behaviour, it >>> > sends/receives full speed for a while, then there is a >>> couple of >>> > minutes >>> > of complete silence where even terminal commands like an >>> "ls" just >>> > waits >>> > until they are executed and then it starts sending full >>> speed again. I >>> > ´ve linked to a screenshot showing this send and pause >>> behaviour. The >>> > blue line is the total, green is SMB and turquoise is NFS >>> over jumbo >>> > frames. It behaves this way regardless of the protocol. >>> > >>> > http://oi62.tinypic.com/33xvjb6.jpg >>> > >>> > The problem is that these pauses can sometimes be so long >>> that >>> > connections drop. Like someone is copying files over SMB or >>> iSCSI and >>> > suddenly they get an error message saying that the transfer >>> failed and >>> > they have to start over with the file(s). That´s horrible! >>> > >>> > So far NFS has proven to be the most resillient, it´s stupid >>> simple >>> > nature just waits and resumes transfer when pause is over. >>> Kudus for >>> > that. >>> > >>> > The server is driven by a Supermicro X9SRL-F, a Xeon 1620v2 >>> and 64GB >>> > ECC >>> > RAM. The hardware has been ruled out, we happened to have a >>> identical >>> > MB >>> > and CPU lying around and that didn´t improve things. We have >>> also >>> > installed a Intel PRO 100/1000 Quad-port ethernet adapter to >>> test if >>> > that would change things, but it hasn´t, it still behaves >>> this way. >>> > >>> > The two built-in NIC's are Intel 82574L and the Quad-port >>> NIC's are >>> > Intel 82571EB, so both em(4) driven. I happen to know that >>> the em >>> > driver >>> > has updated between 9.3 and 10.1. Perhaps that is to blame, >>> but I have >>> > no idea. >>> > >>> > Is there anyone that can make sense of this? >>> > >>> > [1]: >>> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197164 >>> > >>> > [2]: >>> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191348 >>> > >>> > /K >>> > >>> > >>> >>> >>> Another observation I´ve made is that during these pauses, the >>> entire >>> system is put on hold, even ZFS scrub stops and then resumes >>> after a >>> while. Looking in top, the system is completly idle. >>> >>> Normally during scrub, the kernel eats 20-30% CPU, but during >>> a pause, >>> even the [kernel] goes down to 0.00%. Makes me think the >>> networking has >>> nothing to do with it. >>> >>> What´s then to blame? ZFS? >>> >>> /K >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to >>> "freebsd-fs-unsubscribe@freebsd.org" >>> >>> >>> Hello, >>> >>> >>> does this happen when clients are only reading from server? >> >> Yes it happens when clients are only reading from the server. >> >>> Otherwise I would suspect that it could be caused by ZFS writing out a >>> large chunck of data sitting in its caches, and until that is complete >>> I/O is stalled. >> >> That´s what so strange, we have three more systems set up about the same >> size and none of others are acting this way. >> >> The only thing I can think of that differs that we haven´t tested ruling >> out yet is ctld, the other systems are still running istgt as their >> iSCSI daemon. > > So, were you able to rule out ctld? > > Do you have local, or terminal, access to the machine? When the problem > manifests, do local commands work? In other words, is the whole machine > wedged, or just the network? If it's just the network, it might be > caused by ctld consuming all available mbufs. You could run "netstat -m" > before and after to check that. > You already checked (doublechecked) HBA Firmware etc? Cabling is fine? I expect you already disabled tso, gro, rxcsum, txcsum on your NIC(s). I had similar effects, with all those fancy uberfeatures enabled. Give it a try... ifconfig foo0 -rxcsum -txcsum -tso -gro Capturing a few MB of Traffic before/after could be also very helpful to see if... > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Sun Jun 14 21:00:29 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8AE7917 for ; Sun, 14 Jun 2015 21:00:29 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 90C01A35 for ; Sun, 14 Jun 2015 21:00:29 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t5EL0Tv5004508 for ; Sun, 14 Jun 2015 21:00:29 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201506142100.t5EL0Tv5004508@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 Date: Sun, 14 Jun 2015 21:00:29 +0000 Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jun 2015 21:00:29 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f 3 problems total for which you should take action. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 15 21:30:33 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C5C96BA for ; Mon, 15 Jun 2015 21:30:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 66A64126 for ; Mon, 15 Jun 2015 21:30:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t5FLUX3k093679 for ; Mon, 15 Jun 2015 21:30:33 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 199189] SWAP on ZFS can crash server Date: Mon, 15 Jun 2015 21:30:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: marcus@blazingdot.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jun 2015 21:30:33 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199189 --- Comment #15 from Marcus Reid --- Just another anecdotal observation.. I've got a fresh -CURRENT machine that I set up with swap on a zvol and am able to get it to fail reliably. There's a git process that eats all ram and then goes about 1GB into swap (with 4GB available) and then swap just appears to stop working. A bunch of processes get stuck in a pfault state and the box becomes unusable and needs to be reset. The zvol is set up with compression, dedup, primarycache, sync all off. Something is definitely still very broken. My workaround in this case is to use a vnode-backed md device as a swap file, which is still not ideal but is stable enough until I can rebuild the machine with normal swap partitions. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 16 06:21:36 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AC650CF5 for ; Tue, 16 Jun 2015 06:21:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9730A32F for ; Tue, 16 Jun 2015 06:21:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t5G6La7m058276 for ; Tue, 16 Jun 2015 06:21:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 199189] SWAP on ZFS can crash server Date: Tue, 16 Jun 2015 06:21:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: marcus@blazingdot.com X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jun 2015 06:21:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199189 --- Comment #16 from Marcus Reid --- As for the machine in my last comment, I've tuned the problem away and I'd like to share how: vm.v_free_target=60000 vm.v_free_min=55000 vm.v_free_severe=50000 vm.v_free_reserved=32000 vfs.zfs.arc_free_target=180000 Now, free memory never gets down below about 50MB or so, and normally stays up above 150MB even when under pressure and swapping. I'm swapping to a zvol with all of the swap tunables set. The default values for these look more like: vm.v_free_target: 42948 vm.v_free_min: 12741 vm.v_free_severe: 7706 vm.v_free_reserved: 2672 vfs.zfs.arc_free_target: 14014 Note that I increased vm.v_free_reserved quite a bit, but I don't know exactly what that one does. vm.v_free_reserved: Pages reserved for deadlock The description seems to make it a prime candidate for tuning in this situation, so even though nobody has mentioned it yet I raised it. Things work great. I will post again if I manage to break it again. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 17 01:51:52 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC5A46B5 for ; Wed, 17 Jun 2015 01:51:52 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-ie0-x243.google.com (mail-ie0-x243.google.com [IPv6:2607:f8b0:4001:c03::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BB476901 for ; Wed, 17 Jun 2015 01:51:52 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: by ierx19 with SMTP id x19so1492476ier.0 for ; Tue, 16 Jun 2015 18:51:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=3ONL3MHd2mgGOXBCLO/QJV9vfGA0DfeJ2pPVVmC1WNE=; b=TVUsNf2EUAFXjGOoBhur9B0wgd45W/RFbJP0f7FQZCffWcijEFPg5hgj7zTAufUsvv Xx7gsP/3/tCCvVDNHV3d5WTOHnSwy7N9ru5wRS03wmZuxP9soVaA71pJ6t6gr7CqznlZ pmtCIPbx8DAEd0VynsHoiEgtYyga24oOqhe/xYz2IPSBmI5YDe0fEH3J2gPadQWekRW4 BfJtu/Yq70IoBKxpEM3mbRKJRnaNvFj0JmdISdgA9t21CSTRTFPA+0xtFzcaKFLK09N+ JNchBJzYuIMcY8j6bCQSGIW8W0ZvJeSi5swhPuNxNynxEtNoERZ1UWLDKxpya/MFOcmh 9kzw== MIME-Version: 1.0 X-Received: by 10.42.120.66 with SMTP id e2mr6073031icr.37.1434505912151; Tue, 16 Jun 2015 18:51:52 -0700 (PDT) Received: by 10.36.51.76 with HTTP; Tue, 16 Jun 2015 18:51:52 -0700 (PDT) Date: Tue, 16 Jun 2015 21:51:52 -0400 Message-ID: Subject: TRIM erases user data From: grarpamp To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 01:51:53 -0000 https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/ https://github.com/torvalds/linux/blob/e64f638483a21105c7ce330d543fa1f1c35b5bc7/drivers/ata/libata-core.c#L4109-L4286 http://www.aerospike.com/docs/operations/plan/ssd/ssd_certification.html From owner-freebsd-fs@FreeBSD.ORG Wed Jun 17 12:26:41 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F2B31F21; Wed, 17 Jun 2015 12:26:41 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4F16135; Wed, 17 Jun 2015 12:26:40 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:Subject:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=GkheXm+arWIdhTUndK33yJIWRBPzOty112R+yTfrVgQ=; b=IIi2+JqefyNjd5Psl+QgcN/T/NsGjdDs+iNdMmb7L0HBgfgLnL3fPKK2cnmOkSZSJ4EZ+nmvNdQRXdhKQCgtU1a5SReGkLwHBDUTyXFqptwYTJ9RFk4y1yTtF3hpOtCNCmtLDEheLXMyrbVuGkL0tgiDqucscm0S35wr4/hPuuQ=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:33785 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1Z5CQN-000Cly-OB; Wed, 17 Jun 2015 07:26:35 -0500 Received: from 104-54-221-134.lightspeed.austtx.sbcglobal.net ([104.54.221.134]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Wed, 17 Jun 2015 07:26:35 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 17 Jun 2015 07:26:35 -0500 From: Larry Rosenman To: Freebsd fs , rmacklem@FreeBSD.org Subject: NFS Mount and LARGE amounts of "INACT" memory Message-ID: <3a845167bb6123139c4585659e8f0d43@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.1.1 X-Spam-Score: -1.0 (-) X-LERCTR-Spam-Score: -1.0 (-) X-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1,SHORTCIRCUIT=-0.0001 X-LERCTR-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1, SHORTCIRCUIT=-0.0001 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 12:26:41 -0000 I have a 64G memory FreeBSD 11-CURRENT system that has a couple of mounts to a FreeNAS (FreeBSD 9.3) system. When my rsync from a different system to one of the NFS mounts runs, I get like 48G of Inactive memory that goes back to free if I umount the share. I'm wondering why this memory moves from ZFS ARC to INACT. And, is this expected? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Wed Jun 17 15:40:34 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2EB7A75C for ; Wed, 17 Jun 2015 15:40:34 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.81]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E8829ACF for ; Wed, 17 Jun 2015 15:40:33 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1Z5FRx-0001L5-GH for freebsd-fs@freebsd.org; Wed, 17 Jun 2015 17:40:30 +0200 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-fs@freebsd.org Date: Wed, 17 Jun 2015 17:40:24 +0200 Subject: 11-CURRENT does not mount my root ZFS MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.2 X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 autolearn=disabled version=3.3.2 X-Scan-Signature: 74bd734068bee68206891dc8710ce62a X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 15:40:34 -0000 Hello, I'm running 10-STABLE on my laptop on ZFS for a while already. Today I compiled and installed a 11-CURRENT kernel. After boot the kernel gives this error at the moment of mountroot. 'Mounting from zfs:zroot/root failed with error 45.' What can be the cause of this? gpart show => 34 976773101 ada0 GPT (466G) 34 1024 1 freebsd-boot (512K) 1058 5086 - free - (2.5M) 6144 8386560 2 freebsd-swap (4.0G) 8392704 968380416 3 freebsd-zfs (462G) 976773120 15 - free - (7.5K) pool: zroot state: ONLINE scan: scrub repaired 0 in 0h34m with 0 errors on Thu May 21 03:37:36 2015 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 gptid/d7420f5f-0db1-11e4-af73-002170466cda ONLINE 0 0 0 zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 95.2G 350G 144K none zroot/data 1.52G 350G 96K /data zroot/data/ronald 1.52G 350G 434M /data/ronald zroot/home 50.5G 350G 45.4G /home zroot/root 1.71G 350G 427M / zroot/usr 35.7G 350G 4.23G /usr zroot/usr/obj 5.14G 350G 1.45G /usr/obj zroot/usr/obj-arm 5.98G 350G 1.14G /usr/obj-arm zroot/usr/ports 9.39G 350G 1.27G /usr/ports zroot/usr/ports/distfiles 6.95G 350G 1.09G /usr/ports/distfiles zroot/usr/ports/packages 198M 350G 112K /usr/ports/packages zroot/usr/src 1.53G 350G 1.23G /usr/src zroot/usr/src-arm 2.63G 350G 1.97G /usr/src-arm zroot/var 5.62G 350G 1.63G /var If have tried with vfs.root.mountfrom="zfs:zroot/root" in loader.conf as well as with bootfs=zroot/root property on the pool. What could be the cause of this? Can I provide more information? Regards, Ronald. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 17 17:02:38 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 73DEC686 for ; Wed, 17 Jun 2015 17:02:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A55303FD for ; Wed, 17 Jun 2015 17:02:34 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA00580; Wed, 17 Jun 2015 20:02:31 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Z5GjP-000CAD-4V; Wed, 17 Jun 2015 20:02:31 +0300 Message-ID: <5581A7EF.5080606@FreeBSD.org> Date: Wed, 17 Jun 2015 20:01:35 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Ronald Klop , freebsd-fs@FreeBSD.org Subject: Re: 11-CURRENT does not mount my root ZFS References: In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 17:02:38 -0000 On 17/06/2015 18:40, Ronald Klop wrote: > Hello, > > I'm running 10-STABLE on my laptop on ZFS for a while already. > Today I compiled and installed a 11-CURRENT kernel. After boot the kernel gives > this error at the moment of mountroot. [snip] > > What could be the cause of this? Can I provide more information? That would be very weird but perhaps the problem is caused by a mismatch in pool features? Of course, it's hard to imagine that the CURRENT kernel would not support something that 10-STABLE supported... However, zpool get all output might still be informative. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Wed Jun 17 17:22:34 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10D6960B; Wed, 17 Jun 2015 17:22:34 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id ACF2FB13; Wed, 17 Jun 2015 17:22:33 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.9/8.14.9) with ESMTP id t5HHMNEY017421 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 17 Jun 2015 11:22:31 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.9/8.14.9/Submit) with ESMTP id t5HHMNdk017418; Wed, 17 Jun 2015 11:22:23 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Wed, 17 Jun 2015 11:22:23 -0600 (MDT) From: Warren Block To: Andriy Gapon cc: Ronald Klop , freebsd-fs@FreeBSD.org Subject: Re: 11-CURRENT does not mount my root ZFS In-Reply-To: <5581A7EF.5080606@FreeBSD.org> Message-ID: References: <5581A7EF.5080606@FreeBSD.org> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Wed, 17 Jun 2015 11:22:32 -0600 (MDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 17:22:34 -0000 On Wed, 17 Jun 2015, Andriy Gapon wrote: > On 17/06/2015 18:40, Ronald Klop wrote: >> Hello, >> >> I'm running 10-STABLE on my laptop on ZFS for a while already. >> Today I compiled and installed a 11-CURRENT kernel. After boot the kernel gives >> this error at the moment of mountroot. > [snip] >> >> What could be the cause of this? Can I provide more information? > > That would be very weird but perhaps the problem is caused by a mismatch in pool > features? Of course, it's hard to imagine that the CURRENT kernel would not > support something that 10-STABLE supported... > However, zpool get all output might still be informative. Outdated boot code on all but one drive? (Sorry, no experience booting ZFS from MBR.) From owner-freebsd-fs@FreeBSD.ORG Wed Jun 17 21:46:10 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADA3D21C for ; Wed, 17 Jun 2015 21:46:10 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: from mail-pa0-f53.google.com (mail-pa0-f53.google.com [209.85.220.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8533AA3D for ; Wed, 17 Jun 2015 21:46:10 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: by padev16 with SMTP id ev16so45393242pad.0 for ; Wed, 17 Jun 2015 14:46:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=nBG1fFA41pnWaEjJpGQzm0kMJ4bhBi4uVPaXlBw8yPU=; b=jSKMUybaRsJkQHMLU8+/XFWDGH0L/yyuM075Yq3LCaDniXIYwa7wA+7LAC64i/KOyF GdK/OYP07kbg0ys1Il3eeJFrFkmSUL/9mC3hpcmC9lJAH22/NhUgnCAzddeOqj8a/aUS DCbeM4apOHtebqQmp97ghRG6zCkMomjO3sDiOmHJQO2aE+Fc8dlAqEZVPaNBKYUyebDK 7L6jsxovENB8HlLxFZNrFyycdXquO+FKzku0rRpzeeGy/3cyUzTVVuHpDOD77rFBk+GR y3fE0usWvHeV7tzwjtL6s1z7Z7+Pv1FchLGA3M2bXufWQs/3A8Jom6kGxwsxAWfN2lwI DRGw== X-Gm-Message-State: ALoCoQn3c6Zvlx/mRK0h0IlSk5fTPsz/uwBTER0EaAkT1C6i1bAzf5og+HlJANtHcmTXTbG87tSw X-Received: by 10.70.33.67 with SMTP id p3mr14865026pdi.126.1434577206097; Wed, 17 Jun 2015 14:40:06 -0700 (PDT) Received: from ?IPv6:2602:306:c4aa:cfc0:7c50:77a5:d6fa:8d13? ([2602:306:c4aa:cfc0:7c50:77a5:d6fa:8d13]) by mx.google.com with ESMTPSA id om4sm5717224pdb.68.2015.06.17.14.40.03 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 17 Jun 2015 14:40:04 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: TRIM erases user data From: steven@multiplay.co.uk X-Mailer: iPad Mail (12F69) In-Reply-To: Date: Wed, 17 Jun 2015 14:40:02 -0700 Cc: "freebsd-fs@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <49FEE0B5-D924-4FB6-889B-54F18667239B@multiplay.co.uk> References: To: grarpamp X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 21:46:10 -0000 This issue centers around queued TRIM requests at the ATA layer, which is an= extension that allows NCQ support for TRIM in the SATA 3.1 spec. This is no= t something FreeBSD currently supports so is unaffected by the issue at this= time. Sent from my iPad > On 16 Jun 2015, at 18:51, grarpamp wrote: >=20 > https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/ > https://github.com/torvalds/linux/blob/e64f638483a21105c7ce330d543fa1f1c35= b5bc7/drivers/ata/libata-core.c#L4109-L4286 > http://www.aerospike.com/docs/operations/plan/ssd/ssd_certification.html > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Wed Jun 17 22:58:29 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E3A94B2D for ; Wed, 17 Jun 2015 22:58:29 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 977FDDA1 for ; Wed, 17 Jun 2015 22:58:29 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from amavis-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3mBhdY1grnz1Sc for ; Thu, 18 Jun 2015 00:58:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla4; t=1434581901; x=1437173902; bh=tzX wa6J4oytFwRUBlBHkUiL+eMbRXwOmOZ5LNpnyt5g=; b=CM6PFYeSCJI1MR5Sq3K uHEBniWGqFem/GsBAyO30fA2MEcowG9ItFBAn1QBfN7fEcRz5VEwqm22QVy8ctJ0 yVUSeNHVr0klrCNMO8ZI2mpoDPaDulwGskxBLLlVdQaQsdxQr55cK9kaOXvnG7mT nWno35JpQekoFPM0n35/8BSY= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10026) with LMTP id GqGiJGGVj6h1 for ; Thu, 18 Jun 2015 00:58:21 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP id 3mBhdT6R71z1Sb for ; Thu, 18 Jun 2015 00:58:21 +0200 (CEST) Received: from neli.ijs.si (periskop.ijs.si [IPv6:2001:1470:ff80:88::161:2]) by mildred.ijs.si (Postfix) with ESMTP id 3mBhdT6C5pzNd for ; Thu, 18 Jun 2015 00:58:21 +0200 (CEST) Received: from sleepy.ijs.si ([2001:1470:ff80:e001::1:1]) by neli.ijs.si with HTTP (HTTP/1.1 POST); Thu, 18 Jun 2015 00:58:21 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 18 Jun 2015 00:58:21 +0200 From: Mark Martinec To: freebsd-fs@freebsd.org Subject: Re: TRIM erases user data Organization: J. Stefan Institute In-Reply-To: <49FEE0B5-D924-4FB6-889B-54F18667239B@multiplay.co.uk> References: <49FEE0B5-D924-4FB6-889B-54F18667239B@multiplay.co.uk> Message-ID: X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.1.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 22:58:30 -0000 >> On 16 Jun 2015, at 18:51, grarpamp wrote: >> https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/ >> https://github.com/torvalds/linux/blob/e64f638483a21105c7ce330d543fa1f1c35b5bc7/drivers/ata/libata-core.c#L4109-L4286 >> http://www.aerospike.com/docs/operations/plan/ssd/ssd_certification.html steven@multiplay.co.uk wrote: > This issue centers around queued TRIM requests at the ATA layer, which > is an extension that allows NCQ support for TRIM in the SATA 3.1 spec. > This is not something FreeBSD currently supports so is unaffected by > the issue at this time. Are you sure? The article explicitly states they were not using queued TRIM: | A lot of discussions started pointing out that the issue is related | to the newly introduced queued TRIM. This is not correct. The TRIM | on our drives is un-queued and the issue we have found is not related | to the latest changes in the Linux Kernel to disable this features. Mark From owner-freebsd-fs@FreeBSD.ORG Wed Jun 17 23:25:08 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 612DAFE0 for ; Wed, 17 Jun 2015 23:25:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4AA5674C for ; Wed, 17 Jun 2015 23:25:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t5HNP8Lt048019 for ; Wed, 17 Jun 2015 23:25:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 200585] [nlm] Fatal trap 9 when printing out KASSERT trying to run umount -f on an NFS share while it's trying to print out "lockd not responding" in nlm(4) Date: Wed, 17 Jun 2015 23:25:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 23:25:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200585 --- Comment #9 from commit-hook@freebsd.org --- A commit references this bug: Author: rmacklem Date: Wed Jun 17 23:24:47 UTC 2015 New revision: 284531 URL: https://svnweb.freebsd.org/changeset/base/284531 Log: Document that a forced dismount of an NFSv3 mount when the NLM (rpc.lockd) is running can crash the system. Unfortunately this is not easy to fix, but I have left PR#200585 open. PR: 200585 MFC after: 3 days Changes: head/sbin/umount/umount.8 -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 17 23:55:54 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3E37CAAE for ; Wed, 17 Jun 2015 23:55:54 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: from mail-pa0-f48.google.com (mail-pa0-f48.google.com [209.85.220.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1578AE07 for ; Wed, 17 Jun 2015 23:55:53 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: by pacyx8 with SMTP id yx8so47417600pac.2 for ; Wed, 17 Jun 2015 16:55:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=tSdeCoK1GHDrKrvzXGgX5uTiEeC9OlogghfqOKqUL5c=; b=j/2MkQW4bj0uX7PIBdAp+LqNBO+q9Tkw4lgqOSK9w6B7F43tRCStPIRzOQyAoQfGHT XU6ZDmswlKL3xhbnvVb483qSXRzwuJxfei7WUA9jf3tjQtTWzpvW+/K9z1cVEjwtaJRV z5HCTVHDcyv+fy0lLSjVkKaL4WGlEEbaO5P3VrdiOFtrwEPYWKZ/3k2Ij0/fIUJnKjuZ fUAk0/yw4tuz7yuqN8gv1q6cLCBOSXFOctUv4udmX+wI6QWHXyYqLVVS7y2oIKUxwGbE UU5M5YCg65ZXxpx2xwfnRceXsXVwwy3/PphiCowlmB3j+lHel6u77MHAV4jDNYJfeaf1 JZfA== X-Gm-Message-State: ALoCoQnkXopluJ3FlV5cnWBhJUuRRPdLz0gP914+lEDLQlamIb6v8cIq4/EBua4FORIKfhQglcP6 X-Received: by 10.66.148.34 with SMTP id tp2mr15599794pab.65.1434585346570; Wed, 17 Jun 2015 16:55:46 -0700 (PDT) Received: from [192.168.1.138] (108-74-172-252.lightspeed.lsanca.sbcglobal.net. [108.74.172.252]) by mx.google.com with ESMTPSA id db1sm5912114pdb.50.2015.06.17.16.55.43 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 17 Jun 2015 16:55:45 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: TRIM erases user data From: steven@multiplay.co.uk X-Mailer: iPad Mail (12F69) In-Reply-To: Date: Wed, 17 Jun 2015 16:55:39 -0700 Cc: "freebsd-fs@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <7B981BB3-800B-4F59-B842-4D08ECDC5228@multiplay.co.uk> References: <49FEE0B5-D924-4FB6-889B-54F18667239B@multiplay.co.uk> To: Mark Martinec X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2015 23:55:54 -0000 We've used Samsung 840 Pro's on FreeBSD with ZFS for a long time (which has T= RIM enabled by default) so as far unless this has been broken on a FW or HW v= ersion we've not used then I have no reason to believe we're effected by thi= s issue at this time. Sent from my iPad On 17 Jun 2015, at 15:58, Mark Martinec wrote= : >>> On 16 Jun 2015, at 18:51, grarpamp wrote: >>> https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/ >>> https://github.com/torvalds/linux/blob/e64f638483a21105c7ce330d543fa1f1c= 35b5bc7/drivers/ata/libata-core.c#L4109-L4286 >>> http://www.aerospike.com/docs/operations/plan/ssd/ssd_certification.html= >=20 > steven@multiplay.co.uk wrote: >> This issue centers around queued TRIM requests at the ATA layer, which >> is an extension that allows NCQ support for TRIM in the SATA 3.1 spec. >> This is not something FreeBSD currently supports so is unaffected by >> the issue at this time. >=20 > Are you sure? The article explicitly states they were not using > queued TRIM: >=20 > | A lot of discussions started pointing out that the issue is related > | to the newly introduced queued TRIM. This is not correct. The TRIM > | on our drives is un-queued and the issue we have found is not related > | to the latest changes in the Linux Kernel to disable this features. >=20 >=20 > Mark > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Thu Jun 18 00:31:05 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2246711F for ; Thu, 18 Jun 2015 00:31:05 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: from mail-pd0-f180.google.com (mail-pd0-f180.google.com [209.85.192.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EB9BC82D for ; Thu, 18 Jun 2015 00:31:04 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: by pdbki1 with SMTP id ki1so52687264pdb.1 for ; Wed, 17 Jun 2015 17:31:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=RvQaJMPI6XCFxl98LhONjgR6LE5RVlRT3W2BjIpBlkg=; b=i1c3tg+ECKXsLYIS2ut0g4iKSPWvNTn/iNJmKcbtLa8bcpwFo6YYEO/LD86hSGVQP+ qpRFGIQ6cI9ChY6+h7J/OzVtz2FApP+Y3ercluHPw9BDZD9gYtIpSi3PgeJTQMUdvKLE wJTLJpGVbG7Y6W/BD//ys8XigQXPS2kIh577jZnNDcp6RV/isF0g7QJMyX+Ce5C/Jn+b Wj+OIWw+GfjjeKck558LqQ347DORRj8nuTctgIuDE7b6uB7Ml0wW/46M3F4Q9i93Tcsp 8EGVyM4LCEdFS4d1a1faeDoc9CovQmirJpPOmruSAfu89QflvAqjkuTyjE4LUnL8YtoO 0EvQ== X-Gm-Message-State: ALoCoQlB9CeStqYAU44bPLN+7MZCs9jQQlNGlqpKdELA5fVAl8lPk7Khck7bYm6X0rwJcmn9vHyl X-Received: by 10.68.241.164 with SMTP id wj4mr15625252pbc.37.1434587463435; Wed, 17 Jun 2015 17:31:03 -0700 (PDT) Received: from [192.168.1.138] (108-74-172-252.lightspeed.lsanca.sbcglobal.net. [108.74.172.252]) by mx.google.com with ESMTPSA id ho2sm5940971pbb.14.2015.06.17.17.31.01 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 17 Jun 2015 17:31:01 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: TRIM erases user data From: Steven Hartland X-Mailer: iPad Mail (12F69) In-Reply-To: Date: Wed, 17 Jun 2015 17:30:59 -0700 Cc: Mark Martinec , "freebsd-fs@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <99F249A6-F285-44D0-B003-5A7C96D20FD7@multiplay.co.uk> References: <49FEE0B5-D924-4FB6-889B-54F18667239B@multiplay.co.uk> <7B981BB3-800B-4F59-B842-4D08ECDC5228@multiplay.co.uk> To: Mark Saad X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2015 00:31:05 -0000 Just for the record illumos doesn't have any ZFS TRIM support. > On 17 Jun 2015, at 17:25, Mark Saad wrote: >=20 >=20 >=20 >> On Jun 17, 2015, at 7:55 PM, steven@multiplay.co.uk wrote: >>=20 >> We've used Samsung 840 Pro's on FreeBSD with ZFS for a long time (which h= as TRIM enabled by default) so as far unless this has been broken on a FW or= HW version we've not used then I have no reason to believe we're effected b= y this issue at this time. >=20 > I was using 840 pros for zfs on freebsd 9.3 , 10.1 for 2 years with no iss= ues . It was mostly for archival video storage and used as l2arc .=20 >=20 > I am working with omnios and smartos (illumos / opensolaris) almost exclu= sively with zpools on ssds of various vendors and I haven't see this issue o= r similar issues .=20 >=20 > I am still using 4 840ev on a 10.1 zfs box for the pool , and one for a l2= arc and it appears to be working as expected .=20 >=20 > For what my opinion is worth of this was some weird hardware bug , it woul= dn't just effect Ubuntu. I think Ubuntu or better said the culture of Ubuntu= might have a hand in this blog post . >=20 >>=20 >> Sent from my iPad >>=20 >> On 17 Jun 2015, at 15:58, Mark Martinec wr= ote: >>=20 >>>>> On 16 Jun 2015, at 18:51, grarpamp wrote: >>>>> https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/ >>>>> https://github.com/torvalds/linux/blob/e64f638483a21105c7ce330d543fa1f= 1c35b5bc7/drivers/ata/libata-core.c#L4109-L4286 >>>>> http://www.aerospike.com/docs/operations/plan/ssd/ssd_certification.ht= ml >>>=20 >>> steven@multiplay.co.uk wrote: >>>> This issue centers around queued TRIM requests at the ATA layer, which >>>> is an extension that allows NCQ support for TRIM in the SATA 3.1 spec. >>>> This is not something FreeBSD currently supports so is unaffected by >>>> the issue at this time. >>>=20 >>> Are you sure? The article explicitly states they were not using >>> queued TRIM: >>>=20 >>> | A lot of discussions started pointing out that the issue is related >>> | to the newly introduced queued TRIM. This is not correct. The TRIM >>> | on our drives is un-queued and the issue we have found is not related >>> | to the latest changes in the Linux Kernel to disable this features. >>>=20 >>>=20 >>> Mark >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >=20 >=20 > --- > Mark Saad | nonesuch@longcount.org From owner-freebsd-fs@FreeBSD.ORG Thu Jun 18 00:48:01 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7884C361 for ; Thu, 18 Jun 2015 00:48:01 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-qk0-f177.google.com (mail-qk0-f177.google.com [209.85.220.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 37F40BB6 for ; Thu, 18 Jun 2015 00:48:00 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by qkhu186 with SMTP id u186so36075522qkh.0 for ; Wed, 17 Jun 2015 17:47:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=juACtFw1bJ2DxQo+j5qED2YTyMtvv6UhI2PvqbkEe1Y=; b=UMp12ZNufttnO7+oQhSWwTlCPw6zFEKvraimN54rogzsfUGBs4kBGZK1VDdFyhJEax BeM3loOAPd2bi9/JyZ8GetGz17/mKLpkjKmeZ/FFi49UNr/3sSMIh39acHrPTi1EMDYt ZwydQ4Z8kREy0pi4BvR3iKk0z0Aq0wIg/rl8jeXAsp6NuMs14GVgJm1JgcLZB4tgB8Zw 370hEssewI9Xj+mJ5MIHHavi3c0Mv+dPlZYzuNNAw9h1zA6dtjU+F7q4+ioyhZ0YVkTB tPanP396fxwlnFDSO227t0RFejIRUxpn870qm94pOCP7RVxljanZnZR9jlcKK6ZoR4mt ElLw== X-Gm-Message-State: ALoCoQnzKPkfjDzycpWVE9tAkQLTzwHnkiF6RnLvg/7hFHP/968BkgQu+/gS1LG8AA/PjJ7rJmHa X-Received: by 10.55.15.99 with SMTP id z96mr18722595qkg.75.1434587127984; Wed, 17 Jun 2015 17:25:27 -0700 (PDT) Received: from [192.168.1.113] (ool-182c6ffa.dyn.optonline.net. [24.44.111.250]) by mx.google.com with ESMTPSA id m48sm3057789qgd.35.2015.06.17.17.25.27 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 17 Jun 2015 17:25:27 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: TRIM erases user data From: Mark Saad X-Mailer: iPhone Mail (12F70) In-Reply-To: <7B981BB3-800B-4F59-B842-4D08ECDC5228@multiplay.co.uk> Date: Wed, 17 Jun 2015 20:25:27 -0400 Cc: Mark Martinec , "freebsd-fs@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <49FEE0B5-D924-4FB6-889B-54F18667239B@multiplay.co.uk> <7B981BB3-800B-4F59-B842-4D08ECDC5228@multiplay.co.uk> To: "steven@multiplay.co.uk" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2015 00:48:01 -0000 > On Jun 17, 2015, at 7:55 PM, steven@multiplay.co.uk wrote: >=20 > We've used Samsung 840 Pro's on FreeBSD with ZFS for a long time (which ha= s TRIM enabled by default) so as far unless this has been broken on a FW or H= W version we've not used then I have no reason to believe we're effected by t= his issue at this time. I was using 840 pros for zfs on freebsd 9.3 , 10.1 for 2 years with no issue= s . It was mostly for archival video storage and used as l2arc .=20 I am working with omnios and smartos (illumos / opensolaris) almost exclusi= vely with zpools on ssds of various vendors and I haven't see this issue or s= imilar issues .=20 I am still using 4 840ev on a 10.1 zfs box for the pool , and one for a l2ar= c and it appears to be working as expected .=20 For what my opinion is worth of this was some weird hardware bug , it wouldn= 't just effect Ubuntu. I think Ubuntu or better said the culture of Ubuntu m= ight have a hand in this blog post . >=20 > Sent from my iPad >=20 > On 17 Jun 2015, at 15:58, Mark Martinec wro= te: >=20 >>>> On 16 Jun 2015, at 18:51, grarpamp wrote: >>>> https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/ >>>> https://github.com/torvalds/linux/blob/e64f638483a21105c7ce330d543fa1f1= c35b5bc7/drivers/ata/libata-core.c#L4109-L4286 >>>> http://www.aerospike.com/docs/operations/plan/ssd/ssd_certification.htm= l >>=20 >> steven@multiplay.co.uk wrote: >>> This issue centers around queued TRIM requests at the ATA layer, which >>> is an extension that allows NCQ support for TRIM in the SATA 3.1 spec. >>> This is not something FreeBSD currently supports so is unaffected by >>> the issue at this time. >>=20 >> Are you sure? The article explicitly states they were not using >> queued TRIM: >>=20 >> | A lot of discussions started pointing out that the issue is related >> | to the newly introduced queued TRIM. This is not correct. The TRIM >> | on our drives is un-queued and the issue we have found is not related >> | to the latest changes in the Linux Kernel to disable this features. >>=20 >>=20 >> Mark >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --- Mark Saad | nonesuch@longcount.org= From owner-freebsd-fs@FreeBSD.ORG Thu Jun 18 01:38:43 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D04DC3B for ; Thu, 18 Jun 2015 01:38:43 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: from mail-yk0-f169.google.com (mail-yk0-f169.google.com [209.85.160.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3406935 for ; Thu, 18 Jun 2015 01:38:41 +0000 (UTC) (envelope-from nonesuch@longcount.org) Received: by ykfr66 with SMTP id r66so55037097ykf.0 for ; Wed, 17 Jun 2015 18:38:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=MVMqnRC/ST/F0Uy8BzVJ0K+/cpJQt4L6AM9kUHDjByo=; b=VZlDqmH7Nt43YAxHrUyb/rAnlGSvfcMG/lSCYsGhDGA8HmeK4TFhFVvTJMi3zVxOB4 K9Wusv+ih7jVMUGo0WX4+CEOo9KUXSiqCp7exFqyzD8T0fwvK9dD3ALppIXLmqSIewVK uWtCOS45f4Ky4ZlSIi+bh3KD9+AW8szgQb+f9wj2tzaBoXjq+hLAJyQMss+9TntXT0b1 2TP0Eg+x0Wf9DbgbiXWDDT2n9203/aJ62iEN/k0nsl9d51e8C9T5wFOE03z11CDYnybd mpO/lZOGmHEbZS9HympUA/suF5DnjMmW2LRJoUrqCn9VDcYw5Fbl22S7Do7OamkeotJq crmg== X-Gm-Message-State: ALoCoQm8WL3KpPK9WgSTxe5LM4Txh4Rmtwr5huiVkagaG2FD2edPQmE+sjSbR171c7bVfaBPseA5 MIME-Version: 1.0 X-Received: by 10.129.32.4 with SMTP id g4mr10916992ywg.94.1434591514929; Wed, 17 Jun 2015 18:38:34 -0700 (PDT) Received: by 10.13.208.6 with HTTP; Wed, 17 Jun 2015 18:38:34 -0700 (PDT) X-Originating-IP: [24.44.111.250] In-Reply-To: <99F249A6-F285-44D0-B003-5A7C96D20FD7@multiplay.co.uk> References: <49FEE0B5-D924-4FB6-889B-54F18667239B@multiplay.co.uk> <7B981BB3-800B-4F59-B842-4D08ECDC5228@multiplay.co.uk> <99F249A6-F285-44D0-B003-5A7C96D20FD7@multiplay.co.uk> Date: Wed, 17 Jun 2015 21:38:34 -0400 Message-ID: Subject: Re: TRIM erases user data From: Mark Saad To: Steven Hartland Cc: Mark Martinec , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2015 01:38:43 -0000 On Wed, Jun 17, 2015 at 8:30 PM, Steven Hartland wrote: > Just for the record illumos doesn't have any ZFS TRIM support. > > There are lots of things in illumos that applys to :(. In any case I wasn't aware that illumos zfs was sans-trim . > > On 17 Jun 2015, at 17:25, Mark Saad wrote: > > > > > > > >> On Jun 17, 2015, at 7:55 PM, steven@multiplay.co.uk wrote: > >> > >> We've used Samsung 840 Pro's on FreeBSD with ZFS for a long time (which > has TRIM enabled by default) so as far unless this has been broken on a FW > or HW version we've not used then I have no reason to believe we're > effected by this issue at this time. > > > > I was using 840 pros for zfs on freebsd 9.3 , 10.1 for 2 years with no > issues . It was mostly for archival video storage and used as l2arc . > > > > I am working with omnios and smartos (illumos / opensolaris) almost > exclusively with zpools on ssds of various vendors and I haven't see this > issue or similar issues . > > > > I am still using 4 840ev on a 10.1 zfs box for the pool , and one for a > l2arc and it appears to be working as expected . > > > > For what my opinion is worth of this was some weird hardware bug , it > wouldn't just effect Ubuntu. I think Ubuntu or better said the culture of > Ubuntu might have a hand in this blog post . > > > >> > >> Sent from my iPad > >> > >> On 17 Jun 2015, at 15:58, Mark Martinec > wrote: > >> > >>>>> On 16 Jun 2015, at 18:51, grarpamp wrote: > >>>>> https://blog.algolia.com/when-solid-state-drives-are-not-that-solid/ > >>>>> > https://github.com/torvalds/linux/blob/e64f638483a21105c7ce330d543fa1f1c35b5bc7/drivers/ata/libata-core.c#L4109-L4286 > >>>>> > http://www.aerospike.com/docs/operations/plan/ssd/ssd_certification.html > >>> > >>> steven@multiplay.co.uk wrote: > >>>> This issue centers around queued TRIM requests at the ATA layer, which > >>>> is an extension that allows NCQ support for TRIM in the SATA 3.1 spec. > >>>> This is not something FreeBSD currently supports so is unaffected by > >>>> the issue at this time. > >>> > >>> Are you sure? The article explicitly states they were not using > >>> queued TRIM: > >>> > >>> | A lot of discussions started pointing out that the issue is related > >>> | to the newly introduced queued TRIM. This is not correct. The TRIM > >>> | on our drives is un-queued and the issue we have found is not related > >>> | to the latest changes in the Linux Kernel to disable this features. > >>> > >>> > >>> Mark > >>> _______________________________________________ > >>> freebsd-fs@freebsd.org mailing list > >>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs > >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >> _______________________________________________ > >> freebsd-fs@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > > > > --- > > Mark Saad | nonesuch@longcount.org > -- mark saad | nonesuch@longcount.org From owner-freebsd-fs@FreeBSD.ORG Thu Jun 18 10:00:11 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2B2D16CD for ; Thu, 18 Jun 2015 10:00:11 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E1443D2F for ; Thu, 18 Jun 2015 10:00:08 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 1DB5716A404 for ; Thu, 18 Jun 2015 11:54:16 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EI4dtVNj8Y8p; Thu, 18 Jun 2015 11:53:57 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:301d:d194:f8e3:4290] (unknown [IPv6:2001:4cb8:3:1:301d:d194:f8e3:4290]) by smtp.digiware.nl (Postfix) with ESMTP id 4315816A403 for ; Thu, 18 Jun 2015 11:53:57 +0200 (CEST) Message-ID: <5582952E.6020202@digiware.nl> Date: Thu, 18 Jun 2015 11:53:50 +0200 From: Willem Jan Withagen Organization: Digiware Management b.v. User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Layout of zpool list -v Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2015 10:00:11 -0000 Hi, Call me picky, but then it confuses me every time. I have this mirrored set with both cache and log. Looking at it with zpool it looks like: zdata 360G 50.4G 310G - 9% 14% ........ mirror 360G 50.4G 310G - 9% 14% gpt/data0 - - - - - - gpt/data1 - - - - - - mirror 496M 800K 495M - 15% 0% gpt/log0 - - - - - - gpt/log1 - - - - - - cache - - - - - - gpt/cachedata0 31.7G 118M 31.6G - 0% 0% gpt/cachedata1 31.7G 128M 31.6G - 0% 0% Which (could) suggest that there are 2 mirrors in a raid0 config, whereas it actually is a log/mirror attached. Would it not be more informative if it looks like: (if at all possible) zdata 360G 50.4G 310G - 9% 14% ........ mirror 360G 50.4G 310G - 9% 14% gpt/data0 - - - - - - gpt/data1 - - - - - - log - - - - - - mirror 496M 800K 495M - 15% 0% gpt/log0 - - - - - - gpt/log1 - - - - - - cache - - - - - - gpt/cachedata0 31.7G 118M 31.6G - 0% 0% gpt/cachedata1 31.7G 128M 31.6G - 0% 0% Note that I've also indented the cache set. I was going to add a patch, since it does not seem too complex, but I'm really lost in the zfs/zpool code to get anywhere near the code that actually does the work... Single stepping it in gdb is sort of a pain since zpool is threaded?? If somebody could give me some pointers I'll have another go at it. Thanx, --WjW From owner-freebsd-fs@FreeBSD.ORG Thu Jun 18 12:58:46 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8083B453; Thu, 18 Jun 2015 12:58:46 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5599F316; Thu, 18 Jun 2015 12:58:46 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=fISGMPfAhpfVVT6jc6psmik4Y5OIYhvLncmz/S5Lfuk=; b=AWG6fe3i3ev5KIEdv/fpAMZs3vmFifDjiPOTe+Hedy49+KI4rywPnGKdFJum7nVIwq0XSlUpVw/j9+4RgqEgCUE57vDEBM/3J7gov5MSWwJMz+0mtQoSsIVU03teyMvPTOsOeyNaw9VJ6qXyRedXGUBRpFe+kv12N9QGQWHCg2c=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:22724 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1Z5ZP3-0002da-EY; Thu, 18 Jun 2015 07:58:45 -0500 Received: from 104-54-221-134.lightspeed.austtx.sbcglobal.net ([104.54.221.134]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 18 Jun 2015 07:58:45 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 18 Jun 2015 07:58:45 -0500 From: Larry Rosenman To: Freebsd fs , rmacklem@freebsd.org Subject: Re: NFS Mount and LARGE amounts of "INACT" memory In-Reply-To: <3a845167bb6123139c4585659e8f0d43@thebighonker.lerctr.org> References: <3a845167bb6123139c4585659e8f0d43@thebighonker.lerctr.org> Message-ID: <0842fc47d1f138870671b1ed201e686b@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.1.1 X-Spam-Score: -1.0 (-) X-LERCTR-Spam-Score: -1.0 (-) X-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1,SHORTCIRCUIT=-0.0001 X-LERCTR-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1, SHORTCIRCUIT=-0.0001 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2015 12:58:46 -0000 On 2015-06-17 07:26, Larry Rosenman wrote: > I have a 64G memory FreeBSD 11-CURRENT system that has a couple of > mounts to a FreeNAS (FreeBSD 9.3) system. > > When my rsync from a different system to one of the NFS mounts runs, I > get like 48G of Inactive memory that goes back to > free if I umount the share. > > I'm wondering why this memory moves from ZFS ARC to INACT. > > And, is this expected? I've posted screenshots at: http://www.lerctr.org/~ler/FreeBSD_inact/ -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Thu Jun 18 18:55:03 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E9AAD130 for ; Thu, 18 Jun 2015 18:55:03 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "raven.bwct.de", Issuer "BWCT" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 80650D00 for ; Thu, 18 Jun 2015 18:55:02 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id t5IIsbCn056512 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Thu, 18 Jun 2015 20:54:42 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.5/8.14.4) with ESMTP id t5IIsVU5014150 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 18 Jun 2015 20:54:31 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id t5IIsVTD029470; Thu, 18 Jun 2015 20:54:31 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id t5IIsVdq029469; Thu, 18 Jun 2015 20:54:31 +0200 (CEST) (envelope-from ticso) Date: Thu, 18 Jun 2015 20:54:31 +0200 From: Bernd Walter To: freebsd-fs@freebsd.org Cc: Bernd Walter Subject: reads on inactive zpools Message-ID: <20150618185430.GA27522@cicely7.cicely.de> Reply-To: ticso@cicely.de Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 autolearn=ham version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2015 18:55:04 -0000 I have multiple zpools with USB drives. Under normal conditions everything is idle as it should. But when I zfs send | zfs receive from one pool to another the unrelated pools have one read/s on each disk. Once the zfs send/receive is stopped the reads to the unrelated pools also stop. Size of the unexpected reads varying below 8k, usually 7.xk, but I've also seen 4.xk. The read sizes on all disks of all pools are identic. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-fs@FreeBSD.ORG Thu Jun 18 20:33:25 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BD098692 for ; Thu, 18 Jun 2015 20:33:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A693E98C for ; Thu, 18 Jun 2015 20:33:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t5IKXPhN092880 for ; Thu, 18 Jun 2015 20:33:25 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 197789] (zfs+i386 No PAE) panic: kmem_malloc(36864): kmem_map too small: 431976448 total allocated Date: Thu, 18 Jun 2015 20:33:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 9.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: michelle@sorbs.net X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2015 20:33:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197789 --- Comment #9 from Michelle Sullivan --- I think I might be one step closer to the cause. I've noticed when using memory based disk on both i386 and amd64 the memory used (some/all?) cases doesn't appear in in the memory stats shown in 'top' ... however the memory is 'missing' .. could this be fooling the VM manager into thinking there is memory free when there is not and therefore screwing up the memory pressure handler...? -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri Jun 19 23:58:23 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AC1C5957; Fri, 19 Jun 2015 23:58:23 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 5D962A6B; Fri, 19 Jun 2015 23:58:22 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DJBACmq4RV/95baINZAxaDTl8Ggxi8SQqFLkoCgXQQAQEBAQEBAYEKhCIBAQEDAQEBASArIAsFCQ0OCgICDRkCHQwBCSYGCAcEAQgUBIgGCA2vcJY4AQEBAQEBAQEBAQEBAQEBAQEBAQEBFwSBHYokhC0HAQEFFyQQBxGCHDsSgTEFk3yEVoQvhESPJYcOJmODMiIxB3wJFyOBAgEBAQ X-IronPort-AV: E=Sophos;i="5.13,646,1427774400"; d="scan'208";a="219184483" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 19 Jun 2015 19:58:15 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 8FE63B3FD4; Fri, 19 Jun 2015 19:58:15 -0400 (EDT) Date: Fri, 19 Jun 2015 19:58:15 -0400 (EDT) From: Rick Macklem To: Larry Rosenman Cc: Freebsd fs , rmacklem@freebsd.org Message-ID: <228350188.61172889.1434758295576.JavaMail.root@uoguelph.ca> In-Reply-To: <0842fc47d1f138870671b1ed201e686b@thebighonker.lerctr.org> Subject: Re: NFS Mount and LARGE amounts of "INACT" memory MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jun 2015 23:58:23 -0000 Larry Rosenman wrote: > On 2015-06-17 07:26, Larry Rosenman wrote: > > I have a 64G memory FreeBSD 11-CURRENT system that has a couple of > > mounts to a FreeNAS (FreeBSD 9.3) system. > > > > When my rsync from a different system to one of the NFS mounts > > runs, I > > get like 48G of Inactive memory that goes back to > > free if I umount the share. > > > > I'm wondering why this memory moves from ZFS ARC to INACT. > > > > And, is this expected? A wild ass guess would be yes. Assuming you are referring to the NFS client (and not FreeNAS server) and guessing that rsync uses mmap'd I/O... - The pages will be associated with the file's vnode until that vnode is recycled. (mmap'd I/O can continue after the file is closed.) This could take a long time. I am not knowledgible w.r.t. the VM subsystem, but I'm guessing that there is some way for these pages to be reused if memory is limited? (Hopefully someone with VM knowledge can comment on this?) rick > I've posted screenshots at: > > http://www.lerctr.org/~ler/FreeBSD_inact/ > > > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 00:00:58 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4DEFA06; Sat, 20 Jun 2015 00:00:58 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F552BCD; Sat, 20 Jun 2015 00:00:58 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=6YKQja9e0K1QOWqNgo6F1dR/VB6xolrISLY54+Nyxdg=; b=Lld2Nbb5coVzJztk8NBbfG88qMMeXozERdDl23pDfrXC/NO5j815z3NTbNgbsF1N7Z3e23tlxktUboK4guMilPTzrIdjOz3MOLvH5TWCMBlaZ6oKateJO8q79WW5JNGz+Du9JxJqNzyj+WpOIe5EyU2KQGhS19n60PpzYCGyqGY=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:18339 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1Z66DQ-000PQw-7C; Fri, 19 Jun 2015 19:00:56 -0500 Received: from 104-54-221-134.lightspeed.austtx.sbcglobal.net ([104.54.221.134]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Fri, 19 Jun 2015 19:00:56 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 19 Jun 2015 19:00:56 -0500 From: Larry Rosenman To: Rick Macklem Cc: Freebsd fs , rmacklem@freebsd.org Subject: Re: NFS Mount and LARGE amounts of "INACT" memory In-Reply-To: <228350188.61172889.1434758295576.JavaMail.root@uoguelph.ca> References: <228350188.61172889.1434758295576.JavaMail.root@uoguelph.ca> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.1.1 X-Spam-Score: -1.0 (-) X-LERCTR-Spam-Score: -1.0 (-) X-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1,SHORTCIRCUIT=-0.0001 X-LERCTR-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1, SHORTCIRCUIT=-0.0001 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 00:00:58 -0000 On 2015-06-19 18:58, Rick Macklem wrote: > Larry Rosenman wrote: >> On 2015-06-17 07:26, Larry Rosenman wrote: >> > I have a 64G memory FreeBSD 11-CURRENT system that has a couple of >> > mounts to a FreeNAS (FreeBSD 9.3) system. >> > >> > When my rsync from a different system to one of the NFS mounts >> > runs, I >> > get like 48G of Inactive memory that goes back to >> > free if I umount the share. >> > >> > I'm wondering why this memory moves from ZFS ARC to INACT. >> > >> > And, is this expected? > A wild ass guess would be yes. Assuming you are referring to the NFS > client (and not FreeNAS server) and guessing that rsync uses mmap'd > I/O... > - The pages will be associated with the file's vnode until that vnode > is recycled. (mmap'd I/O can continue after the file is closed.) > This could take a long time. > I am not knowledgible w.r.t. the VM subsystem, but I'm guessing that > there is some way for these pages to be reused if memory is limited? > (Hopefully someone with VM knowledge can comment on this?) > Yes, this is the NFS Client, not sure on mmap(2), but that would make sense BUT, I don't like that it kills my ZFS ARC.... VM Guys? > rick > >> I've posted screenshots at: >> >> http://www.lerctr.org/~ler/FreeBSD_inact/ >> >> >> -- >> Larry Rosenman http://www.lerctr.org/~ler >> Phone: +1 214-642-9640 E-Mail: ler@lerctr.org >> US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 00:30:54 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6D35957; Sat, 20 Jun 2015 00:30:53 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BBA03212; Sat, 20 Jun 2015 00:30:53 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=PwjIujR1Dm2xA/MVk2O72mfbIDvIB8CoOIt83H4+dCg=; b=q3qXIyUsgePu9i9ZtRUC3YuS1THwSd8IVOqhnrRcoTCGk+kpSEUkw7BmRAASVY7EaH/DXTOTUKGkJiWpGeUyS0f5vpPA1RR31or6ikaJvlrzz5hY4BCHEI67NAc10qYaPZjWu0OLLI6AlHnivXwQUhAtTIJeCCh01EDGOfwTU3A=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:45951 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES128-SHA:128) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1Z66gO-000Pkc-M5; Fri, 19 Jun 2015 19:30:52 -0500 Received: from 104-54-221-134.lightspeed.austtx.sbcglobal.net ([104.54.221.134]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Fri, 19 Jun 2015 19:30:52 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 19 Jun 2015 19:30:52 -0500 From: Larry Rosenman To: Rick Macklem Cc: Freebsd fs , rmacklem@freebsd.org Subject: Re: NFS Mount and LARGE amounts of "INACT" memory In-Reply-To: References: <228350188.61172889.1434758295576.JavaMail.root@uoguelph.ca> Message-ID: <7f8b3449973cff790d996bb1f169b8e0@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.1.1 X-Spam-Score: -1.0 (-) X-LERCTR-Spam-Score: -1.0 (-) X-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1,SHORTCIRCUIT=-0.0001 X-LERCTR-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1, SHORTCIRCUIT=-0.0001 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 00:30:54 -0000 On 2015-06-19 19:00, Larry Rosenman wrote: > On 2015-06-19 18:58, Rick Macklem wrote: >> Larry Rosenman wrote: >>> On 2015-06-17 07:26, Larry Rosenman wrote: >>> > I have a 64G memory FreeBSD 11-CURRENT system that has a couple of >>> > mounts to a FreeNAS (FreeBSD 9.3) system. >>> > >>> > When my rsync from a different system to one of the NFS mounts >>> > runs, I >>> > get like 48G of Inactive memory that goes back to >>> > free if I umount the share. >>> > >>> > I'm wondering why this memory moves from ZFS ARC to INACT. >>> > >>> > And, is this expected? >> A wild ass guess would be yes. Assuming you are referring to the NFS >> client (and not FreeNAS server) and guessing that rsync uses mmap'd >> I/O... >> - The pages will be associated with the file's vnode until that vnode >> is recycled. (mmap'd I/O can continue after the file is closed.) >> This could take a long time. >> I am not knowledgible w.r.t. the VM subsystem, but I'm guessing that >> there is some way for these pages to be reused if memory is limited? >> (Hopefully someone with VM knowledge can comment on this?) >> > Yes, this is the NFS Client, not sure on mmap(2), but that would make > sense > > BUT, I don't like that it kills my ZFS ARC.... > > VM Guys? > BTW, a quick grep if the rsync sources shows it does NOT use mmap, but has some mmap-like routines, so I'm at a loss.... >> rick >> >>> I've posted screenshots at: >>> >>> http://www.lerctr.org/~ler/FreeBSD_inact/ >>> >>> >>> -- >>> Larry Rosenman http://www.lerctr.org/~ler >>> Phone: +1 214-642-9640 E-Mail: ler@lerctr.org >>> US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>> -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 01:24:26 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C53455D for ; Sat, 20 Jun 2015 01:24:26 +0000 (UTC) (envelope-from quartz@sneakertech.com) Received: from douhisi.pair.com (unknown [IPv6:2607:f440::d144:5b3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1CC16BF for ; Sat, 20 Jun 2015 01:24:26 +0000 (UTC) (envelope-from quartz@sneakertech.com) Received: from [10.2.2.1] (pool-173-48-121-235.bstnma.fios.verizon.net [173.48.121.235]) by douhisi.pair.com (Postfix) with ESMTPSA id 25F083F71D for ; Fri, 19 Jun 2015 21:24:17 -0400 (EDT) Message-ID: <5584C0BC.9070707@sneakertech.com> Date: Fri, 19 Jun 2015 21:24:12 -0400 From: Quartz User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2 MIME-Version: 1.0 To: Freebsd fs Subject: ZFS pool restructuring and emergency repair Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 01:24:26 -0000 I'm wondering if anyone can help me clear up a few questions and concerns I have about ZFS. It seems to me that ZFS is really not terribly flexible when it comes to changing a pool's structure after the fact, and once you set something up you're pretty much stuck with it, making future administration and repairs complicated. To be fair, I'm not really clear on what all the available tools can do and what the options are- I haven't really been keeping up with ZFS development over the past few years so I'm not sure how much of my knowledge is out of date. What are people's responses and recommendations given the following hypothetical situations: - A server is set up with a pool created a certain way. A couple years later it's determined that the pool configuration wasn't a good choice for the workload and it should be redone. As I understand it, ZFS has no capability to reorganize, remove, or re-type vdevs, so your only option is completely starting over with another whole pool. Is this still true? If so, is there a correct way to copy an entire pool to another set of disks in a way that preserves all the metadata and hierarchical dataset information? (snapshots, noatime, compression, dedupe, quotas, mountpoints, etc). It looks like 'send' and 'receive' might do it, but I'm having trouble finding detailed information on exactly what they copy, how much of a skeleton on the receiving end I need to manually create first, and what breaks if we have a root-on-ZFS setup. - A server is set up with a pool created a certain way, for the sake of argument let's say it's a raidz-2 comprised of 6x 2TB disks. There's only actually ~1TB of data currently on the server though. Let's say there's a catastrophic emergency where one of the disks needs to be replaced, but the only available spare is an old 500GB. As I understand it, you're basically SOL. Even though a 6x500 (really 4x500) is more than enough to hold 1Tb of data, you can't do anything in this situation since although ZFS can expand a pool to fit larger disks, it can't shrink one under any circumstance. Is my understanding still correct or is there a way around this issue now? From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 02:27:11 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6712E95C for ; Sat, 20 Jun 2015 02:27:11 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1DF39FA9 for ; Sat, 20 Jun 2015 02:27:10 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id t5K2KNNn008606; Fri, 19 Jun 2015 21:20:23 -0500 (CDT) Date: Fri, 19 Jun 2015 21:20:23 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Quartz cc: Freebsd fs Subject: Re: ZFS pool restructuring and emergency repair In-Reply-To: <5584C0BC.9070707@sneakertech.com> Message-ID: References: <5584C0BC.9070707@sneakertech.com> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Fri, 19 Jun 2015 21:20:24 -0500 (CDT) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 02:27:11 -0000 On Fri, 19 Jun 2015, Quartz wrote: > > What are people's responses and recommendations given the following > hypothetical situations: > > - A server is set up with a pool created a certain way. A couple years later > it's determined that the pool configuration wasn't a good choice for the > workload and it should be redone. As I understand it, ZFS has no capability > to reorganize, remove, or re-type vdevs, so your only option is completely > starting over with another whole pool. Is this still true? If so, is there a > correct way to copy an entire pool to another set of disks in a way that > preserves all the metadata and hierarchical dataset information? (snapshots, > noatime, compression, dedupe, quotas, mountpoints, etc). It looks like 'send' > and 'receive' might do it, but I'm having trouble finding detailed > information on exactly what they copy, how much of a skeleton on the > receiving end I need to manually create first, and what breaks if we have a > root-on-ZFS setup. You can use 'send' and 'receive' to send all the data and the metadata associated with that data. I believe that you are correct that filesystem properties (like 'compression') are not preserved. Snapshots can be sent and preserved and indeed send/receive is based on snapshots. You can recover filesystem properties at the source via 'zpool history' but then need to filter out only what is needed and do any necessary edits to produce a script for the receiving end. I have a system which sends several filesystems to another system every night via send/receive without any problems, and with different properties (e.g. read-only) set on the receive filesystem. I also have nightly send from one pool to another on the same system (all for redundant backups). It is very easy to expand/enlarge a pool (given the physical ability to do so) but the only way to restructure the existing parts of a pool is to create a new one. > - A server is set up with a pool created a certain way, for the sake of > argument let's say it's a raidz-2 comprised of 6x 2TB disks. There's only > actually ~1TB of data currently on the server though. Let's say there's a > catastrophic emergency where one of the disks needs to be replaced, but the > only available spare is an old 500GB. As I understand it, you're basically > SOL. Even though a 6x500 (really 4x500) is more than enough to hold 1Tb of > data, you can't do anything in this situation since although ZFS can expand a > pool to fit larger disks, it can't shrink one under any circumstance. Is my > understanding still correct or is there a way around this issue now? You will still need to supply a replacement disk at least as large as the disk being replaced. If the disks can be assured to be 100% reliable and the pool is not very full, then there are tricky ways to create a new pool on the same disks that the old pool occupied. I am not brave enough to do this but others have accomplished it successfully. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 05:21:10 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 47D3F305 for ; Sat, 20 Jun 2015 05:21:10 +0000 (UTC) (envelope-from mcdouga9@egr.msu.edu) Received: from mail.egr.msu.edu (boomhauer.egr.msu.edu [35.9.37.167]) by mx1.freebsd.org (Postfix) with ESMTP id 20FE2EF6 for ; Sat, 20 Jun 2015 05:21:09 +0000 (UTC) (envelope-from mcdouga9@egr.msu.edu) Received: from boomhauer (localhost [127.0.0.1]) by mail.egr.msu.edu (Postfix) with ESMTP id 24E2729661 for ; Sat, 20 Jun 2015 01:21:02 -0400 (EDT) X-Virus-Scanned: amavisd-new at egr.msu.edu Received: from mail.egr.msu.edu ([127.0.0.1]) by boomhauer (boomhauer.egr.msu.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lf-bmuzz5NbR for ; Sat, 20 Jun 2015 01:21:01 -0400 (EDT) Received: from EGR authenticated sender mcdouga9 Message-ID: <5584F83D.1040702@egr.msu.edu> Date: Sat, 20 Jun 2015 01:21:01 -0400 From: Adam McDougall User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: ZFS pool restructuring and emergency repair References: <5584C0BC.9070707@sneakertech.com> In-Reply-To: <5584C0BC.9070707@sneakertech.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 05:21:10 -0000 On 06/19/2015 21:24, Quartz wrote: > I'm wondering if anyone can help me clear up a few questions and > concerns I have about ZFS. > > It seems to me that ZFS is really not terribly flexible when it comes to > changing a pool's structure after the fact, and once you set something > up you're pretty much stuck with it, making future administration and > repairs complicated. To be fair, I'm not really clear on what all the > available tools can do and what the options are- I haven't really been > keeping up with ZFS development over the past few years so I'm not sure > how much of my knowledge is out of date. > > What are people's responses and recommendations given the following > hypothetical situations: > > - A server is set up with a pool created a certain way. A couple years > later it's determined that the pool configuration wasn't a good choice > for the workload and it should be redone. As I understand it, ZFS has no > capability to reorganize, remove, or re-type vdevs, so your only option > is completely starting over with another whole pool. Is this still true? > If so, is there a correct way to copy an entire pool to another set of > disks in a way that preserves all the metadata and hierarchical dataset > information? (snapshots, noatime, compression, dedupe, quotas, > mountpoints, etc). It looks like 'send' and 'receive' might do it, but > I'm having trouble finding detailed information on exactly what they > copy, how much of a skeleton on the receiving end I need to manually > create first, and what breaks if we have a root-on-ZFS setup. The manpage for zfs says: (under zfs send) -R Generate a replication stream package, which will replicate the specified filesystem, and all descendent file systems, up to the named snapshot. When received, all properties, snap‐ shots, descendent file systems, and clones are preserved. > > > - A server is set up with a pool created a certain way, for the sake of > argument let's say it's a raidz-2 comprised of 6x 2TB disks. There's > only actually ~1TB of data currently on the server though. Let's say > there's a catastrophic emergency where one of the disks needs to be > replaced, but the only available spare is an old 500GB. As I understand > it, you're basically SOL. Even though a 6x500 (really 4x500) is more > than enough to hold 1Tb of data, you can't do anything in this situation > since although ZFS can expand a pool to fit larger disks, it can't > shrink one under any circumstance. Is my understanding still correct or > is there a way around this issue now? man gvirstor which lets you create an arbitrarily large storage device backed by chunks of storage based on how much you are actually using. I have not used it. From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 06:18:31 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D97AB7F for ; Sat, 20 Jun 2015 06:18:31 +0000 (UTC) (envelope-from alex.burlyga.ietf@gmail.com) Received: from mail-yk0-x22c.google.com (mail-yk0-x22c.google.com [IPv6:2607:f8b0:4002:c07::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1864DCE2 for ; Sat, 20 Jun 2015 06:18:31 +0000 (UTC) (envelope-from alex.burlyga.ietf@gmail.com) Received: by ykdr198 with SMTP id r198so104585739ykd.3 for ; Fri, 19 Jun 2015 23:18:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=UaUa6gy4GHoikgiEtdFPz+dcTJweksX5dDiYSSZuB8Q=; b=cEDzEzlZqj3ghYRUoCG8Yryv7AcVzrej15AKzNLtMsWNKLEc1PYIaWnUcSfM4qQTyW KAFUWwHkElwa84/uWAgY0lPIHQiZjlpj0hNVE1pjVVraT7jHvB4SLfGiLWjEn5IfjaZV KG3VuCYV+5+vfEYI+yX1NWczLOwyjhgaXg2yXwKOi/BEXaZdJRg1npEFie5XeCvalbUU IJfkU0haU9v0aNoGBlH0kVkR0JD31kYKAtz6Ng5qRHadvhO+wxot/2fDQeEf8Tdx5+CO 6XN6iMoPY8SwJZ400rcFqQJuzPluEYSusKYy+uiPCRcgR2NMOeMcaGzAEY9gZSjgQPtA lBfw== MIME-Version: 1.0 X-Received: by 10.129.108.12 with SMTP id h12mr24410390ywc.161.1434781110065; Fri, 19 Jun 2015 23:18:30 -0700 (PDT) Received: by 10.13.244.65 with HTTP; Fri, 19 Jun 2015 23:18:29 -0700 (PDT) Received: by 10.13.244.65 with HTTP; Fri, 19 Jun 2015 23:18:29 -0700 (PDT) In-Reply-To: References: Date: Fri, 19 Jun 2015 23:18:29 -0700 Message-ID: Subject: [nfs][client] - Question about handling of the NFS3_EEXIST error in SYMLINK rpc From: "alex.burlyga.ietf alex.burlyga.ietf" To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 06:18:31 -0000 Hi, NFS client code in nfsrpc_symlink() masks server returned NFS3_EEXIST error code by returning 0 to the upper layers. I'm assuming this was an attempt to work around some server's broken replay cache out there, however, it breaks a more common case where server is returning EEXIST for legitimate reason and application is expecting this error code and equipped to deal with it. To fix it I see three ways of doing this: * Remove offending code * Make it optional, sysctl? * On NFS3_EEXIST send READLINK rpc to make sure symlink content is right Which of the ways will maximize the chances of getting this fix upstream? One more point, old client circa FreeBSD 7.0 does not exhibit this problem. Alex From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 10:11:03 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 913C52A1 for ; Sat, 20 Jun 2015 10:11:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B4E79D9 for ; Sat, 20 Jun 2015 10:11:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id t5KAB3tM071378 for ; Sat, 20 Jun 2015 10:11:03 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 198242] [zfs] L2ARC degraded. Checksum errors, I/O errors Date: Sat, 20 Jun 2015 10:11:00 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: gkontos@aicom.gr X-Bugzilla-Status: New X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 10:11:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=198242 --- Comment #12 from gkontos@aicom.gr --- Looks ok here after 2 weeks of testing. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 14:50:58 2015 Return-Path: Delivered-To: fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 307D2976 for ; Sat, 20 Jun 2015 14:50:58 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [31.223.170.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E67F3F4 for ; Sat, 20 Jun 2015 14:50:57 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from rack1.digiware.nl (unknown [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 9CB0D16A407 for ; Sat, 20 Jun 2015 16:50:49 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.nl Received: from smtp.digiware.nl ([127.0.0.1]) by rack1.digiware.nl (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IBGSGUpD3BOH; Sat, 20 Jun 2015 16:50:23 +0200 (CEST) Received: from [IPv6:2001:4cb8:3:1:8a1:f529:c01f:472d] (unknown [IPv6:2001:4cb8:3:1:8a1:f529:c01f:472d]) by smtp.digiware.nl (Postfix) with ESMTPA id E5C8916A401 for ; Sat, 20 Jun 2015 16:19:38 +0200 (CEST) Message-ID: <5585767B.4000206@digiware.nl> Date: Sat, 20 Jun 2015 16:19:39 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: fs@freebsd.org Subject: This diskfailure should not panic a system, but just disconnect disk from ZFS Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 14:50:58 -0000 Hi, Found my system rebooted this morning: Jun 20 05:28:33 zfs kernel: sonewconn: pcb 0xfffff8011b6da498: Listen queue overflow: 8 already in queue awaiting acceptance (48 occurrences) Jun 20 05:28:33 zfs kernel: panic: I/O to pool 'zfsraid' appears to be hung on vdev guid 18180224580327100979 at '/dev/da0'. Jun 20 05:28:33 zfs kernel: cpuid = 0 Jun 20 05:28:33 zfs kernel: Uptime: 8d9h7m9s Jun 20 05:28:33 zfs kernel: Dumping 6445 out of 8174 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Which leads me to believe that /dev/da0 went out on vacation, leaving ZFS into trouble.... But the array is: ---- NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP zfsraid 32.5T 13.3T 19.2T - 7% 41% 1.00x ONLINE - raidz2 16.2T 6.67T 9.58T - 8% 41% da0 - - - - - - da1 - - - - - - da2 - - - - - - da3 - - - - - - da4 - - - - - - da5 - - - - - - raidz2 16.2T 6.67T 9.58T - 7% 41% da6 - - - - - - da7 - - - - - - ada4 - - - - - - ada5 - - - - - - ada6 - - - - - - ada7 - - - - - - mirror 504M 1.73M 502M - 39% 0% gpt/log0 - - - - - - gpt/log1 - - - - - - cache - - - - - - gpt/raidcache0 109G 1.34G 107G - 0% 1% gpt/raidcache1 109G 787M 108G - 0% 0% ---- And thus I'd would have expected that ZFS would disconnect /dev/da0 and then switch to DEGRADED state and continue, letting the operator fix the broken disk. Instead it chooses to panic, which is not a nice thing to do. :) Or do I have to high hopes of ZFS? Next question to answer is why this WD RED on: arcmsr0@pci0:7:14:0: class=0x010400 card=0x112017d3 chip=0x112017d3 rev=0x00 hdr=0x00 vendor = 'Areca Technology Corp.' device = 'ARC-1120 8-Port PCI-X to SATA RAID Controller' class = mass storage subclass = RAID got hung, and nothing for this shows in SMART.... Thanx, --WjW (If needed vmcore available) From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 16:11:44 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4E31ADA3 for ; Sat, 20 Jun 2015 16:11:44 +0000 (UTC) (envelope-from daryl@isletech.net) Received: from mail.isletech.net (mail.isletech.net [IPv6:2001:470:1d:c44::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 289E682D for ; Sat, 20 Jun 2015 16:11:44 +0000 (UTC) (envelope-from daryl@isletech.net) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=isletech.net; s=isle2014; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:References:Subject:To:MIME-Version:From:Date:Message-ID; bh=JNBaezXt2HzlR3AaUA0UMOzvY2J2n/2AGaNhtYqznbs=; b=Tm4g4IP9cE/lR/+fsBVSY0khhrXbJVio8vae/EUNBu105AJ0YZ055GjnaLme5MFtKiPqdZSqlwxcNRaNEc9DL/1fMt6DaVZ4SfrsOwajc4TBJYXv0+gDkZ8bFQ8DU+sMVIZ6sO18ccFEZM8Ztxm5K5GKlxAcK30s2T8cpiwIe5A=; Message-ID: <558590BD.40603@isletech.net> Date: Sat, 20 Jun 2015 12:11:41 -0400 From: Daryl Richards User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: This diskfailure should not panic a system, but just disconnect disk from ZFS References: <5585767B.4000206@digiware.nl> In-Reply-To: <5585767B.4000206@digiware.nl> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 16:11:44 -0000 Check the failmode setting on your pool. From man zpool: failmode=wait | continue | panic Controls the system behavior in the event of catastrophic pool failure. This condition is typically a result of a loss of connectivity to the underlying storage device(s) or a failure of all devices within the pool. The behavior of such an event is determined as follows: wait Blocks all I/O access until the device connectivity is recovered and the errors are cleared. This is the default behavior. continue Returns EIO to any new write I/O requests but allows reads to any of the remaining healthy devices. Any write requests that have yet to be committed to disk would be blocked. panic Prints out a message to the console and generates a system crash dump. On 2015-06-20 10:19 AM, Willem Jan Withagen wrote: > Hi, > > Found my system rebooted this morning: > > Jun 20 05:28:33 zfs kernel: sonewconn: pcb 0xfffff8011b6da498: Listen > queue overflow: 8 already in queue awaiting acceptance (48 occurrences) > Jun 20 05:28:33 zfs kernel: panic: I/O to pool 'zfsraid' appears to be > hung on vdev guid 18180224580327100979 at '/dev/da0'. > Jun 20 05:28:33 zfs kernel: cpuid = 0 > Jun 20 05:28:33 zfs kernel: Uptime: 8d9h7m9s > Jun 20 05:28:33 zfs kernel: Dumping 6445 out of 8174 > MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > Which leads me to believe that /dev/da0 went out on vacation, leaving > ZFS into trouble.... But the array is: > ---- > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP > zfsraid 32.5T 13.3T 19.2T - 7% 41% 1.00x > ONLINE - > raidz2 16.2T 6.67T 9.58T - 8% 41% > da0 - - - - - - > da1 - - - - - - > da2 - - - - - - > da3 - - - - - - > da4 - - - - - - > da5 - - - - - - > raidz2 16.2T 6.67T 9.58T - 7% 41% > da6 - - - - - - > da7 - - - - - - > ada4 - - - - - - > ada5 - - - - - - > ada6 - - - - - - > ada7 - - - - - - > mirror 504M 1.73M 502M - 39% 0% > gpt/log0 - - - - - - > gpt/log1 - - - - - - > cache - - - - - - > gpt/raidcache0 109G 1.34G 107G - 0% 1% > gpt/raidcache1 109G 787M 108G - 0% 0% > ---- > > And thus I'd would have expected that ZFS would disconnect /dev/da0 and > then switch to DEGRADED state and continue, letting the operator fix the > broken disk. > Instead it chooses to panic, which is not a nice thing to do. :) > > Or do I have to high hopes of ZFS? > > Next question to answer is why this WD RED on: > > arcmsr0@pci0:7:14:0: class=0x010400 card=0x112017d3 chip=0x112017d3 > rev=0x00 hdr=0x00 > vendor = 'Areca Technology Corp.' > device = 'ARC-1120 8-Port PCI-X to SATA RAID Controller' > class = mass storage > subclass = RAID > > got hung, and nothing for this shows in SMART.... > > > Thanx, > --WjW > > (If needed vmcore available) > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- Daryl Richards Isle Technical Services Inc. From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 22:14:48 2015 Return-Path: Delivered-To: fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3919E6AC for ; Sat, 20 Jun 2015 22:14:48 +0000 (UTC) (envelope-from swills@mouf.net) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EC722280 for ; Sat, 20 Jun 2015 22:14:47 +0000 (UTC) (envelope-from swills@mouf.net) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.9/8.14.9) with ESMTP id t5KMEXFd031117 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sat, 20 Jun 2015 22:14:38 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.9/8.14.9/Submit) id t5KMEWiS031116; Sat, 20 Jun 2015 22:14:32 GMT (envelope-from swills) Date: Sat, 20 Jun 2015 22:14:32 +0000 From: Steve Wills To: Willem Jan Withagen Cc: fs@freebsd.org Subject: Re: This diskfailure should not panic a system, but just disconnect disk from ZFS Message-ID: <20150620221431.GB26416@mouf.net> References: <5585767B.4000206@digiware.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5585767B.4000206@digiware.nl> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Sat, 20 Jun 2015 22:14:38 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=HEADER_FROM_DIFFERENT_DOMAINS autolearn=unavailable autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mouf.net X-Virus-Scanned: clamav-milter 0.98.7 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 22:14:48 -0000 On Sat, Jun 20, 2015 at 04:19:39PM +0200, Willem Jan Withagen wrote: > Hi, > > Found my system rebooted this morning: > > Jun 20 05:28:33 zfs kernel: sonewconn: pcb 0xfffff8011b6da498: Listen > queue overflow: 8 already in queue awaiting acceptance (48 occurrences) > Jun 20 05:28:33 zfs kernel: panic: I/O to pool 'zfsraid' appears to be > hung on vdev guid 18180224580327100979 at '/dev/da0'. > Jun 20 05:28:33 zfs kernel: cpuid = 0 > Jun 20 05:28:33 zfs kernel: Uptime: 8d9h7m9s > Jun 20 05:28:33 zfs kernel: Dumping 6445 out of 8174 > MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > Which leads me to believe that /dev/da0 went out on vacation, leaving > ZFS into trouble.... But the array is: > ---- > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP > zfsraid 32.5T 13.3T 19.2T - 7% 41% 1.00x > ONLINE - > raidz2 16.2T 6.67T 9.58T - 8% 41% > da0 - - - - - - > da1 - - - - - - > da2 - - - - - - > da3 - - - - - - > da4 - - - - - - > da5 - - - - - - > raidz2 16.2T 6.67T 9.58T - 7% 41% > da6 - - - - - - > da7 - - - - - - > ada4 - - - - - - > ada5 - - - - - - > ada6 - - - - - - > ada7 - - - - - - > mirror 504M 1.73M 502M - 39% 0% > gpt/log0 - - - - - - > gpt/log1 - - - - - - > cache - - - - - - > gpt/raidcache0 109G 1.34G 107G - 0% 1% > gpt/raidcache1 109G 787M 108G - 0% 0% > ---- > > And thus I'd would have expected that ZFS would disconnect /dev/da0 and > then switch to DEGRADED state and continue, letting the operator fix the > broken disk. > Instead it chooses to panic, which is not a nice thing to do. :) > > Or do I have to high hopes of ZFS? > > Next question to answer is why this WD RED on: > > arcmsr0@pci0:7:14:0: class=0x010400 card=0x112017d3 chip=0x112017d3 > rev=0x00 hdr=0x00 > vendor = 'Areca Technology Corp.' > device = 'ARC-1120 8-Port PCI-X to SATA RAID Controller' > class = mass storage > subclass = RAID > > got hung, and nothing for this shows in SMART.... > You may be hitting the zfs deadman panic, which is triggered when the controller hangs. This can in some cases be caused by disks that die in unusual ways. > > (If needed vmcore available) > The backtrace might confirm or dispute my theory. Steve From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 23:06:10 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 06CC8C5A for ; Sat, 20 Jun 2015 23:06:10 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id C2DE3F1B for ; Sat, 20 Jun 2015 23:06:09 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2ABBADI8IVV/95baINcg2RfBoMYumcJgVwKhS5KAoFbFAEBAQEBAQGBCoQiAQEBAwEBAQEgKyALBRYYAgINGQIpAQkmBggHBAEcAgKIBggNsFCVdgEBAQEGAQEBAQEBARuBIYokhDQBAQUXNAeCaIFDBZN9hFaEL4REi3SKQiZjgVmBWSIxB4EFOoECAQEB X-IronPort-AV: E=Sophos;i="5.13,651,1427774400"; d="scan'208";a="217616347" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 20 Jun 2015 19:06:03 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 34EFDB3F78; Sat, 20 Jun 2015 19:06:03 -0400 (EDT) Date: Sat, 20 Jun 2015 19:06:03 -0400 (EDT) From: Rick Macklem To: "alex.burlyga.ietf alex.burlyga.ietf" Cc: freebsd-fs@freebsd.org Message-ID: <616007475.61421820.1434841563204.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: [nfs][client] - Question about handling of the NFS3_EEXIST error in SYMLINK rpc MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 23:06:10 -0000 Alex Burlyga wrote: > Hi, > > NFS client code in nfsrpc_symlink() masks server returned NFS3_EEXIST > error > code > by returning 0 to the upper layers. I'm assuming this was an attempt > to > work around > some server's broken replay cache out there, however, it breaks a > more > common > case where server is returning EEXIST for legitimate reason and > application > is expecting this error code and equipped to deal with it. > Btw, since a DRC is a cache, any DRC can fail to handle a duplicate (because it has fallen out of the cache). Further, many DRCs are only used for UDP and when a client ends up doing a TCP reconnect (due to a network partitioning breaking the TCP connection or similar) it will resend any outstanding RPCs. - In other words, a DRC doesn`t guarantee that a duplicate won`t be performed, it just reduces the likelyhood. NFSv4.1 fixes this via sessions, which guarantee exactly once RPC semantics. > To fix it I see three ways of doing this: > * Remove offending code > * Make it optional, sysctl? > * On NFS3_EEXIST send READLINK rpc to make sure symlink content is > right > > Which of the ways will maximize the chances of getting this fix > upstream? > > One more point, old client circa FreeBSD 7.0 does not exhibit this > problem. > Since the old client didn`t do this, I think that turning it off by default and having a sysctl to enable it would be fine. (It is probably left over from the OpenBSD code which was the original base for my NFSv4 work.) rick > Alex > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Sat Jun 20 23:22:38 2015 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E0463E9B for ; Sat, 20 Jun 2015 23:22:38 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id A92043F1 for ; Sat, 20 Jun 2015 23:22:38 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2ABBACF9IVV/95baINcg2RfBoMYumcJgVwKhS5KAoFbFAEBAQEBAQGBCoQiAQEBAwEBAQEgKyALBRYYAgINGQIpAQkmBggHBAEcBIgGCA2wUZV2AQEBAQEFAQEBAQEBAQEagSGKJIQ0AQEFFzQHgmiBQwWTfYRWhC+ERIt0ikImY4FZgVkiMQeBBTqBAgEBAQ X-IronPort-AV: E=Sophos;i="5.13,651,1427774400"; d="scan'208";a="217618327" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 20 Jun 2015 19:22:37 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 7D409B3FB3; Sat, 20 Jun 2015 19:22:37 -0400 (EDT) Date: Sat, 20 Jun 2015 19:22:37 -0400 (EDT) From: Rick Macklem To: "alex.burlyga.ietf alex.burlyga.ietf" Cc: freebsd-fs@freebsd.org Message-ID: <154571553.61423606.1434842557502.JavaMail.root@uoguelph.ca> In-Reply-To: Subject: Re: [nfs][client] - Question about handling of the NFS3_EEXIST error in SYMLINK rpc MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Jun 2015 23:22:39 -0000 Alex Burlyga wrote: > Hi, > > NFS client code in nfsrpc_symlink() masks server returned NFS3_EEXIST > error > code > by returning 0 to the upper layers. I'm assuming this was an attempt > to > work around > some server's broken replay cache out there, however, it breaks a > more > common > case where server is returning EEXIST for legitimate reason and > application > is expecting this error code and equipped to deal with it. > > To fix it I see three ways of doing this: > * Remove offending code > * Make it optional, sysctl? > * On NFS3_EEXIST send READLINK rpc to make sure symlink content is > right > > Which of the ways will maximize the chances of getting this fix > upstream? > Btw, I felt #2 above was appropriate but forgot to ask what others thought w.r.t. this? Thanks, rick > One more point, old client circa FreeBSD 7.0 does not exhibit this > problem. > > Alex > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >