From owner-freebsd-fs@FreeBSD.ORG Mon Nov 12 08:06:00 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9363D68C for ; Mon, 12 Nov 2012 08:06:00 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230]) by mx1.freebsd.org (Postfix) with ESMTP id BF3638FC08 for ; Mon, 12 Nov 2012 08:05:58 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [192.92.129.5]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.5/8.14.5) with ESMTP id qAC85kK9088455 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 12 Nov 2012 10:05:47 +0200 (EET) (envelope-from daniel@digsys.bg) Message-ID: <50A0ADDA.9040205@digsys.bg> Date: Mon, 12 Nov 2012 10:05:46 +0200 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.10) Gecko/20121029 Thunderbird/10.0.10 MIME-Version: 1.0 To: Freddie Cash Subject: Re: ZFS HBAs + LSI chip sets (Was: ZFS hang (system #2)) References: <1350698905.86715.33.camel@btw.pki2.com> <1350711509.86715.59.camel@btw.pki2.com> <50825598.3070505@FreeBSD.org> <1350744349.88577.10.camel@btw.pki2.com> <1350765093.86715.69.camel@btw.pki2.com> <508322EC.4080700@FreeBSD.org> <1350778257.86715.106.camel@btw.pki2.com> <5084F6D5.5080400@digsys.bg> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Nov 2012 08:06:00 -0000 On 22.10.12 16:47, Freddie Cash wrote: > > I'll double-check when I get to work, but I'm pretty sure it's > 10.something. > > On Oct 22, 2012 12:34 AM, "Daniel Kalchev" > wrote: > > > > On 21.10.12 09:52, Freddie Cash wrote: > > [...] > > All three run without any serious issues. The only issues > we've had are 3, maybe 4, situations where I've tried to > destroy multi-TB filesystems without enough RAM in the > machine. We're now running a minimum of 32 GB of RAM with 64 > GB in one box. > > > What is the firmware on your LSI2008 controllers? > > I am having weird situation with one server that has LSI2008, on > 9-stable and all SSD configuration. One or two of the drives would > drop off the bus for no reason sometimes few times a day and > because the current driver ignores bus reset, someone has to > physically remove and re-insert the drives for them to come back. > Real pain. > My firmware version is 12.00.00.00 -- perhaps it is buggy? > As weird as it sounds, I discovered that my SSD-only zpool (raidz1) was using ashift=9. So, changed to ashift=12 and not seen disconnects anymore for a week now. Is 4k good for these SSDs? Or 8k is better, or larger? It seems it's really an SSD firmware problem, as the SSDs are likely doing more work when used with 512b sectors and from time to time fail to communicate properly with the bus. The SSDs are OCZ-VERTEX4 (firmware 1.5). Sometimes it seems to be drive related problem and perhaps the mps driver/hardware is too sensitive to drive issues. Daniel