From owner-freebsd-fs@FreeBSD.ORG Tue Jan 22 14:34:47 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AE2EB6BD for ; Tue, 22 Jan 2013 14:34:47 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 46D8399C for ; Tue, 22 Jan 2013 14:34:46 +0000 (UTC) Received: from [127.0.0.1] (Scott4long@pooker.samsco.org [168.103.85.57]) (authenticated bits=0) by pooker.samsco.org (8.14.5/8.14.5) with ESMTP id r0MEXxPb077679; Tue, 22 Jan 2013 07:33:59 -0700 (MST) (envelope-from scottl@samsco.org) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: RFC: Suggesting ZFS "best practices" in FreeBSD From: Scott Long In-Reply-To: <314B600D-E8E6-4300-B60F-33D5FA5A39CF@sarenet.es> Date: Tue, 22 Jan 2013 07:33:59 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <565CB55B-9A75-47F4-A88B-18FA8556E6A2@samsco.org> References: <314B600D-E8E6-4300-B60F-33D5FA5A39CF@sarenet.es> To: Borja Marcos X-Mailer: Apple Mail (2.1499) X-Spam-Status: No, score=-50.0 required=3.8 tests=ALL_TRUSTED, T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on pooker.samsco.org Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2013 14:34:47 -0000 On Jan 22, 2013, at 4:03 AM, Borja Marcos wrote: > (Scott, I hope you don't mind to be CC'd, I'm not sure you read the = -FS mailing list, and this is a SCSI//FS issue) >=20 >=20 >=20 > Hi :) >=20 > Hope nobody will hate me too much, but ZFS usage under FreeBSD is = still chaotic. We badly need a well proven "doctrine" in order to avoid = problems. Especially, we need to avoid the braindead Linux HOWTO-esque = crap of endless commands for which no rationale is offered at all, and = which mix personal preferences and even misconceptions as "advice" (I = saw one of those howtos which suggested disabling checksums "because = they are useless"). >=20 > ZFS is a very different beast from other filesystems, and the setup = can involve some non-obvious decisions. Worse, Windows oriented server = vendors insist on bundling servers with crappy raid controllers which = tend to make things worse. >=20 > Since I've been using ZFS on FreeBSD (from the first versions) I have = noticed several serious problems. I try to explain some of them, and my = suggestions for a solution. We should collect more use cases and issues = and try to reach a consensus.=20 >=20 >=20 >=20 > 1- Dynamic disk naming -> We should use static naming (GPT labels, for = instance) >=20 > ZFS was born in a system with static device naming (Solaris). When you = plug a disk it gets a fixed name. As far as I know, at least from my = experience with Sun boxes, c1t3d12 is always c1t3d12. FreeBSD's dynamic = naming can be very problematic. >=20 Look up SCSI device wiring in /sys/conf/NOTES. That's one solution to = static naming, just with a slightly different angle than Solaris. I do = agree with your general thesis here, and either wiring should be made a = much more visible and documented feature, or a new mechanism should be = developed to provide naming stability. Please let me know what you = think of the wiring mechanic. >=20 >=20 > 2- RAID cards. >=20 > Simply: Avoid them like the pest. ZFS is designed to operate on bare = disks. And it does an amazingly good job. Any additional software layer = you add on top will compromise it. I have had bad experiences with "mfi" = and "aac" cards.=20 >=20 Agree 200%. Despite the best effort of sales and marketing people, RAID = cards do not make good HBAs. At best they add latency. At worst, they = add a lot of latency and extra failure modes. Scott