From owner-freebsd-fs@FreeBSD.ORG Tue Jul 10 18:57:38 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BBB12106564A for ; Tue, 10 Jul 2012 18:57:38 +0000 (UTC) (envelope-from jusher71@yahoo.com) Received: from nm15-vm6.bullet.mail.ne1.yahoo.com (nm15-vm6.bullet.mail.ne1.yahoo.com [98.138.91.108]) by mx1.freebsd.org (Postfix) with SMTP id 527448FC12 for ; Tue, 10 Jul 2012 18:57:38 +0000 (UTC) Received: from [98.138.90.51] by nm15.bullet.mail.ne1.yahoo.com with NNFMP; 10 Jul 2012 18:57:37 -0000 Received: from [98.138.89.248] by tm4.bullet.mail.ne1.yahoo.com with NNFMP; 10 Jul 2012 18:57:37 -0000 Received: from [127.0.0.1] by omp1040.mail.ne1.yahoo.com with NNFMP; 10 Jul 2012 18:57:37 -0000 X-Yahoo-Newman-Property: ymail-5 X-Yahoo-Newman-Id: 561835.97288.bm@omp1040.mail.ne1.yahoo.com Received: (qmail 66562 invoked by uid 60001); 10 Jul 2012 18:57:37 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1341946657; bh=kqUJLmv1ubi23eNij34HyHuxtIJmJdykilJezcvlrDw=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=JuZN5DjjOsJSZ7yRiNwOaDvXaMhlP7Wys02QQBxBpLCkDHnqQHFJ7KkmTuSS4Qs49mG831wunpR84UDZQmwfCg7NMub0FL9NlvMECqZp46ZfpHI+Yys+QnfnUe5V3sk02bP4AQB/a1hWeMDrDRuX5TKb+4frodfhNVZN9YtIxlI= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=0pGDsiRVDpfzx6Q31fnsNdOu4MVHPOaR/P0TeauIc4IJzFlCRr4izInORWi6tU7AZSCUu3qftNk5zH1xxw/+uHFdekBXyUpVSBNv2BIvmvqD9b5GrlwIp6WwWT9NSI9bg3krSM64h8vzD0efuvPsY6zBy4anEuBoc+TmgDiGE3s=; X-YMail-OSG: uJ9owasVM1nMMBdYPPcpa__jIrB5Ys53Phm_N9YHIBA1fPP th7KtXtigRiz_Ix8c1v7yGvbigwuYyzu3CtCRhQz_MVTgn193Oj9f3X0fCup qV.YJ0JR8YIQX5v2pFVZ1N41ltAiwtJcmj.D5UXY8us0Ig7EaZdILFbJHm6S ivFndhlmwcoLmaZu5y2NNxlFPQ0b7gHSxQeyKWLLZ8xgfB7aVksvslRMPUbI KYKVrMO1hL_XW6G2etfa_O.ZxT9mzMajKfBsaPy92GomaHsRUy4IYrWWoRrX UEXmz1e29pLKvtT4zARVZ5PiVDW9i9NZqbXvcksLRDS0hgSosD2WJMRMuv9N TcglK5qGfDytAw5T6XjjYZVpFYQLJ5s0yEzSWw84r9rAOLp4WFHtMyMPEOLz Y Received: from [32.178.134.54] by web122505.mail.ne1.yahoo.com via HTTP; Tue, 10 Jul 2012 11:57:37 PDT X-Mailer: YahooMailClassic/15.0.8 YahooMailWebService/0.8.120.356233 Message-ID: <1341946657.18535.YahooMailClassic@web122505.mail.ne1.yahoo.com> Date: Tue, 10 Jul 2012 11:57:37 -0700 (PDT) From: Jason Usher To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: chaining JBOD chassic to server ... why am I scared ? (ZFS) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jul 2012 18:57:38 -0000 The de-facto configuration the smart folks are using for ZFS seems to be: - 16/24/36 drive supermicro chassis - LSI 9211-8i internal cards - ZFS and probably raidz2 or raidz3 vdevs Ok, fine. But then I see some even smarter folks attaching the 48-drive 4U JBOD chassis to this configuration, probably using a different LSI card that has an external SAS cable. So ... 84 drives accessible to ZFS on one system. In terms of space and money efficiency, it sounds really great - fewer systems to manage, etc. But this scares me ... - two different power sources - so the "head unit" can lose power independent of the JBOD device ... how well does that turn out ? - external cabling - has anyone just yanked that external SAS cable a few times, and what does that look like ? - If you have a single SLOG, or a single L2ARC device, where do you put it ? And then what happens if "the other half" of the system detaches from the half that the SLOG/L2ARC is in ? - ... any number of other weird things ? Just how well does ZFS v28 deal with these kind of situations, and do I have a good reason to be awfully shy about doing this ?