From owner-freebsd-fs@FreeBSD.ORG Mon Jul 8 23:17:47 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 715FC4BE for ; Mon, 8 Jul 2013 23:17:47 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by mx1.freebsd.org (Postfix) with ESMTP id 165D91E93 for ; Mon, 8 Jul 2013 23:17:47 +0000 (UTC) Received: from mfilter4-d.gandi.net (mfilter4-d.gandi.net [217.70.178.134]) by relay5-d.mail.gandi.net (Postfix) with ESMTP id 9E14441C064; Tue, 9 Jul 2013 01:17:30 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mfilter4-d.gandi.net Received: from relay5-d.mail.gandi.net ([217.70.183.197]) by mfilter4-d.gandi.net (mfilter4-d.gandi.net [10.0.15.180]) (amavisd-new, port 10024) with ESMTP id efbVz2KavqF0; Tue, 9 Jul 2013 01:17:29 +0200 (CEST) X-Originating-IP: 76.102.14.35 Received: from jdc.koitsu.org (c-76-102-14-35.hsd1.ca.comcast.net [76.102.14.35]) (Authenticated sender: jdc@koitsu.org) by relay5-d.mail.gandi.net (Postfix) with ESMTPSA id 261ED41C061; Tue, 9 Jul 2013 01:17:28 +0200 (CEST) Received: by icarus.home.lan (Postfix, from userid 1000) id 3A28B73A31; Mon, 8 Jul 2013 16:17:26 -0700 (PDT) Date: Mon, 8 Jul 2013 16:17:26 -0700 From: Jeremy Chadwick To: Berend de Boer Subject: Re: EBS snapshot backups from a FreeBSD zfs file system: zpool freeze? Message-ID: <20130708231726.GA91280@icarus.home.lan> References: <20130703055047.GA54853@icarus.home.lan> <6488DECC-2455-4E92-B432-C39490D18484@dragondata.com> <14A2336A-969C-4A13-9EFA-C0C42A12039F@hostpoint.ch> <87zjty11gn.wl%berend@pobox.com> <41CC5720-B1EA-4841-8BA5-893F4A628EAD@hostpoint.ch> <877gh024vy.wl%berend@pobox.com> <20130708210145.GA89605@icarus.home.lan> <87wqp0zo24.wl%berend@pobox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87wqp0zo24.wl%berend@pobox.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Jul 2013 23:17:47 -0000 On Tue, Jul 09, 2013 at 10:25:55AM +1200, Berend de Boer wrote: > >>>>> "Jeremy" == Jeremy Chadwick writes: > > Jeremy> Therefore, the answer/solution for you at this stage seems > Jeremy> to be: use Linux. Linux does what you need -- it offers > Jeremy> you guest-level utilities that interface with the > Jeremy> proprietary storage system back-end (EBS) that is offered > Jeremy> by your choice of hosting vendor (Amazon). > > Ding dong, nothing to do with guest level utilities. Completely > irrelevant, and I've repeated that numerous times here. All Amazon's > guest level tools work perfectly fine on FreeBSD. Amazon has ZERO > Linux/Windows/FreeBSD/whatever specific stuff. Then please explain this comment: > On Tue, Jul 09, 2013 at 08:06:25AM +1200, Berend de Boer wrote: > > >>>>> "Markus" == Markus Gebert writes: > > Markus> What you really need in that case is the ability to > > Markus> snapshot all EBS disks as group. > > > > Which Linux offers. Because what you're saying here, vs. what you said above, is highly conflicting as I read it. I do not understand how you can say "Amazon's guest-level tools work on FreeBSD", then immediately say "Amazon has zero OS-specific stuff", *while* saying "Linux offers {said thing I want}". These statements are extremely confusing. I think it would help if you would really start to provide *actual commands* of what you're doing to accomplish the tasks you want to accomplish -- both as you do them now on Linux, as well as the ZFS-related stuff you've done on FreeBSD. To my knowledge, no where in this thread have you actually shown scrollback/proof/etc. of what you've been doing, just magical one-liners that seem strange (for example, the "empty disk" thing would imply that EBS isn't even doing storage caching correctly, which makes no sense and cannot be true else *nothing* would work!). It might turn out to be that what you're seeing can be traced down to user error, or it can be traced back to the controller driver (on FreeBSD) not behaving how you want (might require SCSI quirks, etc.), etc... The possibilities are endless, and without hard data, I think everyone is struggling/throwing their arms up in the air. The reason I mention storage controller drivers is, for example, on ESXi (I forget what versions) it was discovered that cache flushing (the OS's way of telling the storage driver, which then tells the disks, "make sure everything is written to the platters") was a no-op of sorts -- ESXi would say "yep I've done it" when in fact it hadn't yet. With virtualisation this is a common situation, **solely because** of all the abstraction and caching (at so many layers that it's almost unfathomable -- no joke). I put EBS into the same category. Because really, all you know ***TRULY*** is "I see disk daX using controller xyzX", and the rest you have to pray/hope works (from the OS driver level all the way down to the physical media this stuff is written to on the back end / within EBS). The more abstraction = the more chances something will not behave how bare metal expects. > Jeremy> There is absolutely no shame in any way/shape/form in > Jeremy> using an OS that meets your needs/requirements. > > And that's why I'm trying to use FreeBSD as it has features that Linux > doesn't have. Trust me, if Linux had everything FreeBSD has, why would > I use FreeBSD? It's even more expensive on AWS! I'm glad you're trying it, but if you're at the stage where you're saying "Linux can do X/Y/Z", then it's up to **you** to weigh the pros and cons of moving away from Linux. Nobody here can make that decision for you, whether it be based on technical need or money or whatever else. > Jeremy> EBS is a closed, black-box-proprietary thing. > > Ding dong, it's just block storage like every other block storage out > there. Sigh. You once again generalise. Every kind of storage back-end is different, I don't know why you have such trouble understanding this. You're quite literally telling me that EBS is the same as iSCSI is the same as an ESXi file-based backing store is the same as a physical disk (remember your words: "disks are just software"), which is completely and entirely wrong -- their behaviours differ severely, not just as solutions/methods, but in *how they behave* when submit with I/O requests. All that "black magic" that goes on under the hood? It matters. Every single bit of it. > I fail to see that mentioning something Linux can do on this mailing > list would upset you? You don't want to know or read that?? I am one of the most "anti-FreeBSD" people you will ever encounter -- go ahead, ask anyone on this list, you will find that I am the one who is usually quick to take my "well then FreeBSD needs to get it's sh** together on a server level" soapbox. And that doesn't come lightly given that I've been using/working professionally with FreeBSD since the 2.2.x days. Prior to that I ran Linux (0.99pl45 until early 1.3.x). I'm just trying to give you some perspective into how I am, if it matters. What upsets me is that you're saying "Linux has XYZ", in turn creating a "FreeBSD vs. Linux" conflict, when I personally strive to see that kind of "OS war" advocacy end. I want to see people use whatever tool/OS/thing solves their dilemmas. If that thing is Linux, cool. If it's FreeBSD, cool. If it's Windows, cool. The dilemmas, limitations, and needs/etc. vary per person, per org, per company. I am against all forms of OS advocacy. I prefer to keep an open mind. So while I give you two thumbs up for giving FreeBSD a try, and ZFS too, but if Linux does something that you can't get on FreeBSD, then to me it seems like the choice is obvious. But that's me -- I love oversimplifying. ;-) -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB |