From owner-freebsd-current@FreeBSD.ORG Mon Jan 7 15:37:17 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F64916A468 for ; Mon, 7 Jan 2008 15:37:17 +0000 (UTC) (envelope-from ticso@cicely12.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id D2A7613C4CC for ; Mon, 7 Jan 2008 15:37:16 +0000 (UTC) (envelope-from ticso@cicely12.cicely.de) Received: from cicely5.cicely.de ([10.1.1.7]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id m07FbB1e034900; Mon, 7 Jan 2008 16:37:11 +0100 (CET) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (cicely12.cicely.de [10.1.1.14]) by cicely5.cicely.de (8.13.4/8.13.4) with ESMTP id m07Favro076038 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 7 Jan 2008 16:36:57 +0100 (CET) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (localhost [127.0.0.1]) by cicely12.cicely.de (8.13.4/8.13.3) with ESMTP id m07FavPN076165; Mon, 7 Jan 2008 16:36:57 +0100 (CET) (envelope-from ticso@cicely12.cicely.de) Received: (from ticso@localhost) by cicely12.cicely.de (8.13.4/8.13.3/Submit) id m07Fat2l076164; Mon, 7 Jan 2008 16:36:55 +0100 (CET) (envelope-from ticso) Date: Mon, 7 Jan 2008 16:36:55 +0100 From: Bernd Walter To: Maxim Sobolev Message-ID: <20080107153655.GI65134@cicely12.cicely.de> References: <4780E546.9050303@FreeBSD.org> <9bbcef730801060651y489f1f9bw269d0968407dd8fb@mail.gmail.com> <4780EF09.4090908@FreeBSD.org> <47810BE3.4080601@FreeBSD.org> <4781227B.5020800@rcn.com> <47814EAB.70405@FreeBSD.org> <20080106233254.GE1138@egr.msu.edu> <4781791A.2090401@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4781791A.2090401@FreeBSD.org> X-Operating-System: FreeBSD cicely12.cicely.de 5.4-STABLE alpha User-Agent: Mutt/1.5.9i X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED=-1.8, BAYES_00=-2.599 autolearn=ham version=3.2.3 X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on cicely12.cicely.de Cc: Adam McDougall , freebsd-current@freebsd.org Subject: Re: Should we simply disallow ZFS on FreeBSD/i386? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2008 15:37:17 -0000 On Sun, Jan 06, 2008 at 04:58:02PM -0800, Maxim Sobolev wrote: > Adam McDougall wrote: > >A summary of my opinion on this matter is that some i386 FreeBSD servers > >do have a place running zfs in a useful role, but some dedication and > >patience from the administrator is usually required, and the effort to > >tune at least kmem is nearly > > In Russian we have a good saying: "You can teach a bear to ride a > bicycle, but will it ever enjoy it?" I enjoy my i386/ZFS Servers. It is running with just 384MB RAM as the only instability it currently has is because the / disk ist dying. But even with this it has 71 days uptime. And my backup server is also based on ZFS with just 196MB RAM. This one isn't running stable, but it is stable for just doing the zfs imports and restoring some files. The only 64 bit machines I have at home are alpha and spac64, so no option for ZFS right now. I don't see a real difference between running a 2G i386 and a 2G amd64 server. 8GB amd64 boxes are still not very common. Of course the i386 must be tuned to have enough kmem and KVA and of course doing so reduces the application space, but it is still within the hardware limitations and an NFS-Server doesn't need much application address space anyway. Considered that we seem to have a limitation on running amd64 with more than 2G kmem there is even more to consider. > The same is here - seemingly due to the ZFS design limitations and > limitations of the FreeBSD kernel you can't get ZFS to run reliably out > of the box on i386. Yes, you can probably do some tweaks here and there, > to make it more of less stable given the workload, but that's not what > most of the FreeBSD users expect from the file system. Unlike you, most > of administrators won't even bother to read tweaking documentation > explaining why ZFS is so tricky in i386, let alone doing actual > trial-and-error to determine the right set of tunables. More likely at > the first incident they would just dismiss FreeBSD/ZFS as a crap. This is true however. I'm OK with a big fat warning on i386 and/or a loader env to be set to reenable this for persons who know (or think they do) what they are doing. If we say ZFS on i386 isn't supported than at least it shouldn't be able to be configured without anyone hitting a special knob. But in my opinion ZFS shouldn't overflow kmem storage at first. This is not only an i386 problem as rasing kmem makes ZFS more hungry by default, which is only good if kmem isn't used for something else. I have an amd64 system with 4G RAM and kmem defaults to 419430400 Bytes. On my 384MB i386 box I have (tuned of course) 335544320 Bytes kmem. And with more RAM I could easily go over the default on the amd64 box. Well RAM is too expensive since it is SDRAM, but there are many DDR boxes without amd64 functionality out there, which allows adding affordable memory in the nGB range. I had to reduce ARC sizes to get the 384MB box stable - the OS version is quite old and things have been modified for that in the meantime, but considered that 2G i386 systems still panic with more kmem than I have RAM it simply says that it wouldn't run out of the box on amd64 with the same applications either. -- B.Walter http://www.bwct.de http://www.fizon.de bernd@bwct.de info@bwct.de support@fizon.de