Date: Wed, 09 Jul 2008 12:26:07 +0200 From: Kris Kennaway <kris@FreeBSD.org> To: Jeremy Chadwick <koitsu@FreeBSD.org> Cc: freebsd-fs@freebsd.org Subject: Re: Thinking of using ZFS/FBSD for a backup system Message-ID: <4874923F.8080303@FreeBSD.org> In-Reply-To: <20080709055645.GA40076@eos.sc1.parodius.com> References: <bd9320b30807072315x105cf058tf9f952f0f5bb2a6a@mail.gmail.com> <20080708100701.57031cda@twoflower.in.publishing.hu> <bd9320b30807080131j5e0e02a4y3231d7bfa1738517@mail.gmail.com> <4873C4FA.2020004@FreeBSD.org> <20080708221327.5c1d0e92@mort.in.publishing.hu> <4873CF6C.7000205@FreeBSD.org> <20080708225449.1070252d@mort.in.publishing.hu> <4873F4E9.3040203@FreeBSD.org> <20080709074420.24df3be4@mort.in.publishing.hu> <20080709055645.GA40076@eos.sc1.parodius.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Jeremy Chadwick wrote: > On Wed, Jul 09, 2008 at 07:44:20AM +0200, CZUCZY Gergely wrote: >> On Wed, 09 Jul 2008 01:14:49 +0200 >> Kris Kennaway <kris@FreeBSD.org> wrote: >> >>> CZUCZY Gergely wrote: >>> I don't know; empirically my setup is an upper bound. How large was >>> "as large as it was allowed" for you? >> Well, we cannot buy "upper bounds" all over, just because some >> developer is unable to figure out things. I think you can't expect >> FreeBSD users to spend as much money as possible, just because the devs >> can't tell how much is enough... >> It seems more like a twilight zone then a stable feature now ;) >> >> It was exactly as much as an amd64 installation would allow with 2GB of >> physical memory. We've dismissed the setup around february, and I don't >> have the configs anymore. It was an amd64 setup with 2GB of physical >> memory. > > The bottom line here is that i386 and amd64 both have a kmem_size limit > of 2GB. No, it's the limit on KVA (address space), not kmem_size. On i386 there is only 1GB of KVA by default, so it's even harder to fit ZFS in. I thought you could tune it higher than 2GB if you liked, although this comes out of address space available to user programs (4GB total for user + kernel). > You can throw 32GB of RAM into an amd64 box, but FreeBSD will > only utilise up to 2GB of that for kmem. That is purely a FreeBSD > limitation, and is being dealt with in HEAD by Alan Cox. I believe he > has a patch, or it may have been committed -- I don't follow HEAD. I > can point people to a mailing list URL, if needed. > > This is one of the limitations Gergely is referring to. No it's not, since he has only 2GB of physical memory. > Since ZFS is incredibly memory-hungry, you're forced to tune ZFS to try > and get it to "play nice" with that 2GB limit on STABLE/RELEASE systems. > You also need to keep in mind that you can't just set kmem_size and > kmem_size_max to 2048M, because the kernel needs memory for other > things. > > The tuning parameters I use on my 2GB amd64 and 4GB amd64 boxes are: > > vm.kmem_size="1536M" > vm.kmem_size_max="1536M" > vfs.zfs.arc_min="16M" > vfs.zfs.arc_max="64M" > > If you set kmem_size and kmem_size_max any higher than that, the machine > will panic on boot, stating (indirectly) that there isn't enough memory > available for the kernel to allocate for other things. Yes, I said this earlier :) > Until I added the arc_min and arc_max setting, I could occasionally > panic the machines under very heavy load (heavy zpool I/O), caused by > kmem exhaustion. Since adding the arc_* tunings, I've tried very hard > to crash the machines, and I cannot. Good to hear. > But there's absolutely no guarantee those tuning parameters above will > ensure FreeBSD won't panic due to kmem exhaustion. I believe this is > the point Gergely is making about the "stability" of the whole thing. Not having the resources to run a very memory-intensive filesystem does not make it "unstable", it makes it "too memory intensive". Kris
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4874923F.8080303>