Date: Wed, 9 Jul 2008 12:37:29 +0200 From: CZUCZY Gergely <gergely.czuczy@harmless.hu> To: Jeremy Chadwick <koitsu@FreeBSD.org> Cc: freebsd-fs@freebsd.org, Kris Kennaway <kris@FreeBSD.org> Subject: Re: Thinking of using ZFS/FBSD for a backup system Message-ID: <20080709123729.60d2431a@twoflower.in.publishing.hu> In-Reply-To: <20080709055645.GA40076@eos.sc1.parodius.com> References: <bd9320b30807072315x105cf058tf9f952f0f5bb2a6a@mail.gmail.com> <20080708100701.57031cda@twoflower.in.publishing.hu> <bd9320b30807080131j5e0e02a4y3231d7bfa1738517@mail.gmail.com> <4873C4FA.2020004@FreeBSD.org> <20080708221327.5c1d0e92@mort.in.publishing.hu> <4873CF6C.7000205@FreeBSD.org> <20080708225449.1070252d@mort.in.publishing.hu> <4873F4E9.3040203@FreeBSD.org> <20080709074420.24df3be4@mort.in.publishing.hu> <20080709055645.GA40076@eos.sc1.parodius.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Sig_/R3qcmpkSNJ2xBKy29soX7Ea Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thank you for your thought, Jeremy. Yes, I was trying to refer to these thi= ngs. Putting more memory into the system then ZFS's need doesn't prove anything. There's no _proven_ _garantee_ that it won't panic, it just makes it more difficult (lowers the possibilty) to panic the system. As you said tuning ARC to a lower and kmem_size{,_max} to a higher value ma= kes it less likely to panic, but this won't garantee anything, just makes panic= ing bigger. "Stable ZFS" would mean, that these circumstances are cleared, and there's a proven garantee (either mathematically) that it's _unable_ to panic due to = this memory allocation issue. It's still there, but with a bigger amount of memory it's less likely to ha= ppen. And I haven't tried prefetch_disable back then. So i've got no experiences = on the effects of prefetch_disable. On Tue, 8 Jul 2008 22:56:45 -0700 Jeremy Chadwick <koitsu@FreeBSD.org> wrote: > On Wed, Jul 09, 2008 at 07:44:20AM +0200, CZUCZY Gergely wrote: > > On Wed, 09 Jul 2008 01:14:49 +0200 > > Kris Kennaway <kris@FreeBSD.org> wrote: > >=20 > > > CZUCZY Gergely wrote: > > > I don't know; empirically my setup is an upper bound. How large was > > > "as large as it was allowed" for you? > > Well, we cannot buy "upper bounds" all over, just because some > > developer is unable to figure out things. I think you can't expect > > FreeBSD users to spend as much money as possible, just because the devs > > can't tell how much is enough... > > It seems more like a twilight zone then a stable feature now ;) > >=20 > > It was exactly as much as an amd64 installation would allow with 2GB of > > physical memory. We've dismissed the setup around february, and I don't > > have the configs anymore. It was an amd64 setup with 2GB of physical > > memory. >=20 > The bottom line here is that i386 and amd64 both have a kmem_size limit > of 2GB. You can throw 32GB of RAM into an amd64 box, but FreeBSD will > only utilise up to 2GB of that for kmem. That is purely a FreeBSD > limitation, and is being dealt with in HEAD by Alan Cox. I believe he > has a patch, or it may have been committed -- I don't follow HEAD. I > can point people to a mailing list URL, if needed. >=20 > This is one of the limitations Gergely is referring to. >=20 > Since ZFS is incredibly memory-hungry, you're forced to tune ZFS to try > and get it to "play nice" with that 2GB limit on STABLE/RELEASE systems. > You also need to keep in mind that you can't just set kmem_size and > kmem_size_max to 2048M, because the kernel needs memory for other > things. >=20 > The tuning parameters I use on my 2GB amd64 and 4GB amd64 boxes are: >=20 > vm.kmem_size=3D"1536M" > vm.kmem_size_max=3D"1536M" > vfs.zfs.arc_min=3D"16M" > vfs.zfs.arc_max=3D"64M" >=20 > If you set kmem_size and kmem_size_max any higher than that, the machine > will panic on boot, stating (indirectly) that there isn't enough memory > available for the kernel to allocate for other things. >=20 > Until I added the arc_min and arc_max setting, I could occasionally > panic the machines under very heavy load (heavy zpool I/O), caused by > kmem exhaustion. Since adding the arc_* tunings, I've tried very hard > to crash the machines, and I cannot. >=20 > But there's absolutely no guarantee those tuning parameters above will > ensure FreeBSD won't panic due to kmem exhaustion. I believe this is > the point Gergely is making about the "stability" of the whole thing. >=20 > Now, with regards to prefetch_disable, folks can disable that if they > want. I disable it on my above systems because for what they do, the > overall performance appears better with prefetching disabled. >=20 > I hope this helps shed some light here... >=20 --=20 =C3=9Cdv=C3=B6lettel, Czuczy Gergely Harmless Digital Bt mailto: gergely.czuczy@harmless.hu Tel: +36-30-9702963 --Sig_/R3qcmpkSNJ2xBKy29soX7Ea Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.3 (FreeBSD) iD8DBQFIdJTrzrC0WyuMkpsRAoY2AJwM1zl32WGmIa4hh0vWga7X6ZrgUwCcDdtO Orwh0RNP8VsPdN2HlfwAqdw= =C73I -----END PGP SIGNATURE----- --Sig_/R3qcmpkSNJ2xBKy29soX7Ea--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080709123729.60d2431a>