Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 31 Aug 2007 23:38:13 +0200
From:      Pawel Jakub Dawidek <pjd@FreeBSD.org>
To:        Kenneth Vestergaard Schmidt <kvs@pil.dk>
Cc:        freebsd-fs@freebsd.org
Subject:   Re: ZFS: 'checksum mismatch' all over the place
Message-ID:  <20070831213812.GB13098@garage.freebsd.pl>
In-Reply-To: <m13axza00q.fsf@binarysolutions.dk>
References:  <m1wsvtkviw.fsf@binarysolutions.dk> <20070820112946.GC16977@garage.freebsd.pl> <m1ps1iz9bi.fsf@binarysolutions.dk> <m13axza00q.fsf@binarysolutions.dk>

next in thread | previous in thread | raw e-mail | index | archive | help

--RASg3xLB4tUQ4RcS
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Aug 31, 2007 at 11:04:37PM +0200, Kenneth Vestergaard Schmidt wrote:
> Kenneth Vestergaard Schmidt <kvs@pil.dk> writes:
> >> How do you know it was fine? Did you have something that did
> >> checksumming? You could try geli with integrity verification feature
> >> turned on, fill the disks with some random data and then read it back,
> >> if your controller corrupts the data, geli should tell you this.
> >
> > I may have to do this. The previous drive was almost filled to the brim
> > with data, which rsync looked at each day, and we didn't have a lot of
> > re-transfer, but that doesn't necessarily mean anything.
>=20
> *blush*
>=20
> This turned out to be a firmware-issue with the Eonstor
> RAID-enclosure. After upgrading to v3.47, everything is fine in the
> checksum-department.
>=20
> Now, however, I can't seem to keep the box running. We've rsync'd 1.56
> TB data to an 8.18 TB raidz2 pool, and we're getting panics all the
> time.
>=20
> It's an x86 with 4 GB RAM. I've got the following in /boot/loader.conf:
>=20
>   vfs.zfs.prefetch_disable=3D"1"
>   vfs.zfs.arc_max=3D"107772160"
>   vm.kmem_size_max=3D"629145600"
>   vm.kmem_size_min=3D"629145600"
>=20
> and kern.maxvnodes is set to 50000. When the machine is finished
> booting, 'vmstat -m' says:
>=20
>          Type InUse MemUse HighUse Requests  Size(s)
>       solaris 49972 158199K       -   455307  16,32,64,128,256,512,1024,2=
048,4096
>=20
> and after about an hours worth of rsync'ing, we get:
>=20
>          Type InUse MemUse HighUse Requests  Size(s)
>       solaris 198797 449675K       - 404226785  16,32,64,128,256,512,1024=
,2048,4096
>   panic: kmem_malloc(28672): kmem_map too small: 614682624 total allocated
>=20
> I'm not quite sure what knobs to twiddle with, or what values to watch,
> so any help in this department would be much appreciated. I'm sure it'd
> be nice to update the Wiki, too, with that info, since the values there
> don't make things stable.

Is this recent HEAD? If so, this may be because of too much metadata
caching in ZFS. This was fixed in ZFS, so now ARC can limit metadata
cache, but this change is not yet in the tree (only in my perforce
branch).

--=20
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd@FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!

--RASg3xLB4tUQ4RcS
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)

iD8DBQFG2IpEForvXbEpPzQRAqsAAKCiO4IWQmzSHLAONUJjLqPpktzQZgCgq6Zf
8vwO0vp6HNC+R5JWWgOaTFA=
=PuMI
-----END PGP SIGNATURE-----

--RASg3xLB4tUQ4RcS--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070831213812.GB13098>