Date: Thu, 08 Nov 2012 11:02:45 +0100 From: Xavier Beaudouin <kiwi@oav.net> To: <freebsd-fs@freebsd.org> Subject: 9.0-RELEASE-p4 + NFS + ZFS =?UTF-8?Q?=3D=20issues=2E=2E=2E=20=3A/?= =?UTF-8?Q?=20=28probably=20a=20memory=20leak=29?= Message-ID: <7791bd21138de89e5c47cca5513d0757@oav.net>
next in thread | raw e-mail | index | archive | help
Hello there,
I try to make a FreeBSD ZFS NFS server with the following configuration
:
- Supermicro X7DB8
- 2 L5420
- 12G RAM
- 8 x 300G SCSI drives using the 2 U320 scsi channel.
- lagg (LACP) of 2 e1000
- zpool :
# zpool status
pool: vol
state: ONLINE
scan: scrub repaired 0 in 2h11m with 0 errors on Thu Nov 1 07:42:05
2012
config:
NAME STATE READ WRITE CKSUM
vol ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
da0 ONLINE 0 0 0
da4 ONLINE 0 0 0
raidz1-1 ONLINE 0 0 0
da1 ONLINE 0 0 0
da5 ONLINE 0 0 0
raidz1-2 ONLINE 0 0 0
da2 ONLINE 0 0 0
da6 ONLINE 0 0 0
raidz1-3 ONLINE 0 0 0
da3 ONLINE 0 0 0
da7 ONLINE 0 0 0
I have tried this patch : http://people.freebsd.org/~namei-leak.patch
And issue still here.
Also tried to move from newnfs to oldnfs server and client, same issue.
Issue is comming when copying from a netapp to this machine a 70G data
with bunch of big files (eg 600M ~ 8G) and web files.
All NFS clients use NFSv3 and UDP.
Tried :
nfs client netapp and freebsd on the same machine : 1srt rsync from
netapp to freebsd ok : no issue, second rsync : memory leak, swap is
used (about less than 100Mb, but this trigger the bug.
nfs client of the netapp is the "filer" machine, and the destination is
local zfs spool. The rsync to make up to date data trigger as well the
issue.
/boot/loader.conf
zfs_load="YES"
if_lagg_load="YES"
autoboot_delay="10" # Delay in seconds before autobooting,
start
# http://icesquare.com/wordpress/how-to-improve-zfs-performance/
#I have 8G of Ram
#vfs.zfs.prefetch_disable=0
#If Ram = 4GB, set the value to 512M
#If Ram = 8GB, set the value to 1024M
#vfs.zfs.arc_min="1024M"
#Ram x 0.5 - 512 MB
#vfs.zfs.arc_max="3584M"
#Ram x 2
vm.kmem_size_max="24G"
#Ram x 1.5
#vm.kmem_size="12G"
/etc/sysctl.conf
# $FreeBSD: release/9.0.0/etc/sysctl.conf 112200 2003-03-13 18:43:50Z
mux $
#
# This file is read when going to multi-user and its contents piped
thru
# ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for
details.
#
# Uncomment this to prevent users from seeing information about
processes that
# are being run under another UID.
#security.bsd.see_other_uids=0
kern.maxfiles=64000
vfs.nfsd.maxthreads=16
vfs.zfs.prefetch_disable=1
The latter (vfs.zfs.prefetch_disable=1) make the bug a bit strange.
Inactive memory keeps growing, but active and free is still very low
during the copy. Then machine use all of his memory, and NFS server not
responsive.
/etc/rc.conf
hostname="filernew"
keymap="fr.iso.acc.kbd"
cloned_interfaces="lagg0"
ifconfig_em0="up"
ifconfig_em1="up"
ifconfig_lagg0="inet 172.31.x.x netmask 255.255.0.0 laggproto lacp
laggport em0 laggport em1"
defaultrouter="172.31.x.x"
sshd_enable="YES"
ntpd_enable="YES"
powerd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"
zfs_enable="YES"
puppet_enable="YES"
# NFS
rpcbind_enable="YES"
nfs_server_enable="YES"
nfs_server_flags="-u -t -n 4"
mountd_flags="-r"
mountd_enable="YES"
rpc_lockd_enable="YES"
rpc_statd_enable="YES"
# Added by Puppet
bsnmpd_enable="YES"
Kind regards...
Xavier
--
Xavier Beaudouin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7791bd21138de89e5c47cca5513d0757>