From owner-freebsd-fs@FreeBSD.ORG Thu Nov 8 10:03:00 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B8579432 for ; Thu, 8 Nov 2012 10:03:00 +0000 (UTC) (envelope-from kiwi@oav.net) Received: from mail02.oav.net (mail02.oav.net [IPv6:2001:67c:ec:100::25:2]) by mx1.freebsd.org (Postfix) with ESMTP id 3D43A8FC16 for ; Thu, 8 Nov 2012 10:03:00 +0000 (UTC) Received: from amavis1.local.oav.net (amavis1.local.oav.net [IPv6:2001:67c:ec:100::25:41]) by mail02.oav.net (Postfix) with ESMTP id 440FD61C3F for ; Thu, 8 Nov 2012 11:02:52 +0100 (CET) (envelope-from kiwi@oav.net) X-Virus-Scanned: Amavisd-new at amavis1.local.oav.net.local.oav.net Received: from mail02.oav.net ([172.31.1.2]) by amavis1.local.oav.net (amavis1.local.oav.net [172.31.1.41]) (amavisd-new, port 10026) with LMTP id U1wteWdl0aJT for ; Thu, 8 Nov 2012 11:02:45 +0100 (CET) Received: from nat-out.openvisp.net (nat-out.openvisp.net [193.218.105.29]) (Authenticated sender: kiwi@oav.net) by mail02.oav.net (Postfix) with ESMTPA id A77CC61C3E for ; Thu, 8 Nov 2012 11:02:45 +0100 (CET) (envelope-from kiwi@oav.net) Received: from [2001:67c:2219:7::86:62] by webmail.openvisp.net with HTTP (HTTP/1.1 POST); Thu, 08 Nov 2012 11:02:45 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 08 Nov 2012 11:02:45 +0100 From: Xavier Beaudouin To: Subject: 9.0-RELEASE-p4 + NFS + ZFS =?UTF-8?Q?=3D=20issues=2E=2E=2E=20=3A/?= =?UTF-8?Q?=20=28probably=20a=20memory=20leak=29?= Organization: Association Kazar - http://kazar.net/ Message-ID: <7791bd21138de89e5c47cca5513d0757@oav.net> X-Sender: kiwi@oav.net User-Agent: RoundCube Webmail/0.8.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2012 10:03:00 -0000 Hello there, I try to make a FreeBSD ZFS NFS server with the following configuration : - Supermicro X7DB8 - 2 L5420 - 12G RAM - 8 x 300G SCSI drives using the 2 U320 scsi channel. - lagg (LACP) of 2 e1000 - zpool : # zpool status pool: vol state: ONLINE scan: scrub repaired 0 in 2h11m with 0 errors on Thu Nov 1 07:42:05 2012 config: NAME STATE READ WRITE CKSUM vol ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da0 ONLINE 0 0 0 da4 ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 da1 ONLINE 0 0 0 da5 ONLINE 0 0 0 raidz1-2 ONLINE 0 0 0 da2 ONLINE 0 0 0 da6 ONLINE 0 0 0 raidz1-3 ONLINE 0 0 0 da3 ONLINE 0 0 0 da7 ONLINE 0 0 0 I have tried this patch : http://people.freebsd.org/~namei-leak.patch And issue still here. Also tried to move from newnfs to oldnfs server and client, same issue. Issue is comming when copying from a netapp to this machine a 70G data with bunch of big files (eg 600M ~ 8G) and web files. All NFS clients use NFSv3 and UDP. Tried : nfs client netapp and freebsd on the same machine : 1srt rsync from netapp to freebsd ok : no issue, second rsync : memory leak, swap is used (about less than 100Mb, but this trigger the bug. nfs client of the netapp is the "filer" machine, and the destination is local zfs spool. The rsync to make up to date data trigger as well the issue. /boot/loader.conf zfs_load="YES" if_lagg_load="YES" autoboot_delay="10" # Delay in seconds before autobooting, start # http://icesquare.com/wordpress/how-to-improve-zfs-performance/ #I have 8G of Ram #vfs.zfs.prefetch_disable=0 #If Ram = 4GB, set the value to 512M #If Ram = 8GB, set the value to 1024M #vfs.zfs.arc_min="1024M" #Ram x 0.5 - 512 MB #vfs.zfs.arc_max="3584M" #Ram x 2 vm.kmem_size_max="24G" #Ram x 1.5 #vm.kmem_size="12G" /etc/sysctl.conf # $FreeBSD: release/9.0.0/etc/sysctl.conf 112200 2003-03-13 18:43:50Z mux $ # # This file is read when going to multi-user and its contents piped thru # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. # # Uncomment this to prevent users from seeing information about processes that # are being run under another UID. #security.bsd.see_other_uids=0 kern.maxfiles=64000 vfs.nfsd.maxthreads=16 vfs.zfs.prefetch_disable=1 The latter (vfs.zfs.prefetch_disable=1) make the bug a bit strange. Inactive memory keeps growing, but active and free is still very low during the copy. Then machine use all of his memory, and NFS server not responsive. /etc/rc.conf hostname="filernew" keymap="fr.iso.acc.kbd" cloned_interfaces="lagg0" ifconfig_em0="up" ifconfig_em1="up" ifconfig_lagg0="inet 172.31.x.x netmask 255.255.0.0 laggproto lacp laggport em0 laggport em1" defaultrouter="172.31.x.x" sshd_enable="YES" ntpd_enable="YES" powerd_enable="YES" # Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable dumpdev="NO" zfs_enable="YES" puppet_enable="YES" # NFS rpcbind_enable="YES" nfs_server_enable="YES" nfs_server_flags="-u -t -n 4" mountd_flags="-r" mountd_enable="YES" rpc_lockd_enable="YES" rpc_statd_enable="YES" # Added by Puppet bsnmpd_enable="YES" Kind regards... Xavier -- Xavier Beaudouin