From owner-freebsd-questions@FreeBSD.ORG Thu Jan 31 22:17:17 2013 Return-Path: <owner-freebsd-questions@FreeBSD.ORG> Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6759A4CF for <freebsd-questions@freebsd.org>; Thu, 31 Jan 2013 22:17:17 +0000 (UTC) (envelope-from Albert.Shih@obspm.fr) Received: from smtp-int-m.obspm.fr (smtp-int-m.obspm.fr [145.238.187.15]) by mx1.freebsd.org (Postfix) with ESMTP id E173475F for <freebsd-questions@freebsd.org>; Thu, 31 Jan 2013 22:17:16 +0000 (UTC) Received: from pcjas.obspm.fr (pcjas.obspm.fr [145.238.184.233]) by smtp-int-m.obspm.fr (8.14.3/8.14.3/SIO Observatoire de Paris - 07/2009) with ESMTP id r0VMGR84028638 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 31 Jan 2013 23:16:28 +0100 Date: Thu, 31 Jan 2013 23:16:27 +0100 From: Albert Shih <Albert.Shih@obspm.fr> To: zfs-discuss@opensolaris.org, freebsd-questions@freebsd.org Subject: zfs + NFS + FreeBSD with performance prob Message-ID: <20130131221627.GA91823@pcjas.obspm.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.5.21 (2010-09-15) X-Miltered: at smtp-int-m.obspm.fr with ID 510AED3B.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 510AED3B.000/145.238.184.233/pcjas.obspm.fr/pcjas.obspm.fr/<Albert.Shih@obspm.fr> X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions <freebsd-questions.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-questions>, <mailto:freebsd-questions-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-questions> List-Post: <mailto:freebsd-questions@freebsd.org> List-Help: <mailto:freebsd-questions-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-questions>, <mailto:freebsd-questions-request@freebsd.org?subject=subscribe> X-List-Received-Date: Thu, 31 Jan 2013 22:17:17 -0000 Hi all, I'm not sure if the problem is with FreeBSD or ZFS or both so I cross-post (I known it's bad). Well I've server running FreeBSD 9.0 with (don't count / on differents disks) zfs pool with 36 disk. The performance is very very good on the server. I've one NFS client running FreeBSD 8.3 and the performance over NFS is very good : For example : Read from the client and write over NFS to ZFS: [root@ .tmp]# time tar xf /tmp/linux-3.7.5.tar real 1m7.244s user 0m0.921s sys 0m8.990s this client is on 1Gbits/s network cable and same network switch as the server. I've a second NFS client running FreeBSD 9.1-Stable, and on this second client the performance is catastrophic. After 1 hour the tar isn't finish. OK this second client is connect with 100Mbit/s and not on the same switch. But well from 2 min --> ~ 90 min ...:-( I've try for this second client to change on the ZFS-NFS server the zfs set sync=disabled and that change nothing. On a third NFS client linux (recent Ubuntu) I got the almost same catastrophic performance. With or without sync=disabled. Those three NFS client use TCP. If I do a classic scp I got normal speed ~9-10 Mbytes/s so the network is not the problem. I try to something like (find with google): net.inet.tcp.sendbuf_max: 2097152 -> 16777216 net.inet.tcp.recvbuf_max: 2097152 -> 16777216 net.inet.tcp.sendspace: 32768 -> 262144 net.inet.tcp.recvspace: 65536 -> 262144 net.inet.tcp.mssdflt: 536 -> 1452 net.inet.udp.recvspace: 42080 -> 65535 net.inet.udp.maxdgram: 9216 -> 65535 net.local.stream.recvspace: 8192 -> 65535 net.local.stream.sendspace: 8192 -> 65535 and that change nothing either. Anyone have any idea ? Regards. JAS -- Albert SHIH DIO bâtiment 15 Observatoire de Paris 5 Place Jules Janssen 92195 Meudon Cedex Téléphone : 01 45 07 76 26/06 86 69 95 71 xmpp: jas@obspm.fr Heure local/Local time: jeu 31 jan 2013 23:04:47 CET