From owner-freebsd-questions@FreeBSD.ORG Sat Mar 16 18:49:41 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0CDEFED2 for ; Sat, 16 Mar 2013 18:49:41 +0000 (UTC) (envelope-from tundra@tundraware.com) Received: from ozzie.tundraware.com (ozzie.tundraware.com [75.145.138.73]) by mx1.freebsd.org (Postfix) with ESMTP id B5BBFE32 for ; Sat, 16 Mar 2013 18:49:40 +0000 (UTC) Received: from [192.168.0.2] (viper.tundraware.com [192.168.0.2]) (authenticated bits=0) by ozzie.tundraware.com (8.14.6/8.14.6) with ESMTP id r2GInRaq002582 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Sat, 16 Mar 2013 13:49:27 -0500 (CDT) (envelope-from tundra@tundraware.com) Message-ID: <5144BEB7.3090906@tundraware.com> Date: Sat, 16 Mar 2013 13:49:27 -0500 From: Tim Daneliuk User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130308 Thunderbird/17.0.4 MIME-Version: 1.0 To: FreeBSD Mailing List Subject: NFS Performance: Weirder And Weirder Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (ozzie.tundraware.com [192.168.0.1]); Sat, 16 Mar 2013 13:49:27 -0500 (CDT) X-TundraWare-MailScanner-Information: Please contact the ISP for more information X-TundraWare-MailScanner-ID: r2GInRaq002582 X-TundraWare-MailScanner: Found to be clean X-TundraWare-MailScanner-From: tundra@tundraware.com X-Spam-Status: No X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Mar 2013 18:49:41 -0000 This is really weird. A FreeBSD 9.1 system mounts the following: /dev/ad4s1a 989M 625M 285M 69% / devfs 1.0k 1.0k 0B 100% /dev /dev/ad4s1d 7.8G 1G 6.1G 14% /var /dev/ad4s1e 48G 9.4G 35G 21% /usr /dev/ad4s1f 390G 127G 231G 35% /usr1 /dev/ad6s1d 902G 710G 120G 86% /usr1/BKU /usr1/something (under ad4s1f) and /usr1/BKU (all of ad6s1d) are exported for NFS mounting on the LAN. I have tested the speeds of these two drives locally doing a 'dd if=/dev/zero ....'. Their speeds are quite comparable - around 55-60 MB/s so the problem below is not an artifact of a slow drive. The two mounts are imported like this on a Linux Mint 12 machine: machine:/usr1/BKU /BKU nfs rw,soft,intr 0 0 machine:/usr1/shared /shared nfs rw,soft,intr 0 0 Problem: When I write files from the LM12 machines to /BKU the writes are 1/10 the speed of when writing to /shared. Reads are fine in both cases, at near native disk speeds being reported. Someone here suggested I get rid of any symlinks in the mount and I did that to no avail. Incidentally, the only reason I just noticed this is that I upgraded the NIC on the FreeBSD machine and the switch into which it connects to 1000Base because the LM12 machine had a built in 1000Base NIC. I also changed the cables on both machines to ensure they were not the problem. Prior to this, I was bandwidth constrained by the 100Base so I never saw NFS performance as an issue. When I upgraded, I expected faster transfers and when I didn't get them, I started this whole investigation. So ... I'm stumped: - It's not the drive or SATA ports because both drives show comparable performance. - It's not the cables because I can get great throughput on one of the NFS mountpoints. - It's neither NIC for the same reason. Does anyone: A) Have a clue what might be doing this B) Have a suggestion how to track down the problem Thanks, -- ---------------------------------------------------------------------------- Tim Daneliuk tundra@tundraware.com PGP Key: http://www.tundraware.com/PGP/