From owner-freebsd-performance@FreeBSD.ORG Sat Jan 31 00:08:02 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 61C4010656E4; Sat, 31 Jan 2009 00:08:02 +0000 (UTC) (envelope-from brent@servuhome.net) Received: from mail-bw0-f16.google.com (mail-bw0-f16.google.com [209.85.218.16]) by mx1.freebsd.org (Postfix) with ESMTP id 8FBA38FC18; Sat, 31 Jan 2009 00:08:01 +0000 (UTC) (envelope-from brent@servuhome.net) Received: by bwz9 with SMTP id 9so139942bwz.19 for ; Fri, 30 Jan 2009 16:08:00 -0800 (PST) MIME-Version: 1.0 Received: by 10.180.232.16 with SMTP id e16mr621092bkh.77.1233360480034; Fri, 30 Jan 2009 16:08:00 -0800 (PST) In-Reply-To: <20090129230247.GF4375@elvis.mu.org> References: <20090129230247.GF4375@elvis.mu.org> Date: Fri, 30 Jan 2009 16:07:59 -0800 Message-ID: From: Brent Jones To: Alfred Perlstein Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org, freebsd-stable@freebsd.org Subject: Re: NFS writes calling FSYNC and ASYNC not consistent X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Jan 2009 00:08:03 -0000 On Thu, Jan 29, 2009 at 3:02 PM, Alfred Perlstein wrote: > Apologies for being terse, in a hurry here. > > 1) -o async doesn't work with NFS, don't use that. > 2) how big are the text versus binary files? I tested with a 6MB text file, and a 2GB binary file. Text file would go ~1MB/sec, issuing FSYNCs for every block write size (32KB default) One thing I found that another FreeBSD user discovered exactly explains my situation: In bin/cp/utils.c (source) there is a check, if the file is less than 8MB or so, it uses mmap, if the file is larger, it will use write() I modified the source and recompiled to -never- use mmap, only to use write(), and my performance increased about 100 fold (from 1MB/sec over NFS, to over 100MB/sec). Changed line 143: original: fs->st_size <= 8 * 1048576) { New: fs->st_size <= 8 * 8) { It will use mmap still if the file is larger than 64bytes (if it uses bytes there, pretty sure it does). But is much faster for files ~1KB to 8MB now. Regards > 3) how are you copying them over nfs? > > I suspect, (could be wrong of course) that the ascii files > are a lot smaller than the binary files, so what's happening > is that for binary files, the client is issuing write-behind > async, however for ascii files its issuing the writes at > close time which will force the sync flag. > > -Alfred > -- Brent Jones brent@servuhome.net