From owner-freebsd-bugs@FreeBSD.ORG Fri Oct 21 16:40:16 2005 Return-Path: X-Original-To: freebsd-bugs@hub.freebsd.org Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D1BF716A420 for ; Fri, 21 Oct 2005 16:40:16 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB0D743D49 for ; Fri, 21 Oct 2005 16:40:15 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.3/8.13.3) with ESMTP id j9LGeFt1048426 for ; Fri, 21 Oct 2005 16:40:15 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.3/8.13.1/Submit) id j9LGeFq7048423; Fri, 21 Oct 2005 16:40:15 GMT (envelope-from gnats) Resent-Date: Fri, 21 Oct 2005 16:40:15 GMT Resent-Message-Id: <200510211640.j9LGeFq7048423@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Andre Albsmeier Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9AAC916A41F for ; Fri, 21 Oct 2005 16:32:52 +0000 (GMT) (envelope-from Andre.Albsmeier@siemens.com) Received: from thoth.sbs.de (thoth.sbs.de [192.35.17.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1C69343D5F for ; Fri, 21 Oct 2005 16:32:46 +0000 (GMT) (envelope-from Andre.Albsmeier@siemens.com) Received: from mail3.siemens.de (localhost [127.0.0.1]) by thoth.sbs.de (8.12.6/8.12.6) with ESMTP id j9LGWjOH028281 for ; Fri, 21 Oct 2005 18:32:45 +0200 Received: from ims.mchp.siemens.de (ims.mchp.siemens.de [139.25.31.39]) by mail3.siemens.de (8.12.6/8.12.6) with ESMTP id j9LGWieL031781 for ; Fri, 21 Oct 2005 18:32:44 +0200 Received: from mail-ct.mchp.siemens.de (mail-ct.mchp.siemens.de [139.25.31.51]) by ims.mchp.siemens.de with ESMTP id j9LGWiJQ005266 for ; Fri, 21 Oct 2005 18:32:44 +0200 (MEST) Received: from curry.mchp.siemens.de (curry [139.25.40.130]) by mail-ct.mchp.siemens.de (8.12.11/8.12.11) with ESMTP id j9LGWioS012900 for ; Fri, 21 Oct 2005 18:32:44 +0200 (MEST) Received: (from localhost) by curry.mchp.siemens.de (8.13.4/8.13.4) id j9LGWhc9063475 for FreeBSD-gnats-submit@freebsd.org; Fri, 21 Oct 2005 18:32:43 +0200 (CEST) Message-Id: <200510211632.j9LGWhA3013446@curry.mchp.siemens.de> Date: Fri, 21 Oct 2005 18:32:43 +0200 (CEST) From: Andre Albsmeier To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: Subject: bin/87792: very bad performance of cp(1) via NFS, possibly mmap() problem X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2005 16:40:17 -0000 >Number: 87792 >Category: bin >Synopsis: very bad performance of cp(1) via NFS, possibly mmap() problem >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Oct 21 16:40:15 GMT 2005 >Closed-Date: >Last-Modified: >Originator: Andre Albsmeier >Release: FreeBSD 5.4-STABLE i386 >Organization: >Environment: Two machines, a local one which copies files to a remote machine via NFS using cp. Both machines use FreeBSD 5.4-STABLE. >Description: cp uses mmap() to copy files < 8MB. When files are copied from the local host to a remote box via NFS data is transferred very slowly. For bigger files, cp uses read() and write() alternately which performs very well. >How-To-Repeat: I have two files, one is 6MB in size and the other one is 12MB. I copy them from the local box to an NFS mounted directory named /scratch: andre@bali:/tmp>time cp 12mb /scratch cp 12mb /scratch 0.00s user 0.12s system 10% cpu 1.142 total andre@bali:/tmp>time cp 6mb /scratch cp 6mb /scratch 0.00s user 0.13s system 0% cpu 14.274 total The 12MB filecopy performs quite well as I would expect it from a 100MBit network. The 6mb filecopy I won't comment on :-). (This is an extreme case, other client-server combinations are faster. The "fastest" one I found was 4 seconds which is still bad for a 6MB file). In the mmap() case, the whole source file is mmap'ed and one write() call is used which tries to write the whole file all at once. I have modified copy_file() in utils.c in a way that instead of one "big" write(), several small ones are being made (this is no fix, just for demonstration): --- utils.c.ORI Thu Aug 19 07:04:27 2004 +++ utils.c Fri Oct 21 18:19:59 2005 @@ -140,7 +140,11 @@ wtotal = 0; for (bufp = p, wresid = fs->st_size; ; bufp += wcount, wresid -= (size_t)wcount) { +#if 0 wcount = write(to_fd, bufp, wresid); +#else + wcount = write(to_fd, bufp, wresid > 65536 ? 65536 : wresid ); +#endif wtotal += wcount; if (info) { info = 0; 65536 is just a value for playing; almost everything below 1M and not ridiculous small will do it: andre@bali:/tmp>time cp 6mb /scratch cp 6mb /scratch 0.00s user 0.05s system 8% cpu 0.618 total >Fix: I have no idea. FreeBSD-4 uses the same mmap() thingy in utils.c and here performance is ok so I assume the problem is somewhere else than in the cp sources. >Release-Note: >Audit-Trail: >Unformatted: