From owner-freebsd-questions@FreeBSD.ORG Thu Aug 15 19:26:25 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id F0E92D67 for ; Thu, 15 Aug 2013 19:26:25 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from st11p05mm-asmtp001.mac.com (st11p05mm-asmtp004.mac.com [17.172.108.249]) by mx1.freebsd.org (Postfix) with ESMTP id C6BE72498 for ; Thu, 15 Aug 2013 19:26:25 +0000 (UTC) Received: from cswiger1.apple.com (unknown [17.209.8.53]) by st11p05mm-asmtp001.mac.com (Oracle Communications Messaging Server 7u4-27.07(7.0.4.27.6) 64bit (built Jun 21 2013)) with ESMTPSA id <0MRL00GMQ4JFIE20@st11p05mm-asmtp001.mac.com> for freebsd-questions@freebsd.org; Thu, 15 Aug 2013 18:26:04 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8794,1.0.431,0.0.0000 definitions=2013-08-15_08:2013-08-15,2013-08-15,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1305240000 definitions=main-1308150123 Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Subject: Re: copying milllions of small files and millions of dirs From: Charles Swiger In-reply-to: <7E7AEB5A-7102-424E-8B1E-A33E0A2C8B2C@gmail.com> Date: Thu, 15 Aug 2013 11:26:01 -0700 Content-transfer-encoding: quoted-printable Message-id: References: <7E7AEB5A-7102-424E-8B1E-A33E0A2C8B2C@gmail.com> To: aurfalien X-Mailer: Apple Mail (2.1508) Cc: FreeBSD Questions X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Aug 2013 19:26:26 -0000 On Aug 15, 2013, at 11:13 AM, aurfalien wrote: > Is there a faster way to copy files over NFS? Probably. > Currently breaking up a simple rsync over 7 or so scripts which copies = 22 dirs having ~500,000 dirs or files each. There's a maximum useful concurrency which depends on how many disk = spindles and what flavor of RAID is in use; exceeding it will result in = thrashing the disks and heavily reducing throughput due to competing I/O = requests. Try measuring aggregate performance when running fewer rsyncs = at once and see whether it improves. Of course, putting half a million files into a single directory level is = also a bad idea, even with dirhash support. You'd do better to break = them up into subdirs containing fewer than ~10K files apiece. > Obviously reading all the meta data is a PITA. Yes. > Doin 10Gb/jumbos but in this case it don't make much of a hoot of a = diff. Yeah, probably not-- you're almost certainly I/O bound, not network = bound. Regards, --=20 -Chuck