From owner-cvs-src@FreeBSD.ORG Tue Jul 10 14:17:50 2007 Return-Path: X-Original-To: cvs-src@freebsd.org Delivered-To: cvs-src@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2E8B416A469; Tue, 10 Jul 2007 14:17:50 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from ns.trinitel.com (186.161.36.72.static.reverse.layeredtech.com [72.36.161.186]) by mx1.freebsd.org (Postfix) with ESMTP id EDC8D13C448; Tue, 10 Jul 2007 14:17:49 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from proton.local (209-163-168-124.static.twtelecom.net [209.163.168.124]) (authenticated bits=0) by ns.trinitel.com (8.14.1/8.14.1) with ESMTP id l6AE7grq021851; Tue, 10 Jul 2007 09:07:42 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <469392AD.3000509@freebsd.org> Date: Tue, 10 Jul 2007 09:07:41 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.4 (Macintosh/20070604) MIME-Version: 1.0 To: Bruce Evans References: <200707101320.l6ADKOjq074451@repoman.freebsd.org> In-Reply-To: <200707101320.l6ADKOjq074451@repoman.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on ns.trinitel.com Cc: cvs-src@freebsd.org, src-committers@freebsd.org, cvs-all@freebsd.org Subject: Re: cvs commit: src/sys/fs/msdosfs msdosfs_fat.c msdosfsmount.h X-BeenThere: cvs-src@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Jul 2007 14:17:50 -0000 Bruce Evans wrote: > bde 2007-07-10 13:20:24 UTC > > FreeBSD src repository > > Modified files: > sys/fs/msdosfs msdosfs_fat.c msdosfsmount.h > Log: > Don't use almost perfectly pessimal cluster allocation. Allocation > of the the first cluster in a file (and, if the allocation cannot be > continued contiguously, for subsequent clusters in a file) was randomized > in an attempt to leave space for contiguous allocation of subsequent > clusters in each file when there are multiple writers. This reduced > internal fragmentation by a few percent, but it increased external > fragmentation by up to a few thousand percent. > > Use simple sequential allocation instead. Actually maintain the fsinfo > sequence index for this. The read and write of this index from/to > disk still have many non-critical bugs, but we now write an index that > has something to do with our allocations instead of being modified > garbage. If there is no fsinfo on the disk, then we maintain the index > internally and don't go near the bugs for writing it. > > Allocating the first free cluster gives a layout that is almost as good > (better in some cases), but takes too much CPU if the FAT is large and > the first free cluster is not near the beginning. > > The effect of this change for untar and tar of a slightly reduced copy > of /usr/src on a new file system was: > > Before (msdosfs 4K-clusters): > untar: 459.57 real untar from cached file (actually a pipe) > tar: 342.50 real tar from uncached tree to /dev/zero > Before (ffs2 soft updates 4K-blocks 4K-frags) > untar: 39.18 real > tar: 29.94 real > Before (ffs2 soft updates 16K-blocks 2K-frags) > untar: 31.35 real > tar: 18.30 real > > After (msdosfs 4K-clusters): > untar 54.83 real > tar 16.18 real > > All of these times can be improved further. > > With multiple concurrent writers or readers (especially readers), the > improvement is smaller, but I couldn't find any case where it is > negative. 342 seconds for tarring up about 342 MB on a ~47MB/S partition > is just hard to unimprove on. (This operation would take about 7.3 > seconds with reasonably localized allocation and perfect read-ahead.) > However, for active file systems, 342 seconds is closer to normal than > the 16+ seconds above or the 11 seconds with other changes (best I've > measured -- won easily by msdosfs!). E.g., my active /usr/src on ffs1 > is quite old and fragmented, so reading to prepare for the above > benchmark takes about 6 times longer than reading back the fresh copies > of it. > Impressive - nice work! Eric