From owner-cvs-src@FreeBSD.ORG Thu Sep 8 19:54:38 2005 Return-Path: X-Original-To: cvs-src@freebsd.org Delivered-To: cvs-src@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 49C0216A41F; Thu, 8 Sep 2005 19:54:38 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1.pacific.net.au [61.8.0.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 61FE943D49; Thu, 8 Sep 2005 19:54:37 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.0.87]) by mailout1.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id j88JsLMV027747; Fri, 9 Sep 2005 05:54:21 +1000 Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id j88JsDqM028846; Fri, 9 Sep 2005 05:54:19 +1000 Date: Fri, 9 Sep 2005 05:54:13 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Dmitry Pryanishnikov In-Reply-To: <20050908112746.K43691@atlantis.atlantis.dp.ua> Message-ID: <20050909050838.O23800@delplex.bde.org> References: <20050908094705.R19771@atlantis.atlantis.dp.ua> <20050908024022.G28140@odysseus.silby.com> <20050908112746.K43691@atlantis.atlantis.dp.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: cvs-src@freebsd.org, Mike Silbersack , src-committers@freebsd.org, cvs-all@freebsd.org Subject: Re: cvs commit: src/sys/fs/msdosfs msdosfs_denode.c X-BeenThere: cvs-src@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Sep 2005 19:54:38 -0000 On Thu, 8 Sep 2005, Dmitry Pryanishnikov wrote: > On Thu, 8 Sep 2005, Mike Silbersack wrote: >>> entries begin at byte offsets from the start of the media with identical >>> low-order 32 bits; e.g., 64-bit offsets >>> >>> 0x0000000000001000 and >>> 0x0000000100001000 >> >> Hm, maybe it wouldn't be too difficult to create, then. There is an option >> to have compressed filesystems, so if one wrote a huge filesystem with >> files that all contained zeros, perhaps it would compress well enough. There is an option to create sparse files, and a driver to map files to devices. These can easily be used to create multi-terabyte file systems using only a few KB of physical disk before newfs[_msdosfs] and only a few MB of physical disk after newfs*. Use a maximal blocksize to minimize metadata. I think someone fixed the overflow bug at 4G sectors in md(4), so it is now easy to create multi-petabyte file systems. msdosfs with FAT32 seems to be limited to a measly 8TB (2^28 FAT entries with a cluster size of 32KB; the FAT size for 2^28 entries is 4GB, and FreeBSD would panic trying to malloc() 512MB for the FAT bitmap). >> If you just started creating a lot of equally sized files containing zero >> as their content, maybe it could be done via a script. Yeah, you could >> just call truncate in some sort of shell script loop until you have enough >> files, then go back and try reading file "000001", and that should cause >> the panic, right? > > Our task is slightly different: not our files should start at magic offset, > but their _directory entries_. I think this task is achievable by creating > new FAT32 filesystem, then (in strict order) a directory, a large (approx. > 4Gb) file in it, a second directory, a file in it, then lookup first > file. In order to get a panic whe just have to tune size of the large file. > If I have enough time I'll try to prepare such a regression test. msdosfs has pessimized block allocation to maximize fragmentation. This involves random allocation of the first block in every file (including directories). The randomness makes the chance of a collison very small and not much affected by the presence of large files. However, it is easy to change the allocation policy to maximize collisions. I use the following to test allocation policies: %%% Index: msdosfs_fat.c =================================================================== RCS file: /home/ncvs/src/sys/fs/msdosfs/msdosfs_fat.c,v retrieving revision 1.35 diff -u -2 -r1.35 msdosfs_fat.c --- msdosfs_fat.c 29 Dec 2003 11:59:05 -0000 1.35 +++ msdosfs_fat.c 26 Apr 2004 05:03:55 -0000 @@ -68,4 +68,6 @@ #include +static int fat_allocpolicy = 1; + /* * Fat cache stats. @@ -796,9 +808,30 @@ len = 0; - /* - * Start at a (pseudo) random place to maximize cluster runs - * under multiple writers. - */ - newst = random() % (pmp->pm_maxcluster + 1); + switch (fat_allocpolicy) { + case 0: + newst = start; + break; + case 1: + newst = pmp->pm_nxtfree; + break; + case 5: + newst = (start == 0 ? pmp->pm_nxtfree : start); + break; + case 2: + /* FALLTHROUGH */ + case 3: + if (start != 0) { + newst = fat_allocpolicy == 2 ? start : pmp->pm_nxtfree; + break; + } + /* FALLTHROUGH */ + default: + /* + * Start at a (pseudo) random place to maximize cluster runs + * under multiple writers. + */ + newst = random() % (pmp->pm_maxcluster + 1); + } + foundl = 0; %%% Bruce