From owner-freebsd-fs@FreeBSD.ORG Sun Feb 17 07:02:04 2013 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A007D368 for ; Sun, 17 Feb 2013 07:02:04 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail12.syd.optusnet.com.au (mail12.syd.optusnet.com.au [211.29.132.193]) by mx1.freebsd.org (Postfix) with ESMTP id 9793EC2F for ; Sun, 17 Feb 2013 07:02:02 +0000 (UTC) Received: from c211-30-173-106.carlnfd1.nsw.optusnet.com.au (c211-30-173-106.carlnfd1.nsw.optusnet.com.au [211.30.173.106]) by mail12.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id r1H71osh019511 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 17 Feb 2013 18:01:52 +1100 Date: Sun, 17 Feb 2013 18:01:50 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Konstantin Belousov Subject: Re: cleaning files beyond EOF In-Reply-To: <20130217055528.GB2522@kib.kiev.ua> Message-ID: <20130217172928.C1900@besplex.bde.org> References: <20130217113031.N9271@besplex.bde.org> <20130217055528.GB2522@kib.kiev.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.0 cv=RbTIkCRv c=1 sm=1 a=xK1pj5J4f3QA:10 a=kj9zAlcOel0A:10 a=PO7r1zJSAAAA:8 a=JzwRw_2MAAAA:8 a=GlckP5_kgdUA:10 a=9bBWqGSFoObTARNqQagA:9 a=CjuIK1q_8ugA:10 a=TEtd8y5WR3g2ypngnwZWYw==:117 Cc: fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Feb 2013 07:02:04 -0000 On Sun, 17 Feb 2013, Konstantin Belousov wrote: > On Sun, Feb 17, 2013 at 11:33:58AM +1100, Bruce Evans wrote: >> I have a (possibly damaged) ffs data block with nonzero data beyond >> EOF. Is anything responsible for clearing this data when the file >> is mmapped()? >> >> At least old versions of gcc mmap() the file and have a bug checking >> for EOF. They read the garbage beyond the end and get confused. > > Does the 'damaged' status of the data block mean that it contain the > garbage after EOF on disk ? Yes, it's at most software damage. I used a broken version of vfs_bio_clrbuf() for a long time and it probably left some unusual blocks. This matters suprisingly rarely. I forgot to mention that this is with an old version of FreeBSD, where I changed vfs_bio.c a lot but barely touched vm. > UFS uses a small wrapper around vnode_generic_getpages() as the > VOP_GETPAGES(), the wrapping code can be ignored for the current > purpose. > > vnode_generic_getpages() iterates over the the pages after the bstrategy() > and marks the part of the page after EOF valid and zeroes it, using > vm_page_set_valid_range(). The old version has a large non-wrapper in ffs, and vnode_generic_getpages() uses vm_page_set_validclean(). Maybe the bug is just in the old ffs_getpages(). It seems to do only DEV_BSIZE'ed zeroing stuff. It begins with the same "We have to zero that data" code that forms most of the wrapper in the current version. It normally only returns vnode_pager_generic_getpages() after that if bsize < PAGE_SIZE. However, my version has a variable which I had forgotten about to control this, and the forgotten setting of this variable results in always using vnode_pager_generic_getpages(), as in -current. I probably copied some fixes in -current for this. So the bug can't be just in ffs_getpages(). The "damaged" block is at the end of vfs_default.c. The file size is 25 * PAGE_SIZE + 16. It is in 7 16K blocks, 2 full 2K frags, and 1 frag with 16 bytes valid in it. I have another problem that is apparently with vnode_pager_generic_getpages() and now affects -current from about a year ago in an identical way with the old version: mmap() is very slow in msdosfs. cmp uses mmap() too much, and reading files sequentially using mmap() is 3.4 times slower than reading them using read() on my DVD media/drive. The i/o seems to be correctly clustered for both. with average transaction sizes over 50K but tps much lower for mmap(). Similarly on a (faster) hard disk except the slowness is not as noticeable (drive buffering might hide it completely). However, for ffs files on the hard disk, mmap() is as fast as read(). Bruce