From owner-freebsd-fs@FreeBSD.ORG Mon Oct 31 18:01:41 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9488A16A41F for ; Mon, 31 Oct 2005 18:01:41 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2.pacific.net.au [61.8.0.115]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9BA2643D46 for ; Mon, 31 Oct 2005 18:01:40 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.0.86]) by mailout2.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id j9VI1Vsl004556; Tue, 1 Nov 2005 05:01:31 +1100 Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy1.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id j9VI1RSl013227; Tue, 1 Nov 2005 05:01:28 +1100 Date: Tue, 1 Nov 2005 05:01:27 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Ivan Voras In-Reply-To: <20051031160354.G67271@geri.cc.fer.hr> Message-ID: <20051101042444.K40281@delplex.bde.org> References: <20051030183340.B19470@geri.cc.fer.hr> <46D894BD-16E0-4CBA-B40A-EEBAAC2547D2@classicalguitar.net> <20051031191139.J38757@delplex.bde.org> <20051031160354.G67271@geri.cc.fer.hr> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: ext2 large_file X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Oct 2005 18:01:41 -0000 On Mon, 31 Oct 2005, Ivan Voras wrote: > On Mon, 31 Oct 2005, Bruce Evans wrote: > >> tjr implemented it in FreeBSD almost 2 years ago: > > It doesn't, or something other is wrong. This happens on a freshly created > ext2fs: > >> dd if=/dev/zero of=big_file bs=1m count=2500 > dd: big_file: File too large > 2048+0 records in > 2047+0 records out > 2146435072 bytes transferred in 100.387067 secs (21381590 bytes/sec) > >> l > total 2098180 > -rw-r--r-- 1 ivoras wheel 2146435072 Oct 31 16:09 big_file > > > (btw: the transfer rate is also somewhat bad: 50% of CPU time was taken, ~25% > in sys, ~25% in interrupts. This is UP machine. I think this is because ext2 > support seems to divide I/O requests into 4KB chunks :( ) Ext2fs is less efficient than ffs, but in my experience only by a factor of 2. I see it having 10-15% overhead but this is with a transfer rate of only 8MB/sec. > Here's what dumpe2fs says: > > Filesystem volume name: walker_ext2 > Last mounted on: > Filesystem UUID: 9a920c62-05f6-4631-8c90-30af2c63d5df > Filesystem magic number: 0xEF53 > Filesystem revision #: 1 (dynamic) > Filesystem features: filetype sparse_super I think the rev is high enough, but the feature list would say "large_file" if the file system had that feature. >> Compatibility flags shouldn't be forced on IMO. Linux does it for this >> flag, but this is a bug IMO. It breaks subsequent remounting r/w on >> old or other kernels that don't support large files. > > So, how to set the flag? man pages for tune2fs and mke2fs don't mention the > large_file option. Is there some other utility that does this? Apparently tune2fs doesn't do it because it expects the kernel to. e2fsck seems to have some support for it, at least in e2fsprogs-1.27, but I think that support is limited to setting the flag if there is a large file an maybe unsetting the flag if there isn't. >> msdosfs is physically incapable of supporting large files. Its maximum >> file size is the constant 0xffffffff. > > Yes, I should have said "larger" files :) Current ext2 support in FreeBSD is > limited to 2GB files, while 4GB is enough for me for now. > >> NetBSD got it 9 months ago, only a year after FreeBSD. It refuses to >> create files larger than 2G-1 if the ext2fs rev number is old, and says >> in a comment that Linux silently upgrades the rev number. It silently >> clobbers the compat flag like Linux. Someone has an off-by-power-of-2 >> error -- the corresponding limit in FreeBSD is 4G-1. > > I just tried it - the limit is 2GB on FreeBSD. Yes. I misread the constant. > So, it seems that it boils down to that FreeBSD's ext2 support still cannot > handle large files? Unless the file system already has or had a large file. Possible workarounds: (1) Boot Linux and create a large file. Hopefully e2fsck only sets the flag so you only have to do this once. (2) Edit the limit in FreeBSD (grep for maxfilesize), create a large file, run e2fsck to set the flag, and change the limit back for safety. (3) Modify tune2fs to support setting the flag. (4) Use a disk editor or fs debugger to set the flag. debugfs in e2fsprogs should be able to do this. However, it doesn't seem to be in the FreeBSD package and my old version of it seems to have been broken by removal of block devices (most e2fs utilities want block devices but some barely work without them). Bruce