From owner-freebsd-hackers Tue Aug 4 19:15:53 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id TAA14868 for freebsd-hackers-outgoing; Tue, 4 Aug 1998 19:15:53 -0700 (PDT) (envelope-from owner-freebsd-hackers@FreeBSD.ORG) Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id TAA14844 for ; Tue, 4 Aug 1998 19:15:32 -0700 (PDT) (envelope-from tlambert@usr02.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.8.8/8.8.8) id TAA04763; Tue, 4 Aug 1998 19:15:16 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp03.primenet.com, id smtpd004719; Tue Aug 4 19:15:08 1998 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id TAA01844; Tue, 4 Aug 1998 19:15:04 -0700 (MST) From: Terry Lambert Message-Id: <199808050215.TAA01844@usr02.primenet.com> Subject: Re: couple of questions To: maex-freebsd-hackers@Space.Net (Markus Stumpf) Date: Wed, 5 Aug 1998 02:15:04 +0000 (GMT) Cc: brett@peloton.physics.montana.edu, freebsd-hackers@FreeBSD.ORG In-Reply-To: <19980805004942.L12619@space.net> from "Markus Stumpf" at Aug 5, 98 00:49:42 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > - maximum file/partition size: > > > > - I've seen files (from an ls -l emailed to me) of ~ 7 GB, but > > I'm not sure what the absolute maximum file size is. Is there > > a hard limit? > > Hmmm ... this is from man 2 intro (2.2.6-RELEASE) > > > 27 EFBIG File too large. The size of a file exceeded the maximum (about > > 2.1E9 bytes). > > This is 2 GB, right? But I've heard NetBSD has bigger files. > > \Maex > > N.B. This 2 GB always hits me with backups of large partitions that don't > compress well. Still using amanda-2.3.0, however, amanda-2.4.0 is able > to handle this with artificial splits. The size limit is 549755813888 -- that's (u_quad_t) 1 << 39, or 512 Gig. The limit is a result of a 32 bit block number limitation in the file /usr/src/sbin/newfs.c.\, which in turn is because the on disk fs_size and fs_dsize are stored as int32_t. The size limit is also the size limit for an FS: /* XXX - hack to prevent overflow of a 32bit block number */ sblock.fs_maxfilesize = MIN(sblock.fs_maxfilesize, (u_quad_t) 1 << 39); The size may be further reduced if there are insufficient blocks available to support triple indirect blocks, or double indirect blocks. This number is actually incorrect; 2^31 (1 bit for sign; indirect blocks are internally implemented using negative block offsets to indicate they reference an indirect block) times 2^12 (blocks in this case are 4096 bytes, the minimum FS block size). Assuming this change, and a number of small patches to prevent 32 bit field overflow in the physical block calculation, this puts the number at 8TB for a 4k block size and 16TB for an 8k block size (2^33 and 2^34, respectively). The patches have been available for some time; I believe they were first posted a year or so ago by someone building a huge CCD array at UC Berkeley (Satoshi Asami, maybe?). I believe they haven't been incorporated because > 32 bit calculations for physical (not FS) blocks would slow things down (even then, the 39 is 2^31 * 2^8; physical blocks are 2^9, which should put the FS smack on the 1TB mark). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message