From owner-freebsd-fs@FreeBSD.ORG Fri Sep 24 17:06:59 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CDB7106564A for ; Fri, 24 Sep 2010 17:06:59 +0000 (UTC) (envelope-from josh@tcbug.org) Received: from out2.smtp.messagingengine.com (out2.smtp.messagingengine.com [66.111.4.26]) by mx1.freebsd.org (Postfix) with ESMTP id C3BF58FC0C for ; Fri, 24 Sep 2010 17:06:58 +0000 (UTC) Received: from compute3.internal (compute3.nyi.mail.srv.osa [10.202.2.43]) by gateway1.messagingengine.com (Postfix) with ESMTP id 08BF0295; Fri, 24 Sep 2010 13:06:58 -0400 (EDT) Received: from frontend2.messagingengine.com ([10.202.2.161]) by compute3.internal (MEProxy); Fri, 24 Sep 2010 13:06:58 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=messagingengine.com; h=from:to:subject:date:cc:references:in-reply-to:mime-version:content-type:content-transfer-encoding:message-id; s=smtpout; bh=t8kHC6zgCan6+UKl8phHlvwaDmE=; b=rouSNTB/DNcO7FGkUOzo0a+UmZsJe1Gvr+uSCabA3//OElRV7O9RSxwoYATNYiCvzzBVQqdhWRK4stKXFxEBZahQxLwDvQB0jL6XUXllB2ZDaUNsM5gUBrnNubi8u8q+yp4pg6dQGGPUfUwewRsI9G3lDDclKiTrFzwDcRs+j4w= X-Sasl-enc: MhYX0a80k5o7BOXogD7joENabPLXa8MEjMByVtXdGVzs 1285348016 Received: from tcbug.ixsystems.com (74-34-16-134.dr01.rsmt.mn.frontiernet.net [74.34.16.134]) by mail.messagingengine.com (Postfix) with ESMTPSA id C6C4E5E0F56; Fri, 24 Sep 2010 13:06:56 -0400 (EDT) From: Josh Paetzel To: freebsd-fs@freebsd.org Date: Fri, 24 Sep 2010 12:06:45 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.1-RELEASE; KDE/4.4.5; amd64; ; ) References: <4C9C7C32.3070909@fragfest.com.au> In-Reply-To: <4C9C7C32.3070909@fragfest.com.au> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart15124792.1rSBoADPcJ"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <201009241206.53828.josh@tcbug.org> Cc: Subject: Re: growfs on a large disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2010 17:06:59 -0000 --nextPart15124792.1rSBoADPcJ Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable On Friday, September 24, 2010 05:23:46 am Dean Hamstead wrote: > Evening, >=20 > Im trying to growfs a very large filesystem and not having a lot of luck. >=20 > To give some background, im running the filesystem on a raid array > running on an areca raid card. I have an existing filesytem which was > formatting /dev/da0 and mounting it. (I understand that this is the > preferred alternative to the deprecated 'dangerously dedicated' mode). >=20 > I have expanded the disk array and the volume at the raid card level. > Then rebooted my machine. It was 4x2tb, and is now 5x2tb. Expanding on > areca is non-destructive. >=20 > Here is the current filesystem size when mounted, da0 is the >=20 > ----- >=20 > [root@batou /home/dean]# df -h > Filesystem Size Used Avail Capacity Mounted on > /dev/ad4a 496M 280M 176M 61% / > devfs 1.0K 1.0K 0B 100% /dev > /dev/ad4e 496M 155M 301M 34% /tmp > /dev/ad4f 68G 8.0G 54G 13% /usr > /dev/ad4d 1.9G 529M 1.2G 30% /var > /dev/da1 4.4T 4.4T 28G 99% /volumes/store2 > /dev/da0 5.3T 3.9T 1.4T 74% /volumes/store1 >=20 > ----- >=20 > From dmesg you can see that freebsd knows its now bigger (7.6 rather > than 5.3tb) >=20 > ----- >=20 > da0 at arcmsr0 bus 0 scbus0 target 0 lun 0 > da0: Fixed Direct Access SCSI-5 device > da0: 166.666MB/s transfers (83.333MHz, offset 32, 16bit) > da0: Command Queueing enabled > da0: 7629394MB (15624998912 512 byte sectors: 255H 63S/T 972611C) > da1 at arcmsr0 bus 0 scbus0 target 0 lun 1 > da1: Fixed Direct Access SCSI-5 device > da1: 166.666MB/s transfers (83.333MHz, offset 32, 16bit) > da1: Command Queueing enabled > da1: 4768371MB (9765624320 512 byte sectors: 255H 63S/T 607881C) > SMP: AP CPU #1 Launched! >=20 > ----- >=20 > if we consult diskinfo, it agrees >=20 > ----- >=20 > [root@batou /home/dean]# diskinfo -v /dev/da0 > /dev/da0 > 512 # sectorsize > 7999999442944 # mediasize in bytes (7.3T) > 15624998912 # mediasize in sectors > 0 # stripesize > 0 # stripeoffset > 972611 # Cylinders according to firmware. > 255 # Heads according to firmware. > 63 # Sectors according to firmware. > 0000001379342420 # Disk ident. >=20 >=20 > ----- >=20 > fdisk shows us... >=20 > ----- >=20 > [root@batou /home/dean]# fdisk /dev/da0 > ******* Working on device /dev/da0 ******* > parameters extracted from in-core disklabel are: > cylinders=3D972611 heads=3D255 sectors/track=3D63 (16065 blks/cyl) >=20 > Figures below won't work with BIOS for partitions not in cyl 1 > parameters to be used for BIOS calculations are: > cylinders=3D972611 heads=3D255 sectors/track=3D63 (16065 blks/cyl) >=20 > fdisk: invalid fdisk partition table found > Media sector size is 512 > Warning: BIOS sector numbering starts with sector 1 > Information from DOS bootblock is: > The data for partition 1 is: > sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) > start 63, size 2740094532 (1337936 Meg), flag 80 (active) > beg: cyl 0/ head 1/ sector 1; > end: cyl 578/ head 254/ sector 63 > The data for partition 2 is: > > The data for partition 3 is: > > The data for partition 4 is: > >=20 > ----- >=20 > now if i unmount /dev/da0 and run growfs /dev/da0 i get the following >=20 > ----- >=20 > [root@batou /home/dean]# growfs /dev/da0 > growfs: we are not growing (2929687296->685024256) >=20 >=20 > ----- >=20 > with lots of googling, i have found that there is much talk in various > freebsd forums about the need to use disklabel. however it seems unable > to fathom such a large disk... >=20 > ----- >=20 > [root@batou /home/dean]# disklabel /dev/da0 > disklabel: disks with more than 2^32-1 sectors are not supported >=20 >=20 > ---- >=20 > freebsd version is 8.0 64bit >=20 > ----- >=20 > [root@batou /home/dean]# uname -a > FreeBSD batou.fragfest.net.au 8.0-STABLE FreeBSD 8.0-STABLE #7: Sun Jan > 24 17:13:52 EST 2010 > dean@batou.fragfest.net.au:/usr/obj/usr/src/sys/GENERIC amd64 >=20 > ----- >=20 > hopefully i am doing something wrong, or there is a patch set for growfs? >=20 >=20 >=20 > Dean I've run growfs on raw partitions, it's the easiest way to use growfs, as y= ou=20 don't need to worrk about expanding partitions or slices. Sadly the last time I ran it was expanding a RAID array of 4 320 gig disks= =20 from RAID 10 to RAID 5, so from ~ 640 gigs to 960 gigs. Not sure if you are running into a growfs limitation, you're certainly runn= ing=20 into the limits of UFS with those sizes. Have you considered ZFS? At this point you can't put a bsdlabel on, as your filesystem probably=20 encroaches on the area the label would need anyways. Similarly the fdisk=20 output is fairly meaningless, as you aren't using a fdisk partition. To=20 verify ls /dev/da0* should only show /dev/da0, if it shows slices that woul= d=20 be bad. The man page for growfs mentions that some free space is needed in the firs= t=20 cylinder group to resize. Perhaps that is the problem you are running in t= o. =2D-=20 Thanks, Josh Paetzel --nextPart15124792.1rSBoADPcJ Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) iQEcBAABAgAGBQJMnNqtAAoJEKFq1/n1feG2eKoIAIeA9C8SNghbumk28lWSv5Th IJ7p1KcSW9mNjmOXiEZIXcs3bnNKM/326S57y2mo+aWV+4j1iPeTS+3eOtYNpOcF 2vFdNgfa0RKjDxGPckm8vX1oT6p3cfEIAX5nps+MqmMHKxpraxv+rESbRrWZuuj/ cETJyKThK/1wGSwFK70/b1guIJdhO9gHlyPofHXc+/Y+4CpnxFCPruIouy5ltH3q D+B5U2n9f73ZJgNitJcUKzqQDlOJCf7CqDyEQ6gNZmBeEk4rqvgWKYldi3hCWEEo J33TL2QJ7D75sbKT6zX/PkzYZ02D2II52naes6dASwddSm83a+l0lH9GMpbAI+A= =lObd -----END PGP SIGNATURE----- --nextPart15124792.1rSBoADPcJ--