Date: Fri, 24 Sep 2010 12:06:45 -0500 From: Josh Paetzel <josh@tcbug.org> To: freebsd-fs@freebsd.org Subject: Re: growfs on a large disk Message-ID: <201009241206.53828.josh@tcbug.org> In-Reply-To: <4C9C7C32.3070909@fragfest.com.au> References: <4C9C7C32.3070909@fragfest.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
--nextPart15124792.1rSBoADPcJ Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable On Friday, September 24, 2010 05:23:46 am Dean Hamstead wrote: > Evening, >=20 > Im trying to growfs a very large filesystem and not having a lot of luck. >=20 > To give some background, im running the filesystem on a raid array > running on an areca raid card. I have an existing filesytem which was > formatting /dev/da0 and mounting it. (I understand that this is the > preferred alternative to the deprecated 'dangerously dedicated' mode). >=20 > I have expanded the disk array and the volume at the raid card level. > Then rebooted my machine. It was 4x2tb, and is now 5x2tb. Expanding on > areca is non-destructive. >=20 > Here is the current filesystem size when mounted, da0 is the >=20 > ----- >=20 > [root@batou /home/dean]# df -h > Filesystem Size Used Avail Capacity Mounted on > /dev/ad4a 496M 280M 176M 61% / > devfs 1.0K 1.0K 0B 100% /dev > /dev/ad4e 496M 155M 301M 34% /tmp > /dev/ad4f 68G 8.0G 54G 13% /usr > /dev/ad4d 1.9G 529M 1.2G 30% /var > /dev/da1 4.4T 4.4T 28G 99% /volumes/store2 > /dev/da0 5.3T 3.9T 1.4T 74% /volumes/store1 >=20 > ----- >=20 > From dmesg you can see that freebsd knows its now bigger (7.6 rather > than 5.3tb) >=20 > ----- >=20 > da0 at arcmsr0 bus 0 scbus0 target 0 lun 0 > da0: <Areca ARC-1261-VOL#00 R001> Fixed Direct Access SCSI-5 device > da0: 166.666MB/s transfers (83.333MHz, offset 32, 16bit) > da0: Command Queueing enabled > da0: 7629394MB (15624998912 512 byte sectors: 255H 63S/T 972611C) > da1 at arcmsr0 bus 0 scbus0 target 0 lun 1 > da1: <Areca ARC-1261-VOL#01 R001> Fixed Direct Access SCSI-5 device > da1: 166.666MB/s transfers (83.333MHz, offset 32, 16bit) > da1: Command Queueing enabled > da1: 4768371MB (9765624320 512 byte sectors: 255H 63S/T 607881C) > SMP: AP CPU #1 Launched! >=20 > ----- >=20 > if we consult diskinfo, it agrees >=20 > ----- >=20 > [root@batou /home/dean]# diskinfo -v /dev/da0 > /dev/da0 > 512 # sectorsize > 7999999442944 # mediasize in bytes (7.3T) > 15624998912 # mediasize in sectors > 0 # stripesize > 0 # stripeoffset > 972611 # Cylinders according to firmware. > 255 # Heads according to firmware. > 63 # Sectors according to firmware. > 0000001379342420 # Disk ident. >=20 >=20 > ----- >=20 > fdisk shows us... >=20 > ----- >=20 > [root@batou /home/dean]# fdisk /dev/da0 > ******* Working on device /dev/da0 ******* > parameters extracted from in-core disklabel are: > cylinders=3D972611 heads=3D255 sectors/track=3D63 (16065 blks/cyl) >=20 > Figures below won't work with BIOS for partitions not in cyl 1 > parameters to be used for BIOS calculations are: > cylinders=3D972611 heads=3D255 sectors/track=3D63 (16065 blks/cyl) >=20 > fdisk: invalid fdisk partition table found > Media sector size is 512 > Warning: BIOS sector numbering starts with sector 1 > Information from DOS bootblock is: > The data for partition 1 is: > sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) > start 63, size 2740094532 (1337936 Meg), flag 80 (active) > beg: cyl 0/ head 1/ sector 1; > end: cyl 578/ head 254/ sector 63 > The data for partition 2 is: > <UNUSED> > The data for partition 3 is: > <UNUSED> > The data for partition 4 is: > <UNUSED> >=20 > ----- >=20 > now if i unmount /dev/da0 and run growfs /dev/da0 i get the following >=20 > ----- >=20 > [root@batou /home/dean]# growfs /dev/da0 > growfs: we are not growing (2929687296->685024256) >=20 >=20 > ----- >=20 > with lots of googling, i have found that there is much talk in various > freebsd forums about the need to use disklabel. however it seems unable > to fathom such a large disk... >=20 > ----- >=20 > [root@batou /home/dean]# disklabel /dev/da0 > disklabel: disks with more than 2^32-1 sectors are not supported >=20 >=20 > ---- >=20 > freebsd version is 8.0 64bit >=20 > ----- >=20 > [root@batou /home/dean]# uname -a > FreeBSD batou.fragfest.net.au 8.0-STABLE FreeBSD 8.0-STABLE #7: Sun Jan > 24 17:13:52 EST 2010 > dean@batou.fragfest.net.au:/usr/obj/usr/src/sys/GENERIC amd64 >=20 > ----- >=20 > hopefully i am doing something wrong, or there is a patch set for growfs? >=20 >=20 >=20 > Dean I've run growfs on raw partitions, it's the easiest way to use growfs, as y= ou=20 don't need to worrk about expanding partitions or slices. Sadly the last time I ran it was expanding a RAID array of 4 320 gig disks= =20 from RAID 10 to RAID 5, so from ~ 640 gigs to 960 gigs. Not sure if you are running into a growfs limitation, you're certainly runn= ing=20 into the limits of UFS with those sizes. Have you considered ZFS? At this point you can't put a bsdlabel on, as your filesystem probably=20 encroaches on the area the label would need anyways. Similarly the fdisk=20 output is fairly meaningless, as you aren't using a fdisk partition. To=20 verify ls /dev/da0* should only show /dev/da0, if it shows slices that woul= d=20 be bad. The man page for growfs mentions that some free space is needed in the firs= t=20 cylinder group to resize. Perhaps that is the problem you are running in t= o. =2D-=20 Thanks, Josh Paetzel --nextPart15124792.1rSBoADPcJ Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) iQEcBAABAgAGBQJMnNqtAAoJEKFq1/n1feG2eKoIAIeA9C8SNghbumk28lWSv5Th IJ7p1KcSW9mNjmOXiEZIXcs3bnNKM/326S57y2mo+aWV+4j1iPeTS+3eOtYNpOcF 2vFdNgfa0RKjDxGPckm8vX1oT6p3cfEIAX5nps+MqmMHKxpraxv+rESbRrWZuuj/ cETJyKThK/1wGSwFK70/b1guIJdhO9gHlyPofHXc+/Y+4CpnxFCPruIouy5ltH3q D+B5U2n9f73ZJgNitJcUKzqQDlOJCf7CqDyEQ6gNZmBeEk4rqvgWKYldi3hCWEEo J33TL2QJ7D75sbKT6zX/PkzYZ02D2II52naes6dASwddSm83a+l0lH9GMpbAI+A= =lObd -----END PGP SIGNATURE----- --nextPart15124792.1rSBoADPcJ--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201009241206.53828.josh>