From owner-freebsd-fs@FreeBSD.ORG Tue Oct 12 10:10:50 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8EC08106564A for ; Tue, 12 Oct 2010 10:10:50 +0000 (UTC) (envelope-from torbjoern@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 1B7638FC13 for ; Tue, 12 Oct 2010 10:10:49 +0000 (UTC) Received: by bwz16 with SMTP id 16so1447499bwz.13 for ; Tue, 12 Oct 2010 03:10:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=6JKgssC9PWDudBP2/GcQoN4u8oRuTdkmGhRtmdA4Mlk=; b=uQ1aXrHd8AUlFhd4y4bMzAQo//4Kb5JPTCcLn/69/JQennGQwMH6WBZGUBGE62WyIf tqEZYhbtYu3KmIGkRr1t36+JIwTnxhpxu9mBZdZirqxWooEg/WSW+mdIA3de0m3tpMlU 26USPb3gDDjsRqX2qttLLwukdgGbpzYmzoIEU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=m93XUH0dnC3BJHSidqv+1nYrrGMyFCnp6XP2L9CLIaPtNZnsatsK5vTVIwX2VNdL5K fBcNtU4pz2606CaHbRqEm+jL5xShMQF4FhwM5vcomZ9+vGDq0VPDkj1RewEXMbXm9ZGH LC3gTXZFxiZR20168OVQJvF6ZBCz553UYvEdY= MIME-Version: 1.0 Received: by 10.204.77.137 with SMTP id g9mr5902981bkk.189.1286878247837; Tue, 12 Oct 2010 03:10:47 -0700 (PDT) Received: by 10.204.71.138 with HTTP; Tue, 12 Oct 2010 03:10:47 -0700 (PDT) In-Reply-To: <6AE65535-DCCE-46A5-BBB9-358FEB34C18C@gmail.com> References: <6AE65535-DCCE-46A5-BBB9-358FEB34C18C@gmail.com> Date: Tue, 12 Oct 2010 12:10:47 +0200 Message-ID: From: Torbjorn Kristoffersen To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: Growing large UFS over 16TB? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Oct 2010 10:10:50 -0000 On Mon, Oct 11, 2010 at 10:18 PM, Tobias Fredriksson w= rote: > Hi, > > So I have this UFS, unfortunately it was just done newfs /dev/da1 with no= regards to installing GPT or anything like that. > > It started out as a 6x2TB system raid, then we expanded with 2x2TB. > > Growing this filesystem was not possible with the current tool however wi= th the patch from http://masq.tychl.net/growfs.patch it worked fine. > > Now that I'm trying to grow it from 12TB to closer to 20TB it fails after= about 15 minutes. > It displays all the super-block backups all the way to "39061769312" then= it sits there reading from the volume for about 5-10 minutes. > > The following error message is then output > growfs: rdfs: attempting to read negative block number: Inappropriate ioc= tl for device > > I understand the reason for this, its trying to read a block and the inte= ger just wrapped around. Nice. > The relevant lines from growfs.c are > > static void > rdfs(ufs2_daddr_t bno, size_t size, void *bf, int fsi) > { > =A0 =A0 =A0 =A0[...] > =A0 =A0 =A0 =A0if (bno < 0) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0err(32, "rdfs: attempting to read negative= block number"); > =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0[...] > > Just for fun I commented the if part out and recompiled. > growfs: rdfs: read error: -4889807711788704476: Input/output error > > The only place that ufs2_daddr_t is defined is in /usr/include/ufs/ufs/di= node.h > typedef int64_t =A0 =A0 =A0 ufs2_daddr_t; > > So again for fun I changed this to u_int64_t. I also removed the comments= on that if part in growfs.c > > This caused the same message as last to be repeated. But not the negative= number. > growfs: rdfs: read error: -4889807711788704476: Input/output error > > This leads me to believe that I'm at least doing something partially righ= t. > > The next part that growfs.c is doing in rdfs is > =A0 =A0 =A0 =A0n =3D read(fsi, bf, size); > =A0 =A0 =A0 =A0if (n !=3D (ssize_t)size) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0err(34, "rdfs: read error: %jd", (intmax_t= )bno); > =A0 =A0 =A0 =A0} > > So I changed > "rdfs(ufs2_daddr_t bno, size_t size, void *bf, int fsi)" > to > "rdfs(ufs2_daddr_t bno, size_t size, void *bf, u_int64_t fsi)" > However this changed nothing. Same output. > Since its failing out at the if sentance then of course the problem is in= ssize_t not being big enough. > As such I of course changed the values for this in /usr/include/machine/_= types.h and I also checked out _limit.h in the same dir. > _types.h > typedef __int64_t =A0 =A0 =A0__ssize_t; > to > typedef __uint64_t =A0 =A0 =A0__ssize_t; > > _limit.h > #define __SSIZE_MAX =A0 =A0 __LONG_MAX =A0 =A0 =A0/* max value for a ssiz= e_t */ > #define __SIZE_T_MAX =A0 =A0__ULONG_MAX =A0 =A0 /* max value for a size_t= */ > to > #define __SSIZE_MAX =A0 =A0__ULLONG_MAX =A0 =A0 =A0/* max value for a ssi= ze_t */ > #define __SIZE_T_MAX =A0 =A0__ULLONG_MAX =A0 =A0 /* max value for a size_= t */ > > However this failed to make any sort of change. I returned all of the lat= er values as nothing helped. > > So I'm turning to the fs gurus. At the moment I have no way of moving the= data off and creating this properly. As such changing to another fs is not= an option right now neither. > > So if anybody has any suggestions on what to do to temporarily fix the is= sue until we can move the data of the raid and rebuild it properly, please = let me know. > > [root@stor1 /usr/src/sbin/growfs]# ./growfs /dev/da1 > We strongly recommend you to make a backup before growing the Filesystem > > =A0Did you backup your data (Yes/No) ? Yes > new file systemsize is: 9765570560 frags > Warning: 136832 sector(s) cannot be allocated. > growfs: 19073314.0MB (39062145408 sectors) block size 16384, fragment siz= e 2048 > =A0 =A0 =A0 =A0using 103818 cylinder groups of 183.72MB, 11758 blks, 2355= 2 inodes. > [...] > growfs: rdfs: read error: -4889807711788704476: Input/output error > Unfortunately I can't answer your question, but have you considered using Z= FS?