From owner-freebsd-fs@FreeBSD.ORG Sun Jun 18 23:35:58 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8C1F516A479 for ; Sun, 18 Jun 2006 23:35:58 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4AE6143D48 for ; Sun, 18 Jun 2006 23:35:58 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id DE7F631332; Sun, 18 Jun 2006 19:35:56 -0400 (EDT) Date: Sun, 18 Jun 2006 19:35:56 -0400 (EDT) From: Ensel Sharon To: Scott Long In-Reply-To: <4489E64F.402@samsco.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-fs@freebsd.org Subject: Re: quick check - adaptec 2820sa on FreeBSD 6.1 ? UGH! X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Jun 2006 23:35:58 -0000 On Fri, 9 Jun 2006, Scott Long wrote: > Are you asking because you need advice on whether or not to purchase it? > Yes, it will work, but if you already have the card then you should > test it out for yourself and make sure that it meets your needs. Ok, I bought two of them. Installed. Inserted 6.1-RELEASE cd-rom, booted ... and ... "no disks found" So ... were you thinking of 7.x perhaps ? Am I totally screwed now, or is there some way to salvage this ? The cards work great - I init'd the disks and created a raid6 volume, verified. All done. It's just that FreeBSD 6.1-RELEASE cannot see the 2820sa to install on. Please help ... From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 00:27:23 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4052216A479; Mon, 19 Jun 2006 00:27:23 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id E808A43D45; Mon, 19 Jun 2006 00:27:22 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id 705AF31332; Sun, 18 Jun 2006 20:27:18 -0400 (EDT) Date: Sun, 18 Jun 2006 20:27:18 -0400 (EDT) From: Ensel Sharon To: freebsd-questions@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-fs@freebsd.org Subject: Please help - adaptec 2820sa not allowing installation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 00:27:23 -0000 I did all my due diligence, I contacted freebsd-fs and _made sure_ that even though the 2820sa is not listed by name in the HCL, that I could take a 6.1-RELEASE cd and install freebsd on a 2820sa. I was _assured_ that these cards are supported in 6.1-RELEASE, that all is well, and I could install and that was that. Now I boot off of the 6.1-RELEASE cd, and sysinstall tells me "no disks found". I tried safe mode, same result. I tried disabling all unnecessary items from the BIOS (onboard sata, USB, com2, LPT) and same result. FreeBSD 6.1-RELEASE iso just does not find a disk controller. Period. So ... what do I do ?? I see no way of adding a driver to sysinstall .. .and even if I did I have no idea what to put there, as the published driver from adaptec is only for 5.x. I have to assume that aac is built into the sysinstall kernel, so what gives ? Why can't sysinstall see my 2820sa cards ? (and drives) Thanks. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 00:39:26 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0993416A474 for ; Mon, 19 Jun 2006 00:39:26 +0000 (UTC) (envelope-from jahilliya@gmail.com) Received: from nz-out-0102.google.com (nz-out-0102.google.com [64.233.162.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 41F9543D53 for ; Mon, 19 Jun 2006 00:39:25 +0000 (GMT) (envelope-from jahilliya@gmail.com) Received: by nz-out-0102.google.com with SMTP id m7so478124nzf for ; Sun, 18 Jun 2006 17:39:24 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=VY4V0/zFwbsdXx8fHJrgXpTMV5qKb44KYT8vmWhiQIPC5mwFHDeBRacf91ceZciB7nrWkhK3L+GSDrkqWVwXWCu3D/9dFykEQPZC8a54prgW92hz+f/3uV6c9MNPm7hek232rfX/IVS30Av/aHWebccD//maq5A0eCDJk7FTMas= Received: by 10.36.7.20 with SMTP id 20mr2106158nzg; Sun, 18 Jun 2006 17:39:24 -0700 (PDT) Received: by 10.37.15.74 with HTTP; Sun, 18 Jun 2006 17:39:24 -0700 (PDT) Message-ID: Date: Mon, 19 Jun 2006 08:39:24 +0800 From: Jahilliya To: "Ensel Sharon" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not allowing installation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 00:39:26 -0000 On 6/19/06, Ensel Sharon wrote: > > I did all my due diligence, I contacted freebsd-fs and _made sure_ that > even though the 2820sa is not listed by name in the HCL, that I could take > a 6.1-RELEASE cd and install freebsd on a 2820sa. > > I was _assured_ that these cards are supported in 6.1-RELEASE, that all is > well, and I could install and that was that. > > Now I boot off of the 6.1-RELEASE cd, and sysinstall tells me "no disks > found". > > I tried safe mode, same result. I tried disabling all unnecessary items > from the BIOS (onboard sata, USB, com2, LPT) and same result. > > FreeBSD 6.1-RELEASE iso just does not find a disk controller. Period. > > So ... what do I do ?? I see no way of adding a driver to sysinstall > .. .and even if I did I have no idea what to put there, as the published > driver from adaptec is only for 5.x. > > I have to assume that aac is built into the sysinstall kernel, so what > gives ? Why can't sysinstall see my 2820sa cards ? (and drives) Try hitting scroll lock and then page up to view the dmesg or go to the emergency console and run dmesg to see if it picked up any aac devices. You may need to load the module yourself, you should be able to do this from the emergency console (not sure tho). I've steered clear of the *20SA series of cards from Adaptec (mainly buying the 2410SA) for the two facts, SATA2 isn't listed anywhere on the hardware list and only the xx10SA series SATA cards are listed. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 00:49:25 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3A70916A479; Mon, 19 Jun 2006 00:49:25 +0000 (UTC) (envelope-from john@kozubik.com) Received: from kozubik.com (kozubik.com [69.43.165.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id EDE2243D45; Mon, 19 Jun 2006 00:49:24 +0000 (GMT) (envelope-from john@kozubik.com) Received: from kozubik.com (localhost [127.0.0.1]) by kozubik.com (8.12.11/8.12.11) with ESMTP id k5J0nQ3k003391; Sun, 18 Jun 2006 17:49:26 -0700 (PDT) (envelope-from john@kozubik.com) Received: from localhost (john@localhost) by kozubik.com (8.12.11/8.12.11/Submit) with ESMTP id k5J0nQ0t003388; Sun, 18 Jun 2006 17:49:26 -0700 (PDT) (envelope-from john@kozubik.com) Date: Sun, 18 Jun 2006 17:49:26 -0700 (PDT) From: John Kozubik To: Jahilliya In-Reply-To: Message-ID: <20060618174555.M730@kozubik.com> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-fs@freebsd.org, Ensel Sharon , freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not allowing installation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 00:49:25 -0000 On Mon, 19 Jun 2006, Jahilliya wrote: > Try hitting scroll lock and then page up to view the dmesg or go to > the emergency console and run dmesg to see if it picked up any aac > devices. You may need to load the module yourself, you should be able > to do this from the emergency console (not sure tho). > > I've steered clear of the *20SA series of cards from Adaptec (mainly > buying the 2410SA) for the two facts, SATA2 isn't listed anywhere on > the hardware list and only the xx10SA series SATA cards are listed. Yes, that's a good point - which is why I was very cautious and made doubly sure to ask very clearly on this mailing list "can I install freebsd 6.1 on a 2820sa even though it is not in the HCL" and I was assured that the answer was yes. So you are saying you think that the 6.1-RELEASE iso does not have aac built into it ? After 4-5 years of it being in every install media, even floppies, you think it suddenly is just not there anymore ? I find that difficult to believe... I'll check the dmesg in scroll lock to be sure, but I am pretty sure it comes up with nothing aac-related. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 00:56:18 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BDAB216A47B; Mon, 19 Jun 2006 00:56:18 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4F77D43DA0; Mon, 19 Jun 2006 00:55:56 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id 06A453132F; Sun, 18 Jun 2006 20:55:52 -0400 (EDT) Date: Sun, 18 Jun 2006 20:55:52 -0400 (EDT) From: Ensel Sharon To: Jahilliya In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not allowing installation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 00:56:18 -0000 On Mon, 19 Jun 2006, Jahilliya wrote: > Try hitting scroll lock and then page up to view the dmesg or go to > the emergency console and run dmesg to see if it picked up any aac > devices. You may need to load the module yourself, you should be able > to do this from the emergency console (not sure tho). > > I've steered clear of the *20SA series of cards from Adaptec (mainly > buying the 2410SA) for the two facts, SATA2 isn't listed anywhere on > the hardware list and only the xx10SA series SATA cards are listed. Ok, aac is in the dmesg. I can see both 2820sa cards in the dmesg, and I see no errors, etc. - there are just no drives listed in dmesg. My setup is 8 500 Gb drives in a single raid-6 array, size ~2.8TB. Any problems with that ? Perhaps sysinstall cannot deal with a >2TB drive ? There are no other drives in the system besides the single 2.8TB raid 6 array ... Thanks. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 01:01:00 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3EDA516A4A9 for ; Mon, 19 Jun 2006 01:01:00 +0000 (UTC) (envelope-from jahilliya@gmail.com) Received: from nz-out-0102.google.com (nz-out-0102.google.com [64.233.162.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5438043D72 for ; Mon, 19 Jun 2006 01:00:59 +0000 (GMT) (envelope-from jahilliya@gmail.com) Received: by nz-out-0102.google.com with SMTP id m7so480044nzf for ; Sun, 18 Jun 2006 18:00:58 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=VJPEno0nmk6TFXmQdIICL1MLCvtJTsW30cYiIk9h8Ga83DWGciCBHTsPfRNtilz2QMzjXVn7cpZYPKwcPgMGsA1AsGp5jLk5FJAGuoLem858sQ46zgtXchAKVCap/El2w6Gr9RD2jweD6UJkL/BNYRzQ9dGjZjnBSU/k7Jjqtpo= Received: by 10.36.91.2 with SMTP id o2mr1129593nzb; Sun, 18 Jun 2006 18:00:58 -0700 (PDT) Received: by 10.37.15.74 with HTTP; Sun, 18 Jun 2006 18:00:58 -0700 (PDT) Message-ID: Date: Mon, 19 Jun 2006 09:00:58 +0800 From: Jahilliya To: "John Kozubik" In-Reply-To: <20060618174555.M730@kozubik.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20060618174555.M730@kozubik.com> Cc: freebsd-fs@freebsd.org, Ensel Sharon , freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not allowing installation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 01:01:00 -0000 On 6/19/06, John Kozubik wrote: > > > On Mon, 19 Jun 2006, Jahilliya wrote: > > > Try hitting scroll lock and then page up to view the dmesg or go to > > the emergency console and run dmesg to see if it picked up any aac > > devices. You may need to load the module yourself, you should be able > > to do this from the emergency console (not sure tho). > > > > I've steered clear of the *20SA series of cards from Adaptec (mainly > > buying the 2410SA) for the two facts, SATA2 isn't listed anywhere on > > the hardware list and only the xx10SA series SATA cards are listed. > > > Yes, that's a good point - which is why I was very cautious and made > doubly sure to ask very clearly on this mailing list "can I install > freebsd 6.1 on a 2820sa even though it is not in the HCL" and I was > assured that the answer was yes. > > So you are saying you think that the 6.1-RELEASE iso does not have aac > built into it ? After 4-5 years of it being in every install media, even > floppies, you think it suddenly is just not there anymore ? I find that > difficult to believe... > > I'll check the dmesg in scroll lock to be sure, but I am pretty sure it > comes up with nothing aac-related. I'm saying it's a possibility if you're not seeing anything AAC related in the dmesg and you should make sure that it is in the kernel or loaded as a module by running kldload (kldload will give some error about symbol already existing if it is built into the kernel). From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 01:04:58 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF39416A479 for ; Mon, 19 Jun 2006 01:04:58 +0000 (UTC) (envelope-from jahilliya@gmail.com) Received: from nz-out-0102.google.com (nz-out-0102.google.com [64.233.162.195]) by mx1.FreeBSD.org (Postfix) with ESMTP id CBE0143D6E for ; Mon, 19 Jun 2006 01:04:57 +0000 (GMT) (envelope-from jahilliya@gmail.com) Received: by nz-out-0102.google.com with SMTP id m7so480396nzf for ; Sun, 18 Jun 2006 18:04:57 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=KV30n6GcTdgX0Mn+179vrDDN7oHc2yaPrMMR/jLWBi4lG7Z0k/nRmIE2BcrtQzpF/3wnCRIA5BzUoGP9TCNS2N1dB8570VvKX4nvbGWStM0oQSN1KX62DpcYMBUDmdKlvup3/AU+9bMeGqonfUws8Scn7M+aLpy+DgA2uFe6Vmk= Received: by 10.37.12.45 with SMTP id p45mr6799747nzi; Sun, 18 Jun 2006 18:04:57 -0700 (PDT) Received: by 10.37.15.74 with HTTP; Sun, 18 Jun 2006 18:04:56 -0700 (PDT) Message-ID: Date: Mon, 19 Jun 2006 09:04:56 +0800 From: Jahilliya To: "Ensel Sharon" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not allowing installation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 01:04:58 -0000 > Ok, aac is in the dmesg. > > I can see both 2820sa cards in the dmesg, and I see no errors, etc. - > there are just no drives listed in dmesg. > > My setup is 8 500 Gb drives in a single raid-6 array, size ~2.8TB. Any > problems with that ? Perhaps sysinstall cannot deal with a >2TB drive > ? There are no other drives in the system besides the single 2.8TB raid 6 > array ... > Have you got any other drives you can attach to the raid? If so, disconnect the 8 drives connected, connect up a couple that are not part of the raid and configure them as a simple raid 1 and see if the installers sees that raid. Or try any combination in drives to bring the raid size down below 2TB (I'm sure this limitation has been fixed.) From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 01:08:45 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9B51316A585 for ; Mon, 19 Jun 2006 01:08:45 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4C84F43D79 for ; Mon, 19 Jun 2006 01:08:44 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id 1F8B831309; Sun, 18 Jun 2006 21:08:43 -0400 (EDT) Date: Sun, 18 Jun 2006 21:08:43 -0400 (EDT) From: Ensel Sharon To: Jahilliya In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-fs@freebsd.org Subject: Re: Please help - adaptec 2820sa not allowing installation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 01:08:45 -0000 On Mon, 19 Jun 2006, Jahilliya wrote: > > Ok, aac is in the dmesg. > > > > I can see both 2820sa cards in the dmesg, and I see no errors, etc. - > > there are just no drives listed in dmesg. > > > > My setup is 8 500 Gb drives in a single raid-6 array, size ~2.8TB. Any > > problems with that ? Perhaps sysinstall cannot deal with a >2TB drive > > ? There are no other drives in the system besides the single 2.8TB raid 6 > > array ... > > > > Have you got any other drives you can attach to the raid? > > If so, disconnect the 8 drives connected, connect up a couple that are > not part of the raid and configure them as a simple raid 1 and see if > the installers sees that raid. Or try any combination in drives to > bring the raid size down below 2TB (I'm sure this limitation has been > fixed.) Ok. I will do this. My original strategy was to: 1. create a ~2.8TB raid6 array out of 8 500GB drives 2. just make one big monster partition of the entire 2.8TB under "partition" in sysinstall 3. under "label" in sysinstall, divide that into /, swap, /var and two large data partitions, neither of which exceeds 2TB (since snapshos have trouble over 2tb) do you think the above srategy is reasonable ? From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 01:48:00 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C87116A474; Mon, 19 Jun 2006 01:48:00 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 862CF43D46; Mon, 19 Jun 2006 01:47:59 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id ABD9631324; Sun, 18 Jun 2006 21:47:58 -0400 (EDT) Date: Sun, 18 Jun 2006 21:47:58 -0400 (EDT) From: Ensel Sharon To: freebsd-fs@freebsd.org In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not ... RESOLVED X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 01:48:00 -0000 On Mon, 19 Jun 2006, Jahilliya wrote: > > Ok, aac is in the dmesg. > > > > I can see both 2820sa cards in the dmesg, and I see no errors, etc. - > > there are just no drives listed in dmesg. > > > > My setup is 8 500 Gb drives in a single raid-6 array, size ~2.8TB. Any > > problems with that ? Perhaps sysinstall cannot deal with a >2TB drive > > ? There are no other drives in the system besides the single 2.8TB raid 6 > > array ... > > > > Have you got any other drives you can attach to the raid? > > If so, disconnect the 8 drives connected, connect up a couple that are > not part of the raid and configure them as a simple raid 1 and see if > the installers sees that raid. Or try any combination in drives to > bring the raid size down below 2TB (I'm sure this limitation has been > fixed.) Ok, the answer is that it has not been fixed. 6.1 sysinstall does in fact see both 2820sa controllers, and when I put in a single 160GB sata drive, it does see that single drive and I can install onto it, etc. Sysinstall does _not_ see my 2.7TB raid6 array. I suspect that if it were smaller than 2TB, it would see it correctly. I have a number of options with which to deal with this, all of which involve either wasting money or wasting disk space. Fantastic. From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 02:06:39 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9D2F016A47F; Mon, 19 Jun 2006 02:06:39 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4394F43D45; Mon, 19 Jun 2006 02:06:39 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.22] (andersonbox2.centtech.com [192.168.42.22]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k5J26bHi029249; Sun, 18 Jun 2006 21:06:37 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <449606B8.5080106@centtech.com> Date: Sun, 18 Jun 2006 21:06:48 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: Ensel Sharon References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1549/Sat Jun 17 17:20:39 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not ... RESOLVED X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 02:06:39 -0000 Ensel Sharon wrote: > > On Mon, 19 Jun 2006, Jahilliya wrote: > >>> Ok, aac is in the dmesg. >>> >>> I can see both 2820sa cards in the dmesg, and I see no errors, etc. - >>> there are just no drives listed in dmesg. >>> >>> My setup is 8 500 Gb drives in a single raid-6 array, size ~2.8TB. Any >>> problems with that ? Perhaps sysinstall cannot deal with a >2TB drive >>> ? There are no other drives in the system besides the single 2.8TB raid 6 >>> array ... >>> >> Have you got any other drives you can attach to the raid? >> >> If so, disconnect the 8 drives connected, connect up a couple that are >> not part of the raid and configure them as a simple raid 1 and see if >> the installers sees that raid. Or try any combination in drives to >> bring the raid size down below 2TB (I'm sure this limitation has been >> fixed.) > > > Ok, the answer is that it has not been fixed. > > 6.1 sysinstall does in fact see both 2820sa controllers, and when I put in > a single 160GB sata drive, it does see that single drive and I can install > onto it, etc. > > Sysinstall does _not_ see my 2.7TB raid6 array. I suspect that if it were > smaller than 2TB, it would see it correctly. > > I have a number of options with which to deal with this, all of which > involve either wasting money or wasting disk space. Fantastic. Right - FreeBSD doesn't recognize >2TB LUNs. You should make two LUNs, and concat them or stripe them with GEOM. Don't use the large partition for the OS either. You shouldn't waste either disk space or money. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 02:44:39 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 48A3116A479; Mon, 19 Jun 2006 02:44:39 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2C1D443D6A; Mon, 19 Jun 2006 02:44:38 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id 148C831323; Sun, 18 Jun 2006 22:44:37 -0400 (EDT) Date: Sun, 18 Jun 2006 22:44:36 -0400 (EDT) From: Ensel Sharon To: Eric Anderson In-Reply-To: <449606B8.5080106@centtech.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not ... RESOLVED X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 02:44:39 -0000 On Sun, 18 Jun 2006, Eric Anderson wrote: > > 6.1 sysinstall does in fact see both 2820sa controllers, and when I put in > > a single 160GB sata drive, it does see that single drive and I can install > > onto it, etc. > > > > Sysinstall does _not_ see my 2.7TB raid6 array. I suspect that if it were > > smaller than 2TB, it would see it correctly. > > > > I have a number of options with which to deal with this, all of which > > involve either wasting money or wasting disk space. Fantastic. > > Right - FreeBSD doesn't recognize >2TB LUNs. You should make two LUNs, > and concat them or stripe them with GEOM. Don't use the large partition > for the OS either. > > You shouldn't waste either disk space or money. Let's say I have 8 disks. Let's say I require raid6. If I make one array, I lose 25% to raid overhead. If I make two arrays, I lose 50% to raid overhead. So it would seem that my inability to use a >2TB LUN does indeed lose me both space and waste money. My solution is to use two disks as a mirror, and use the other six for a raid6 array, thus losing 3/8 to raid overhead instead of 4/8, but it's still worse than 2/8 which is what I wanted to do ... Perhaps I misunderstand you ? From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 03:43:42 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AB6B216A479; Mon, 19 Jun 2006 03:43:42 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 394BB43D49; Mon, 19 Jun 2006 03:43:41 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.22] (andersonbox2.centtech.com [192.168.42.22]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k5J3hdU8044161; Sun, 18 Jun 2006 22:43:39 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44961D76.4050509@centtech.com> Date: Sun, 18 Jun 2006 22:43:50 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: Ensel Sharon References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1549/Sat Jun 17 17:20:39 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not ... RESOLVED X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 03:43:42 -0000 Ensel Sharon wrote: > > On Sun, 18 Jun 2006, Eric Anderson wrote: > >>> 6.1 sysinstall does in fact see both 2820sa controllers, and when I put in >>> a single 160GB sata drive, it does see that single drive and I can install >>> onto it, etc. >>> >>> Sysinstall does _not_ see my 2.7TB raid6 array. I suspect that if it were >>> smaller than 2TB, it would see it correctly. >>> >>> I have a number of options with which to deal with this, all of which >>> involve either wasting money or wasting disk space. Fantastic. >> Right - FreeBSD doesn't recognize >2TB LUNs. You should make two LUNs, >> and concat them or stripe them with GEOM. Don't use the large partition >> for the OS either. >> >> You shouldn't waste either disk space or money. > > > Let's say I have 8 disks. > > Let's say I require raid6. > > If I make one array, I lose 25% to raid overhead. > > If I make two arrays, I lose 50% to raid overhead. > > So it would seem that my inability to use a >2TB LUN does indeed lose me > both space and waste money. I suppose if you call increased redundancy 'waste' then yes. Wouldn't two 4 disk RAID 5 arrays give you similar (not exactly, but close) redundancy to the RAID6 option, and keep your space up? If you require RAID6, then the point is mute, and you're stuck with multiple RAID6 arrays unless your controller allows you to carve LUNs from the array, so you could create a 2.7TB 8 disk RAID6, then carve off a few LUNs for the OS to see. By the way - FreeBSD isn't the only OS without the addressing necessary to support >2TB SCSI LUNs. > My solution is to use two disks as a mirror, and use the other six for a > raid6 array, thus losing 3/8 to raid overhead instead of 4/8, but it's > still worse than 2/8 which is what I wanted to do ... > > Perhaps I misunderstand you ? That sounds reasonable, and should give you a working setup. Judging by the disks and setup, it sounds like you are looking for a good archive type storage, and not high performance (high-IO) storage, so the above configuration would suit you just fine I suspect. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 12:49:02 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EDE2316A481; Mon, 19 Jun 2006 12:49:02 +0000 (UTC) (envelope-from daichi@freebsd.org) Received: from natial.ongs.co.jp (natial.ongs.co.jp [202.216.232.58]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5DF6943D77; Mon, 19 Jun 2006 12:49:02 +0000 (GMT) (envelope-from daichi@freebsd.org) Received: from [192.168.1.101] (dullmdaler.ongs.co.jp [202.216.232.62]) by natial.ongs.co.jp (Postfix) with ESMTP id 79E43244C3A; Mon, 19 Jun 2006 21:49:00 +0900 (JST) Message-ID: <44969D39.4060703@freebsd.org> Date: Mon, 19 Jun 2006 21:48:57 +0900 From: Daichi GOTO User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: John Hein References: <44926377.5000005@freebsd.org> <17554.57748.372948.664580@gromit.timing.com> In-Reply-To: <17554.57748.372948.664580@gromit.timing.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Daichi GOTO , Masanori OZAWA Subject: Re: [ANN] unionfs patchset-14 release X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 12:49:03 -0000 John Hein wrote: > I would just remove -r from mount_unionfs.c. Then, in the man page, > perhaps move the discussion about its removal to the HISTORY section > (in addition to some brief words about this rewrite as an evolutionary > milestone). Yeah, you are correct. We'll fix above things at next patchet-15. Thanks for your report :) > Thanks for the good work. -- Daichi GOTO, http://people.freebsd.org/~daichi From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 13:13:59 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E836D16A47A; Mon, 19 Jun 2006 13:13:58 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8E08E43D69; Mon, 19 Jun 2006 13:13:43 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id A483851339; Mon, 19 Jun 2006 15:13:41 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id F377750E81; Mon, 19 Jun 2006 15:13:34 +0200 (CEST) Date: Mon, 19 Jun 2006 15:11:01 +0200 From: Pawel Jakub Dawidek To: freebsd-current@FreeBSD.org Message-ID: <20060619131101.GD1130@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rJwd6BRFiFCcLxzm" Content-Disposition: inline X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 13:13:59 -0000 --rJwd6BRFiFCcLxzm Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello. For the last few months I have been working on gjournal project. To stop confusion right here, I want to note, that this project is not related to gjournal project on which Ivan Voras was working on the last SoC (2005). The lack of journaled file system in FreeBSD was a tendon of achilles for many years. We do have many file systems, but none with journaling: - ext2fs (journaling is in ext3fs), - XFS (read-only), - ReiserFS (read-only), - HFS+ (read-write, but without journaling), - NTFS (read-only). GJournal was designed to journal GEOM providers, so it actually works below file system layer, but it has hooks which allow to work with file systems. In other words, gjournal is not file system-depended, it can work probably with any file system with minimum knowledge about it. I implemented only UFS support. The patches are here: http://people.freebsd.org/~pjd/patches/gjournal.patch (for HEAD) http://people.freebsd.org/~pjd/patches/gjournal6.patch (for RELENG_6) To patch your sources you need to: # cd /usr/src # mkdir sbin/geom/class/journal sys/geom/journal sys/modules/geom/geom_jou= rnal # patch < /path/to/gjournal.patch Add 'options UFS_GJOURNAL' to your kernel configuration file and recompile kernel and world. How it works (in short). You may define one or two providers which gjournal will use. If one provider is given, it will be used for both - data and journal. If two providers are given, one will be used for data and one for journal. Every few seconds (you may define how many) journal is terminated and marked as consistent and gjournal starts to copy data from it to the data provider. In the same time new data are stored in new journal. Let's call the moment in which journal is terminated as "journal switch". Journal switch looks as follows: 1. Start journal switch if we have timeout or if we run out of cache. Don't perform journal switch if there were no write requests. 2. If we have file system, synchronize it. 3. Mark file system as clean. 4. Block all write requests to the file system. 5. Terminate the journal. 6. Eventually wait if copying of the previous journal is not yet finished. 7. Send BIO_FLUSH request (if the given provider supports it). 8. Mark new journal position on the journal provider. 9. Unblock write requests. 10. Start copying data from the terminated journal to the data provider. There were few things I needed to implement outside gjournal to make it work reliable: - The BIO_FLUSH request. Currently we have three I/O requests: BIO_READ, BIO_WRITE and BIO_DELETE. I added BIO_FLUSH, which means "flush your write cache". The request is send always with the biggest bio_offset set (mediasize of the destination provider), so it will work properly with bioq_disksort(). The caller need to stop further I/O requests before BIO_FLUSH return, so we don't have starvation effect. The hard part is that is has to be implemented in every disk driver, because flushing the cache is driver-depended operation. I implemented it for ata(4) disks and amr(4). The good news is that it's easy. GJournal can also work with providers that don't support BIO_FLUSH and in my power-failure tests it worked well (no problems), but it depend on fact, that gjournal cache is bigger than the controller cache, so it is hard to call it reliable. You can read in documentation to many journaled file systems, that you should turn off write cache if you want to use it. This is not the case for gjournal (especially when your disk driver does support BIO_FLUSH). The 'gjournal' mount option. To implement gjournal support in UFS I needed to change the way of how deleted, but still open objects are handled. Currently when file or directory is open and we deleted last name which reference it, it will still be usable by those who keep it open. When the last consumer closes it, the inode and blocks are freed. On journal switch I cannot leave such objects, because after a crash fsck(8) is not used to check the file system, so inode and blocks will never be freed. When file system is mounted with 'gjournal' mount option, such objects are not removed when they are open. When last name is deleted, the file/directory is moved to the .deleted/ directory and removed from there on last close. This way, I can just clean the .deleted/ directory after a crash at mount time. Quick start: # gjournal label /dev/ad0 # gjournal load # newfs /dev/ad0.journal # mount -o async,gjournal /dev/ad0.journal /mnt (yes, with gjournal 'async' is safe) Now, after a power failure or system crash no fsck is needed (yay!). There are two hacks in the current implementation, which I'd like to reimplement. First is how 'gjournal' mount option is implemented. There is a garbage collector thread which is responsible for deleting objects from .deleted/ directory and it is using full paths. Because of this when your mount point is /foo/bar/baz and you rename 'bar' to something else, it will not work. This is not what is often done, but definitely should be fixed and I'm working on it. The second hack is related to communication between gjournal and file system. GJournal decides when to make the switch and has to find file system which is mounted on it. Looking for this file system is not nice and should be reimplemented. There are some additional goods which came with gjournal. For example if gjournal is configured over gmirror or graid3, even on power failure or system crash, there is no need to synchronize mirror/raid3 device, because data will be consistent. I spend a lot of time working on gjournal optimization. Because I've few seconds before the data hit the data provider I can perform things like combining smaller write requests into larger once, ignoring data written twice to the same place, etc. Because of this, operations on small files are quite fast. On the other hand, operations on large files are slower, because I need to write the data twice and there is no place for optimization. Here are some numbers. gjournal(1) - the data provider and the journal provider on the same disk gjournal(2) - the data provider and the journal provider on separate disks Copying one large file: UFS: 8s UFS+SU: 8s gjournal(1): 16s gjournal(2): 14s Copying eight large files in parallel: UFS: 120s UFS+SU: 120s gjournal(1): 184s gjournal(2): 165s Untaring eight src.tgz in parallel: UFS: 791s UFS+SU: 650s gjournal(1): 333s gjournal(2): 309s Reading. grep -r on two src/ directories in parallel: UFS: 84s UFS+SU: 138s gjournal(1): 102s gjournal(2): 89s As you can see, even on one disk, untaring eight src.tgz is two times faster than UFS+SU. I've no idea why gjournal is faster in reading. There are a bunch of sysctls to tune gjournal (kern.geom.journal tree). When only one provider is given for both data and journal, the journal part is placed at the end of the provider, so one can use file system without journaling. If you use such configuration (one disk), it is better for performance to place journal before data, so you may want to create two partitions (eg. 2GB for ad0a and the rest for ad0d) and create gjournal this way: # gjournal label ad0d ad0a Enjoy! The work was sponsored by home.pl (http://home.pl). The work was made by Wheel LTD (http://www.wheel.pl). The work was tested in the netperf cluster. I want to thank Alexander Kabaev (kan@) for the help with VFS and Mike Tancsa for test hardware. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --rJwd6BRFiFCcLxzm Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFElqJlForvXbEpPzQRAtSFAJ9+Q+NjIqImiypsAFNG6bT6+dGu3wCgkOD0 q1HU94X2QsliV8rtIQRNt2s= =HWoE -----END PGP SIGNATURE----- --rJwd6BRFiFCcLxzm-- From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 13:37:08 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 866C016A474 for ; Mon, 19 Jun 2006 13:37:08 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30302.mail.mud.yahoo.com (web30302.mail.mud.yahoo.com [68.142.200.95]) by mx1.FreeBSD.org (Postfix) with SMTP id D06E143D49 for ; Mon, 19 Jun 2006 13:37:07 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 63799 invoked by uid 60001); 19 Jun 2006 13:37:06 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=KIXAjzxgVoVxAHYyberrno/uCLF82sMDszLDE9CAwG7BMlnhuFo7ihE9gYUnxwX6NsIKqyPnjEbRKDhTjn6+581PWdIF/dOagKysRBBSfGBXvGMWretgd+A5Z/M6TPbfj9crsUoWCZft8iJgyQOn3eylb1fseTQ1MuqDg1hj6cE= ; Message-ID: <20060619133706.63795.qmail@web30302.mail.mud.yahoo.com> Received: from [213.54.84.110] by web30302.mail.mud.yahoo.com via HTTP; Mon, 19 Jun 2006 06:37:06 PDT Date: Mon, 19 Jun 2006 06:37:06 -0700 (PDT) From: "R. B. Riddick" To: Pawel Jakub Dawidek In-Reply-To: <20060619131101.GD1130@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-fs@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 13:37:08 -0000 --- Pawel Jakub Dawidek wrote: > 2. If we have file system, synchronize it. > 3. Mark file system as clean. > 4. Block all write requests to the file system. > Shouldn't we do 4. before 2.? Do we write _new_ blocks of a file via the journal or directly to the file system (I mean: If we have a new file, that grows sequentially from 0 bytes to 1GB, it would save some disc access, if we didn't write the data to the journal)? I like the BIO_FLUSH idea... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 14:35:13 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0DC9C16A518 for ; Mon, 19 Jun 2006 14:35:12 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4700143D49 for ; Mon, 19 Jun 2006 14:35:11 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id B5FB65131F; Mon, 19 Jun 2006 16:35:10 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id EE17150EA7; Mon, 19 Jun 2006 16:35:04 +0200 (CEST) Date: Mon, 19 Jun 2006 16:32:31 +0200 From: Pawel Jakub Dawidek To: "R. B. Riddick" Message-ID: <20060619143231.GF1130@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <20060619133706.63795.qmail@web30302.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DrWhICOqskFTAXiy" Content-Disposition: inline In-Reply-To: <20060619133706.63795.qmail@web30302.mail.mud.yahoo.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 14:35:13 -0000 --DrWhICOqskFTAXiy Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 19, 2006 at 06:37:06AM -0700, R. B. Riddick wrote: +> --- Pawel Jakub Dawidek wrote: +> > 2. If we have file system, synchronize it. +> > 3. Mark file system as clean. +> > 4. Block all write requests to the file system. +> > +> Shouldn't we do 4. before 2.? 4 is done via vfs_write_suspend() function, which synchronize file system once again. +> Do we write _new_ blocks of a file via the journal or directly to the fi= le +> system (I mean: If we have a new file, that grows sequentially from 0 by= tes to +> 1GB, it would save some disc access, if we didn't write the data to the +> journal)? As I said gjournal is below file system layer. It receives I/O requests only and cannot say which one is related to growing file, which has metadata inside, etc. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --DrWhICOqskFTAXiy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFElrV/ForvXbEpPzQRArG5AJwPSlBnmkANdK8BeVcc/Z7yyux+IQCgs8AO KOtRRf+sOspu7+/IF6PHXjs= =ypFh -----END PGP SIGNATURE----- --DrWhICOqskFTAXiy-- From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 15:14:00 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ED06216A4A5 for ; Mon, 19 Jun 2006 15:14:00 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 85B9C43E64 for ; Mon, 19 Jun 2006 15:12:04 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (fybura@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k5JFBrWG017900 for ; Mon, 19 Jun 2006 17:12:02 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k5JFBrnY017899; Mon, 19 Jun 2006 17:11:53 +0200 (CEST) (envelope-from olli) Date: Mon, 19 Jun 2006 17:11:53 +0200 (CEST) Message-Id: <200606191511.k5JFBrnY017899@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG In-Reply-To: X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.0-20051224 ("Ronay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Mon, 19 Jun 2006 17:12:02 +0200 (CEST) Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-fs@FreeBSD.ORG List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 15:14:01 -0000 Pawel Jakub Dawidek wrote: > GJournal was designed to journal GEOM providers, so it actually works > below file system layer, but it has hooks which allow to work with > file systems. In other words, gjournal is not file system-depended, > it can work probably with any file system with minimum knowledge > about it. I implemented only UFS support. Very cool. Thanks for providing journaling to FreeBSD. How "stable" should your code be considered? I assume it hasn't been subject to testing by a wider audience yet, so it should be considered "alpha" quality, right? I have a few questions ... First of all, is it possible to add journaling to an existing file system, i.e. without having to dump, newfs and restore? > How it works (in short). You may define one or two providers which > gjournal will use. If one provider is given, it will be used for both - > data and journal. If two providers are given, one will be used for data > and one for journal. In the case of only one provider given, how is the size of the journal specified? > When last > name is deleted, the file/directory is moved to the .deleted/ > directory and removed from there on last close. Is that directory located in the root of the file system, similar to the .snap directory for snapshots? (It reminds me of the similar case when files are deleted from an NFS server. If they're still held open by processes, they get renamed to .nfsXXXX within the original directory. I wish they would be moved into a fixed directory instead, too.) > [...] > Reading. grep -r on two src/ directories in parallel: > UFS: 84s > UFS+SU: 138s > gjournal(1): 102s > gjournal(2): 89s > > As you can see, even on one disk, untaring eight src.tgz is two times > faster than UFS+SU. I've no idea why gjournal is faster in reading. Maybe it is because of the atime updates. You'll get a lot of them when grepping recursively on the src tree. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "Clear perl code is better than unclear awk code; but NOTHING comes close to unclear perl code" (taken from comp.lang.awk FAQ) From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 15:37:30 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8EFB016A484 for ; Mon, 19 Jun 2006 15:37:30 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30312.mail.mud.yahoo.com (web30312.mail.mud.yahoo.com [68.142.201.230]) by mx1.FreeBSD.org (Postfix) with SMTP id 9B10643D46 for ; Mon, 19 Jun 2006 15:37:29 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 97390 invoked by uid 60001); 19 Jun 2006 15:37:29 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=ZaGEWRJRjtkO1iHBi3QUYXGFI/SRrUFu992aIfu7tHPsosqK/avFP36umVMxpvmaqcbTlviJazocYRfqJVMk9WMgY/weTHTeZ6dZSwW1DIs0YV4THKjLo6xRvP0U8KtTJMd0DoqxWiYhDXOEx88LvrDIbBt4kmeh+WMOrA3rrJs= ; Message-ID: <20060619153729.97388.qmail@web30312.mail.mud.yahoo.com> Received: from [213.54.84.110] by web30312.mail.mud.yahoo.com via HTTP; Mon, 19 Jun 2006 08:37:29 PDT Date: Mon, 19 Jun 2006 08:37:29 -0700 (PDT) From: "R. B. Riddick" To: Pawel Jakub Dawidek In-Reply-To: <20060619143231.GF1130@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-fs@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 15:37:30 -0000 --- Pawel Jakub Dawidek wrote: > On Mon, Jun 19, 2006 at 06:37:06AM -0700, R. B. Riddick wrote: > +> --- Pawel Jakub Dawidek wrote: > +> > 2. If we have file system, synchronize it. > +> > 3. Mark file system as clean. > +> > 4. Block all write requests to the file system. > +> > > +> Shouldn't we do 4. before 2.? > > 4 is done via vfs_write_suspend() function, which synchronize file > system once again. > OK. Does vfs_write_suspend mark the file system as clean again, too? I mean: What if bewteen step 3 and 4 someone makes the file system dirty? > As I said gjournal is below file system layer. It receives I/O requests > only and cannot say which one is related to growing file, which has > metadata inside, etc. > Yeah, below -- as u said in the email, I responded to... I thought, the file system code supports gjournal somehow in that point, too... And I was too lazy to look into the patch u provided... Might it be possible to get some more optional assistance by the file system? Maybe we could provide a /dev/ad0.gjournal and a /dev/ad0.fs-data, which would break the GEOM concept a little bit...? *sigh* Maybe the file system code could use negative block numbers or high block numbers (somehow like a hint: U can write it right to the file system without any worries)? Of course those quick-write-through blocks should not be subject to a remove operation in the currently active journal area (I mean just in case, the system crashes while that journal is still active)... *sigh* I hope, I do not disturb the list too much... But I still like the BIO_FLUSH... -Arne --- Arne slow sometimes... :-) __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 16:20:42 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 400F016A47E; Mon, 19 Jun 2006 16:20:42 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3D9EA43D5E; Mon, 19 Jun 2006 16:20:38 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [10.10.3.185] ([69.15.205.254]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k5JGKGAY084368; Mon, 19 Jun 2006 10:20:22 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <4496CEBB.4020200@samsco.org> Date: Mon, 19 Jun 2006 10:20:11 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Eric Anderson References: <449606B8.5080106@centtech.com> In-Reply-To: <449606B8.5080106@centtech.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=3.8 tests=none autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: freebsd-fs@freebsd.org, Ensel Sharon , freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not ... RESOLVED X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 16:20:42 -0000 Eric Anderson wrote: > Ensel Sharon wrote: > >> >> On Mon, 19 Jun 2006, Jahilliya wrote: >> >>>> Ok, aac is in the dmesg. >>>> >>>> I can see both 2820sa cards in the dmesg, and I see no errors, etc. - >>>> there are just no drives listed in dmesg. >>>> >>>> My setup is 8 500 Gb drives in a single raid-6 array, size ~2.8TB. Any >>>> problems with that ? Perhaps sysinstall cannot deal with a >2TB drive >>>> ? There are no other drives in the system besides the single 2.8TB >>>> raid 6 >>>> array ... >>>> >>> Have you got any other drives you can attach to the raid? >>> >>> If so, disconnect the 8 drives connected, connect up a couple that are >>> not part of the raid and configure them as a simple raid 1 and see if >>> the installers sees that raid. Or try any combination in drives to >>> bring the raid size down below 2TB (I'm sure this limitation has been >>> fixed.) >> >> >> >> Ok, the answer is that it has not been fixed. >> >> 6.1 sysinstall does in fact see both 2820sa controllers, and when I >> put in >> a single 160GB sata drive, it does see that single drive and I can >> install >> onto it, etc. >> >> Sysinstall does _not_ see my 2.7TB raid6 array. I suspect that if it >> were >> smaller than 2TB, it would see it correctly. >> >> I have a number of options with which to deal with this, all of which >> involve either wasting money or wasting disk space. Fantastic. > > > Right - FreeBSD doesn't recognize >2TB LUNs. Wrong on several counts. First, the AAC driver does not present arrays to the system as SCSI LUNs. The traditional 2TB limit with 12 byte CDB issue simply doesn't exist with this driver. Second, the FreeBSD SCSI layer knows how to issue 16 byte CDBs to access >2TB, assuming that the target understands the 16-byte protocol. So no, there is no 2TB limit inherent to FreeBSD. The only limit is with individual drivers and with hardware. Scott From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 16:29:55 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DCF4316A474; Mon, 19 Jun 2006 16:29:55 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 009F243D72; Mon, 19 Jun 2006 16:29:25 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [10.10.3.185] ([69.15.205.254]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k5JGTHGU084489; Mon, 19 Jun 2006 10:29:23 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <4496D0D8.8040705@samsco.org> Date: Mon, 19 Jun 2006 10:29:12 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Ensel Sharon References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=3.8 tests=none autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not allowing installation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 16:29:56 -0000 Ensel Sharon wrote: > I did all my due diligence, I contacted freebsd-fs and _made sure_ that > even though the 2820sa is not listed by name in the HCL, that I could take > a 6.1-RELEASE cd and install freebsd on a 2820sa. > > I was _assured_ that these cards are supported in 6.1-RELEASE, that all is > well, and I could install and that was that. > No, you were not assured, at least not by when we discussed this last week. I told you that I personally did not guarantee that it worked, only that I had heard reports that it did. You have two variables here. One is that it's an array that is larger than what the aac driver has supported in the past. Second is that it's RAID-6. Both of these variables should be handled by the aac driver update that happened last year, but again, I couldn't validate it, so I can only go with the reports of others. As others suggested, you need to experiment with simplier configurations. This will help us identify the cause and hopefully implement a fix. No one is asking you to throw away money or resources. Since you've already done the simple test with a single drive, could you do the following two tests: 1. RAID-5, full size (whatever >2TB value you were talking about). 2. RAID-6, <2TB. From there, I'll figure out what needs to be done to get it fully working for you. Scott From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 18:05:59 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9917116A492 for ; Mon, 19 Jun 2006 18:05:59 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id E8A7B43D49 for ; Mon, 19 Jun 2006 18:05:58 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 7366C51852; Mon, 19 Jun 2006 20:05:57 +0200 (CEST) Received: from localhost (djw84.neoplus.adsl.tpnet.pl [83.24.0.84]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 9914D50EA7; Mon, 19 Jun 2006 20:05:50 +0200 (CEST) Date: Mon, 19 Jun 2006 20:03:16 +0200 From: Pawel Jakub Dawidek To: "R. B. Riddick" Message-ID: <20060619180316.GA3460@garage.freebsd.pl> References: <20060619143231.GF1130@garage.freebsd.pl> <20060619153729.97388.qmail@web30312.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="u3/rZRmxL6MmkK24" Content-Disposition: inline In-Reply-To: <20060619153729.97388.qmail@web30312.mail.mud.yahoo.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 18:05:59 -0000 --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 19, 2006 at 08:37:29AM -0700, R. B. Riddick wrote: +> --- Pawel Jakub Dawidek wrote: +> > On Mon, Jun 19, 2006 at 06:37:06AM -0700, R. B. Riddick wrote: +> > +> --- Pawel Jakub Dawidek wrote: +> > +> > 2. If we have file system, synchronize it. +> > +> > 3. Mark file system as clean. +> > +> > 4. Block all write requests to the file system. +> > +> > +> > +> Shouldn't we do 4. before 2.? +> >=20 +> > 4 is done via vfs_write_suspend() function, which synchronize file +> > system once again. +> > +> OK. Does vfs_write_suspend mark the file system as clean again, too? +> I mean: What if bewteen step 3 and 4 someone makes the file system dirty? The order of those points isn't really exact. It was more to illustrate what's going on in there. The order in the code is correct. The file system is marked as clean when there cannot be more write requests and when file system is fully synchronized. +> > As I said gjournal is below file system layer. It receives I/O requests +> > only and cannot say which one is related to growing file, which has +> > metadata inside, etc. +> >=20 +> Yeah, below -- as u said in the email, I responded to... +> I thought, the file system code supports gjournal somehow in that point,= too... +> And I was too lazy to look into the patch u provided... +>=20 +> Might it be possible to get some more optional assistance by the file sy= stem? +> Maybe we could provide a /dev/ad0.gjournal and a /dev/ad0.fs-data, which= would +> break the GEOM concept a little bit...? *sigh* +> Maybe the file system code could use negative block numbers or high block +> numbers (somehow like a hint: U can write it right to the file system wi= thout +> any worries)? +> Of course those quick-write-through blocks should not be subject to a re= move +> operation in the currently active journal area (I mean just in case, the= system +> crashes while that journal is still active)... It's quite complex and not really nice to store such informations inside bio structure, but I hope current gjournal is only a start, there is a lot of room for improvements. For example synchronizing file system isn't cheap, but before gjournal its speed wasn't really important, now it is more important, and speeding it up will speed up gjournal. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --u3/rZRmxL6MmkK24 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFElubkForvXbEpPzQRArw+AKDCuu2y6bN9leuWLgimFsDcpSIldgCg5PJv 51ut2SUF970asHSskv+DMpY= =Fojd -----END PGP SIGNATURE----- --u3/rZRmxL6MmkK24-- From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 18:32:28 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2606F16A47F for ; Mon, 19 Jun 2006 18:32:28 +0000 (UTC) (envelope-from nike_d@cytexbg.com) Received: from mail.interbgc.com (mx04.interbgc.com [217.9.224.231]) by mx1.FreeBSD.org (Postfix) with SMTP id BF6A643D46 for ; Mon, 19 Jun 2006 18:32:25 +0000 (GMT) (envelope-from nike_d@cytexbg.com) Received: (qmail 76788 invoked from network); 19 Jun 2006 18:32:24 -0000 Received: from nike_d@cytexbg.com by keeper.interbgc.com by uid 1002 with qmail-scanner-1.14 (uvscan: v4.2.40/v4374. spamassassin: 2.63. Clear:SA:0(0.1/8.0):. Processed in 3.703325 secs); 19 Jun 2006 18:32:24 -0000 X-Spam-Status: No, hits=0.1 required=8.0 Received: from niked.ddns.cablebg.net (HELO tormentor.totalterror.net) (85.130.14.211) by mx04.interbgc.com with SMTP; 19 Jun 2006 18:32:19 -0000 Received: (qmail 6230 invoked from network); 19 Jun 2006 18:32:19 -0000 Received: from unknown (HELO ?127.0.0.1?) (10.0.0.3) by tormentor.totalterror.net with SMTP; 19 Jun 2006 18:32:19 -0000 Message-ID: <4496EDB2.5040706@cytexbg.com> Date: Mon, 19 Jun 2006 21:32:18 +0300 From: Niki Denev User-Agent: Thunderbird 1.5.0.4 (Windows/20060516) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20060619131101.GD1130@garage.freebsd.pl> In-Reply-To: <20060619131101.GD1130@garage.freebsd.pl> X-Enigmail-Version: 0.94.0.0 OpenPGP: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 18:32:28 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Pawel Jakub Dawidek wrote: > Hello. > > For the last few months I have been working on gjournal project. > To stop confusion right here, I want to note, that this project is not > related to gjournal project on which Ivan Voras was working on the > last SoC (2005). > > The lack of journaled file system in FreeBSD was a tendon of achilles > for many years. We do have many file systems, but none with journaling: > - ext2fs (journaling is in ext3fs), > - XFS (read-only), > - ReiserFS (read-only), > - HFS+ (read-write, but without journaling), > - NTFS (read-only). > > GJournal was designed to journal GEOM providers, so it actually works > below file system layer, but it has hooks which allow to work with > file systems. In other words, gjournal is not file system-depended, > it can work probably with any file system with minimum knowledge > about it. I implemented only UFS support. > > The patches are here: > > http://people.freebsd.org/~pjd/patches/gjournal.patch (for HEAD) > http://people.freebsd.org/~pjd/patches/gjournal6.patch (for RELENG_6) > > To patch your sources you need to: > > # cd /usr/src > # mkdir sbin/geom/class/journal sys/geom/journal sys/modules/geom/geom_journal > # patch < /path/to/gjournal.patch > > Add 'options UFS_GJOURNAL' to your kernel configuration file and > recompile kernel and world. > > How it works (in short). You may define one or two providers which > gjournal will use. If one provider is given, it will be used for both - > data and journal. If two providers are given, one will be used for data > and one for journal. > Every few seconds (you may define how many) journal is terminated and > marked as consistent and gjournal starts to copy data from it to the > data provider. In the same time new data are stored in new journal. > Let's call the moment in which journal is terminated as "journal switch". > Journal switch looks as follows: > 1. Start journal switch if we have timeout or if we run out of cache. > Don't perform journal switch if there were no write requests. > 2. If we have file system, synchronize it. > 3. Mark file system as clean. > 4. Block all write requests to the file system. > 5. Terminate the journal. > 6. Eventually wait if copying of the previous journal is not yet > finished. > 7. Send BIO_FLUSH request (if the given provider supports it). > 8. Mark new journal position on the journal provider. > 9. Unblock write requests. > 10. Start copying data from the terminated journal to the data provider. > > There were few things I needed to implement outside gjournal to make it > work reliable: > > - The BIO_FLUSH request. Currently we have three I/O requests: BIO_READ, > BIO_WRITE and BIO_DELETE. I added BIO_FLUSH, which means "flush your > write cache". The request is send always with the biggest bio_offset set > (mediasize of the destination provider), so it will work properly with > bioq_disksort(). The caller need to stop further I/O requests before > BIO_FLUSH return, so we don't have starvation effect. > The hard part is that is has to be implemented in every disk driver, > because flushing the cache is driver-depended operation. I implemented > it for ata(4) disks and amr(4). The good news is that it's easy. > GJournal can also work with providers that don't support BIO_FLUSH and > in my power-failure tests it worked well (no problems), but it depend > on fact, that gjournal cache is bigger than the controller cache, so it > is hard to call it reliable. > You can read in documentation to many journaled file systems, that you > should turn off write cache if you want to use it. This is not the case > for gjournal (especially when your disk driver does support BIO_FLUSH). > > The 'gjournal' mount option. To implement gjournal support in UFS I > needed to change the way of how deleted, but still open objects are > handled. Currently when file or directory is open and we deleted last > name which reference it, it will still be usable by those who keep it > open. When the last consumer closes it, the inode and blocks are freed. > On journal switch I cannot leave such objects, because after a crash > fsck(8) is not used to check the file system, so inode and blocks will > never be freed. When file system is mounted with 'gjournal' mount > option, such objects are not removed when they are open. When last > name is deleted, the file/directory is moved to the .deleted/ > directory and removed from there on last close. > This way, I can just clean the .deleted/ directory after a crash at > mount time. > > Quick start: > > # gjournal label /dev/ad0 > # gjournal load > # newfs /dev/ad0.journal > # mount -o async,gjournal /dev/ad0.journal /mnt > (yes, with gjournal 'async' is safe) > > Now, after a power failure or system crash no fsck is needed (yay!). > > There are two hacks in the current implementation, which I'd like to > reimplement. First is how 'gjournal' mount option is implemented. > There is a garbage collector thread which is responsible for deleting > objects from .deleted/ directory and it is using full paths. Because > of this when your mount point is /foo/bar/baz and you rename 'bar' to > something else, it will not work. This is not what is often done, but > definitely should be fixed and I'm working on it. The second hack is > related to communication between gjournal and file system. GJournal > decides when to make the switch and has to find file system which is > mounted on it. Looking for this file system is not nice and should be > reimplemented. > > There are some additional goods which came with gjournal. For example > if gjournal is configured over gmirror or graid3, even on power failure > or system crash, there is no need to synchronize mirror/raid3 device, > because data will be consistent. > > I spend a lot of time working on gjournal optimization. Because I've > few seconds before the data hit the data provider I can perform things > like combining smaller write requests into larger once, ignoring data > written twice to the same place, etc. > Because of this, operations on small files are quite fast. On the other > hand, operations on large files are slower, because I need to write the > data twice and there is no place for optimization. Here are some numbers. > gjournal(1) - the data provider and the journal provider on the same disk > gjournal(2) - the data provider and the journal provider on separate > disks > > Copying one large file: > UFS: 8s > UFS+SU: 8s > gjournal(1): 16s > gjournal(2): 14s > > Copying eight large files in parallel: > UFS: 120s > UFS+SU: 120s > gjournal(1): 184s > gjournal(2): 165s > > Untaring eight src.tgz in parallel: > UFS: 791s > UFS+SU: 650s > gjournal(1): 333s > gjournal(2): 309s > > Reading. grep -r on two src/ directories in parallel: > UFS: 84s > UFS+SU: 138s > gjournal(1): 102s > gjournal(2): 89s > > As you can see, even on one disk, untaring eight src.tgz is two times > faster than UFS+SU. I've no idea why gjournal is faster in reading. > > There are a bunch of sysctls to tune gjournal (kern.geom.journal tree). > > When only one provider is given for both data and journal, the journal > part is placed at the end of the provider, so one can use file system > without journaling. If you use such configuration (one disk), it is > better for performance to place journal before data, so you may want to > create two partitions (eg. 2GB for ad0a and the rest for ad0d) and > create gjournal this way: > > # gjournal label ad0d ad0a > > Enjoy! > > The work was sponsored by home.pl (http://home.pl). > > The work was made by Wheel LTD (http://www.wheel.pl). > The work was tested in the netperf cluster. > > I want to thank Alexander Kabaev (kan@) for the help with VFS and > Mike Tancsa for test hardware. > Wow, this looks pretty cool! I wonder if it's possible to use gjournal on existing file system with the journal on a vnode/(swap?) backed md(4) device? (i want to test on a existing installation without free unpartitioned space) And if it is possible, how can i do this for the root filesystem? i'll need the md(4) device before mounting of the root fs which seems hard/impossible? What's going to happen if my root mount is gjournal labeled and has gjournal option in fstab but at boot time the journal GEOM provider does not exist? Thanks for the great work! When finished, this will certainly make FreeBSD much more competitive :) - --niki -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFElu2yHNAJ/fLbfrkRAsVBAKChRFMVLuivXYR1NM3b0u9iVe72uwCfdzH0 DvdjEZwOKjuZu4UV+toVpwo= =+qj/ -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 18:58:01 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5FD7F16A5B0; Mon, 19 Jun 2006 18:58:01 +0000 (UTC) (envelope-from brdavis@odin.ac.hmc.edu) Received: from odin.ac.hmc.edu (Odin.AC.HMC.Edu [134.173.32.75]) by mx1.FreeBSD.org (Postfix) with ESMTP id EF73243D46; Mon, 19 Jun 2006 18:58:00 +0000 (GMT) (envelope-from brdavis@odin.ac.hmc.edu) Received: from odin.ac.hmc.edu (localhost.localdomain [127.0.0.1]) by odin.ac.hmc.edu (8.13.0/8.13.0) with ESMTP id k5JIw0pg023502; Mon, 19 Jun 2006 11:58:00 -0700 Received: (from brdavis@localhost) by odin.ac.hmc.edu (8.13.0/8.13.0/Submit) id k5JIw0Hd023501; Mon, 19 Jun 2006 11:58:00 -0700 Date: Mon, 19 Jun 2006 11:58:00 -0700 From: Brooks Davis To: Pawel Jakub Dawidek Message-ID: <20060619185800.GA22546@odin.ac.hmc.edu> References: <20060619131101.GD1130@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Qxx1br4bt0+wmkIi" Content-Disposition: inline In-Reply-To: <20060619131101.GD1130@garage.freebsd.pl> User-Agent: Mutt/1.4.1i X-Virus-Scanned: by amavisd-new Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 18:58:01 -0000 --Qxx1br4bt0+wmkIi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 19, 2006 at 03:11:01PM +0200, Pawel Jakub Dawidek wrote: >=20 > How it works (in short). You may define one or two providers which > gjournal will use. If one provider is given, it will be used for both - > data and journal. If two providers are given, one will be used for data > and one for journal. > Every few seconds (you may define how many) journal is terminated and > marked as consistent and gjournal starts to copy data from it to the > data provider. In the same time new data are stored in new journal. > Let's call the moment in which journal is terminated as "journal switch". Cool solution! I think I'll give this a try on my redundent mirror server at work. I'd be curious to see how gjournal performs with the journal on a battery backed ram disk like the gigabyte i-RAM: http://www.giga-byte.com/Products/Storage/Products_Overview.aspx?ProductID= =3D2180&ProductName=3DGC-RAMDISK It seems like that could reduce or eliminate many of the performance issues in practice. -- Brooks --Qxx1br4bt0+wmkIi Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQFElvO3XY6L6fI4GtQRAleqAKDQVc6j/LfjPPt4vcvqzz7osVfQbACdFErw Jo9Xaa7JuVQmsQw2u4ohwSQ= =q1/p -----END PGP SIGNATURE----- --Qxx1br4bt0+wmkIi-- From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 19:55:24 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D099016A479 for ; Mon, 19 Jun 2006 19:55:24 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30303.mail.mud.yahoo.com (web30303.mail.mud.yahoo.com [68.142.200.96]) by mx1.FreeBSD.org (Postfix) with SMTP id 8915243D4C for ; Mon, 19 Jun 2006 19:55:23 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 8453 invoked by uid 60001); 19 Jun 2006 19:55:22 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=CVzHRyvC2+CVR1y6hNyEnzKQ7npUfVHqqHx5eyTridSjaxn3xP7bzti6DORpLyG+P/9ckc3MgI5K0j1R/5QaveoEAtQe410fJKFmaFLmh94om4u06I9O6ByjGVYq2xMgzsh3Mwj/a8iSNCIb9isoTWUTNY1ey0UAZRiXOaPYrng= ; Message-ID: <20060619195522.8451.qmail@web30303.mail.mud.yahoo.com> Received: from [213.54.95.153] by web30303.mail.mud.yahoo.com via HTTP; Mon, 19 Jun 2006 12:55:22 PDT Date: Mon, 19 Jun 2006 12:55:22 -0700 (PDT) From: "R. B. Riddick" To: Niki Denev , freebsd-fs@FreeBSD.org In-Reply-To: <4496EDB2.5040706@cytexbg.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 19:55:24 -0000 --- Niki Denev wrote: > I wonder if it's possible to use gjournal on > existing file system with the journal on a vnode/(swap?) backed md(4) device? > (i want to test on a existing installation without free unpartitioned space) > Hmm... I think that would be a not so good idea. Proof: Consider the follow scenario: 1. journal area A is nearly full, so we switch to journal area B 2. switching to B completed 3. starting to execute update order-requests in area A 4. before requests in area A is completely executed, the power fails ==> journal A and B is lost, but file system is not necessarily in a consistent state (since A has not been executed completely) Btw.: I found another big advantage (beside the BIO_FLUSH): NEVER FSCK AGAIN! ;-) -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Mon Jun 19 22:12:10 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2658F16A481 for ; Mon, 19 Jun 2006 22:12:10 +0000 (UTC) (envelope-from hardware@ee.ccny.cuny.edu) Received: from ees1s0.engr.ccny.cuny.edu (ees1s0.engr.ccny.cuny.edu [134.74.16.1]) by mx1.FreeBSD.org (Postfix) with SMTP id 857B743D4C for ; Mon, 19 Jun 2006 22:12:09 +0000 (GMT) (envelope-from hardware@ee.ccny.cuny.edu) Received: (qmail 20675 invoked from network); 19 Jun 2006 22:20:45 -0000 Received: from unknown (HELO eesfc20) (134.74.228.20) by ees1s0.engr.ccny.cuny.edu with SMTP; 19 Jun 2006 22:20:45 -0000 Date: Mon, 19 Jun 2006 18:11:38 -0400 (EDT) From: hardware@ee.ccny.cuny.edu X-X-Sender: hardware@eesfc20.engr.ccny.cuny.edu To: freebsd-fs@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: ext2fs marked dirty when mounted read-only X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2006 22:12:10 -0000 Dear FreeBSD Filesystem List, When I uncleanly unmount an ext2fs filesystem that is mounted read-only it needs to be fsck'ed before the kernel will allow it to remounted. kernel: 6.1-RC1 FreeBSD 6.1-RC1 disk: da0 at umass-sim0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 476940MB (976773168 512 byte sectors: 255H 63S/T 60801C) When I try to mount the ext2fs filesystem in read-only mode, I get this: WARNING: R/W mount denied. Filesystem is not clean - run fsck This problem can be reproduced on my system by the following steps 1) mount -t ext2fs -o ro /dev/da0 /mnt # da0 is external USB 2) ls /mnt 3) UNPLUG THE EXTERNAL USB DISK 4) reboot 5) mount -t ext2fs -o ro /dev/da0 /mnt thanks, -kurt From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 06:35:27 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4462116A47A; Tue, 20 Jun 2006 06:35:27 +0000 (UTC) (envelope-from rodrigc@crodrigues.org) Received: from sccrmhc15.comcast.net (sccrmhc15.comcast.net [63.240.77.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id EFA8E43D46; Tue, 20 Jun 2006 06:35:25 +0000 (GMT) (envelope-from rodrigc@crodrigues.org) Received: from c-71-233-168-2.hsd1.ma.comcast.net ([71.233.168.2]) by comcast.net (sccrmhc15) with ESMTP id <200606200635240150091q9ie>; Tue, 20 Jun 2006 06:35:24 +0000 Received: from c-71-233-168-2.hsd1.ma.comcast.net (localhost [127.0.0.1]) by c-71-233-168-2.hsd1.ma.comcast.net (8.13.6/8.13.1) with ESMTP id k5K6ZP7e011559; Tue, 20 Jun 2006 02:35:25 -0400 (EDT) (envelope-from rodrigc@c-71-233-168-2.hsd1.ma.comcast.net) Received: (from rodrigc@localhost) by c-71-233-168-2.hsd1.ma.comcast.net (8.13.6/8.13.1/Submit) id k5K6ZPF0011558; Tue, 20 Jun 2006 02:35:25 -0400 (EDT) (envelope-from rodrigc) Date: Tue, 20 Jun 2006 02:35:25 -0400 From: Craig Rodrigues To: Pawel Jakub Dawidek Message-ID: <20060620063525.GA11441@crodrigues.org> References: <20060619131101.GD1130@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060619131101.GD1130@garage.freebsd.pl> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 06:35:27 -0000 On Mon, Jun 19, 2006 at 03:11:01PM +0200, Pawel Jakub Dawidek wrote: > http://people.freebsd.org/~pjd/patches/gjournal.patch (for HEAD) > http://people.freebsd.org/~pjd/patches/gjournal6.patch (for RELENG_6) I would recommend that you not introduce a new MNT_GJOURNAL flag to , and that instead you just pass -o gjournal directly down into nmount(). In kernel code, you can use vfs_flagopt()/vfs_getopt() to determine if you have this mount option or not. The mount(8) userland utility would not need any modifications, since it just passes -o options down to nmount(). gjournal looks very interesting! -- Craig Rodrigues rodrigc@crodrigues.org From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 08:39:17 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 379CC16A492; Tue, 20 Jun 2006 08:39:17 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 89D5543D5C; Tue, 20 Jun 2006 08:39:15 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id C469051814; Tue, 20 Jun 2006 10:39:13 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id F252A51307; Tue, 20 Jun 2006 10:39:06 +0200 (CEST) Date: Tue, 20 Jun 2006 10:36:33 +0200 From: Pawel Jakub Dawidek To: Niki Denev Message-ID: <20060620083632.GB6235@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <4496EDB2.5040706@cytexbg.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+g7M9IMkV8truYOl" Content-Disposition: inline In-Reply-To: <4496EDB2.5040706@cytexbg.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 08:39:17 -0000 --+g7M9IMkV8truYOl Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 19, 2006 at 09:32:18PM +0300, Niki Denev wrote: +> I wonder if it's possible to use gjournal on +> existing file system with the journal on a vnode/(swap?) backed md(4) de= vice? +> (i want to test on a existing installation without free unpartitioned sp= ace) Depend on what do you want to test. If you just want to look around, swap-backed md(4) device for journal should be fine. If you want to perform some crash tests, you may want to turn off the swap and use its provider for journal directly (without md(4)), so it will be available after a reboot. You can configure gjournal on an existing file system, but, as always, the last sector will be used for metadata. For example, you have your file system on ad0s1d and swap on ad0s1b. You can try to configure gjournal this way: # swapoff /dev/ad0s1b # umount /dev/ad0s1d # gjournal label ad0s1d ad0s1b Your swap should have at least 2GB if your file system will be heavy loaded. Be warned that this will overwrite the last sector on ad0s1d, which should be safe, but you never know. +> And if it is possible, how can i do this for the root filesystem? i'll n= eed the md(4) +> device before mounting of the root fs which seems hard/impossible? +> What's going to happen if my root mount is gjournal labeled and has gjou= rnal option in +> fstab but at boot time the journal GEOM provider does not exist? I forgot to mention this in my initial mail. This is not yet possible to use gjournal for the root file system. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --+g7M9IMkV8truYOl Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEl7OQForvXbEpPzQRAjDnAJ4zBXaKq7QO6h5tshc4Uc+Z+GeLXwCgjzMw 1lTAcJbB+zfgqC8VzF4DwOg= =Qkqy -----END PGP SIGNATURE----- --+g7M9IMkV8truYOl-- From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 08:43:51 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5C73D16A47A; Tue, 20 Jun 2006 08:43:51 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id B368243D53; Tue, 20 Jun 2006 08:43:50 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 52A9251388; Tue, 20 Jun 2006 10:43:49 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 3C9E951307; Tue, 20 Jun 2006 10:43:44 +0200 (CEST) Date: Tue, 20 Jun 2006 10:41:10 +0200 From: Pawel Jakub Dawidek To: Brooks Davis Message-ID: <20060620084110.GC6235@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <20060619185800.GA22546@odin.ac.hmc.edu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2/5bycvrmDh4d1IB" Content-Disposition: inline In-Reply-To: <20060619185800.GA22546@odin.ac.hmc.edu> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 08:43:51 -0000 --2/5bycvrmDh4d1IB Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jun 19, 2006 at 11:58:00AM -0700, Brooks Davis wrote: +> On Mon, Jun 19, 2006 at 03:11:01PM +0200, Pawel Jakub Dawidek wrote: +> >=20 +> > How it works (in short). You may define one or two providers which +> > gjournal will use. If one provider is given, it will be used for both - +> > data and journal. If two providers are given, one will be used for data +> > and one for journal. +> > Every few seconds (you may define how many) journal is terminated and +> > marked as consistent and gjournal starts to copy data from it to the +> > data provider. In the same time new data are stored in new journal. +> > Let's call the moment in which journal is terminated as "journal switc= h". +>=20 +> Cool solution! I think I'll give this a try on my redundent mirror +> server at work. I'd be curious to see how gjournal performs with the +> journal on a battery backed ram disk like the gigabyte i-RAM: +>=20 +> http://www.giga-byte.com/Products/Storage/Products_Overview.aspx?Product= ID=3D2180&ProductName=3DGC-RAMDISK I am curious too:) But as I said, there is still a lot of room for performance improvements. The bottleneck currently is file system synchronization, I think. I hope our VFS gurus will look into VFS_SYNC() optimizations. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --2/5bycvrmDh4d1IB Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEl7SlForvXbEpPzQRAqB1AJ45J7spbBRtAcRzlA/ZhwzgNLz6PgCgmwRx y9ph5z7m5towyeZISTAnQik= =PdyB -----END PGP SIGNATURE----- --2/5bycvrmDh4d1IB-- From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 08:58:00 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 78D3416A47D for ; Tue, 20 Jun 2006 08:58:00 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7394143D6E for ; Tue, 20 Jun 2006 08:57:59 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (sxeryj@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k5K8vqQa052502 for ; Tue, 20 Jun 2006 10:57:57 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k5K8vqh9052501; Tue, 20 Jun 2006 10:57:52 +0200 (CEST) (envelope-from olli) Date: Tue, 20 Jun 2006 10:57:52 +0200 (CEST) Message-Id: <200606200857.k5K8vqh9052501@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG In-Reply-To: X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.0-20051224 ("Ronay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Tue, 20 Jun 2006 10:57:57 +0200 (CEST) Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-fs@FreeBSD.ORG List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 08:58:00 -0000 Niki Denev wrote: > [long quote snipped] > I wonder if it's possible to use gjournal on > existing file system with the journal on a vnode/(swap?) backed md(4) device? > (i want to test on a existing installation without free unpartitioned space) I think it should be possible. First umount the file system, then create an md(4) device with mdconfig(8), then specify the /dev/md* device as the journal provider to gjournal, and finally mount the file system again with the "gjournal" option. However, note that you should use a vnode-backed md(4) device only. If you use a malloc(9)-backed or swap- backed one, you will lose the journal upon a crash or power failure, defeating the whole purpose of gjournal. It is probably a good idea to put the backing file for the md(4) device on a different file system than the one being journaled. > And if it is possible, how can i do this for the root filesystem? i'll need the md(4) > device before mounting of the root fs which seems hard/impossible? > What's going to happen if my root mount is gjournal labeled and has gjournal option in > fstab but at boot time the journal GEOM provider does not exist? I guess that doesn't currently work, but it shouldn't be too difficult to implement. Pawel, please correct me if I'm wrong. - First, the kernel mounts the root file system read- only, which should always work, no matter if it is dirty or clean, or if it was gjournalled before or not. - Then -- as usual -- the kernel starts /sbin/init which runs /etc/rc (assuming going multi-user mode), which starts the rcNG framework. - In the course of performing the rcNG stuff, an md(4) device for the gjournal has to be configured (just similar to the way md(4) devices are configured for /tmp and other such file systems). - After that, the root file system is re-mounted (-u) with the gjournal and read-write options. With proper rcNG ordering it should work fine. > When finished, this will certainly make FreeBSD much more competitive :) Definitely. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. One Unix to rule them all, One Resolver to find them, One IP to bring them all and in the zone to bind them. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 09:05:40 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D03E616A494 for ; Tue, 20 Jun 2006 09:05:40 +0000 (UTC) (envelope-from ale@FreeBSD.org) Received: from andxor.it (relay.andxor.it [195.223.2.3]) by mx1.FreeBSD.org (Postfix) with SMTP id 3387E43D55 for ; Tue, 20 Jun 2006 09:05:38 +0000 (GMT) (envelope-from ale@FreeBSD.org) Received: (qmail 56922 invoked from network); 20 Jun 2006 09:05:33 -0000 Received: from unknown (HELO ?192.168.2.5?) (192.168.2.5) by andxor.it with SMTP; 20 Jun 2006 09:05:33 -0000 Message-ID: <4497BA5B.8050805@FreeBSD.org> Date: Tue, 20 Jun 2006 11:05:31 +0200 From: Alex Dupre User-Agent: Thunderbird 1.5.0.4 (X11/20060606) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20060619131101.GD1130@garage.freebsd.pl> <4496EDB2.5040706@cytexbg.com> <20060620083632.GB6235@garage.freebsd.pl> In-Reply-To: <20060620083632.GB6235@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 09:05:40 -0000 Pawel Jakub Dawidek wrote: > I forgot to mention this in my initial mail. This is not yet possible to > use gjournal for the root file system. Even if the machine boots from another device and the gjournal kernel module is loaded before mounting the root filesystem? -- Alex Dupre From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 09:15:11 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DA94E16A47A; Tue, 20 Jun 2006 09:15:11 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0A48343D64; Tue, 20 Jun 2006 09:14:57 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id AEEDF51388; Tue, 20 Jun 2006 11:14:56 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 586AA50E81; Tue, 20 Jun 2006 11:14:51 +0200 (CEST) Date: Tue, 20 Jun 2006 11:12:17 +0200 From: Pawel Jakub Dawidek To: Alex Dupre Message-ID: <20060620091216.GD6235@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <4496EDB2.5040706@cytexbg.com> <20060620083632.GB6235@garage.freebsd.pl> <4497BA5B.8050805@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WChQLJJJfbwij+9x" Content-Disposition: inline In-Reply-To: <4497BA5B.8050805@FreeBSD.org> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 09:15:12 -0000 --WChQLJJJfbwij+9x Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 20, 2006 at 11:05:31AM +0200, Alex Dupre wrote: +> Pawel Jakub Dawidek wrote: +> >I forgot to mention this in my initial mail. This is not yet possible to +> >use gjournal for the root file system. +>=20 +> Even if the machine boots from another device and the gjournal kernel mo= dule is loaded before mounting the root filesystem? Yes, even then, because mount(8) utility is responsible for cleaning =2Edeleted/ directory. This can be done when the file system is remounted read-write, but I just didn't have time to work on this yet. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --WChQLJJJfbwij+9x Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEl7vwForvXbEpPzQRAnUtAKDCxpIzoNix1nU0n+xLHGrrsBpMIACgnh/I oRKJQNNV+TRZ0RmJoR2ojP0= =4o5n -----END PGP SIGNATURE----- --WChQLJJJfbwij+9x-- From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 10:29:13 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6372516A479; Tue, 20 Jun 2006 10:29:13 +0000 (UTC) (envelope-from regnauld@x0.dk) Received: from x0.dk (x0.dk [62.242.165.154]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE76843D46; Tue, 20 Jun 2006 10:29:12 +0000 (GMT) (envelope-from regnauld@x0.dk) Received: from localhost (unknown [127.0.0.1]) by tetard.starbsd.org (Postfix) with ESMTP id 36F5B3565C; Tue, 20 Jun 2006 12:29:11 +0200 (CEST) Received: from tetard.starbsd.org ([127.0.0.1]) by localhost (tetard.starbsd.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 27908-05; Tue, 20 Jun 2006 12:29:10 +0200 (CEST) Received: by tetard.starbsd.org (Postfix, from userid 1001) id 4556E35667; Tue, 20 Jun 2006 12:29:10 +0200 (CEST) Date: Tue, 20 Jun 2006 12:29:10 +0200 From: Phil Regnauld To: Pawel Jakub Dawidek Message-ID: <20060620102910.GG27055@tetard.starbsd.org> References: <20060619131101.GD1130@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060619131101.GD1130@garage.freebsd.pl> X-Operating-System: FreeBSD 6.1-STABLE i386 Organization: *BSD User-Agent: Mutt/1.5.11 X-Virus-Scanned: amavisd-new at starbsd.org Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 10:29:13 -0000 On Mon, Jun 19, 2006 at 03:11:01PM +0200, Pawel Jakub Dawidek wrote: > > Copying one large file: > UFS: 8s > UFS+SU: 8s > gjournal(1): 16s > gjournal(2): 14s This is very very interesting work! I am definitely going to test this. I know this is too early to ask considering the optimizations that can be done, but do you have any idea how this would perform compared to ReiserFS on similar operations as the ones you benchmarked ? PS: is it me or is the patch missing a gjournal command, as invoked in your examples ? Cheers, Phil From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 13:30:25 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C6F2516A47B for ; Tue, 20 Jun 2006 13:30:25 +0000 (UTC) (envelope-from stas@310.ru) Received: from com1.ht-systems.ru (com1.ht-systems.ru [83.97.104.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0C8A8441F6 for ; Tue, 20 Jun 2006 13:30:24 +0000 (GMT) (envelope-from stas@310.ru) Received: from [80.250.160.136] (helo=fonon.realnet) by com1.ht-systems.ru with esmtpa (Exim 4.62) (envelope-from ) id 1FsgHp-0001oa-9U for freebsd-fs@freebsd.org; Tue, 20 Jun 2006 17:28:58 +0400 Received: from fonon.realnet (localhost [127.0.0.1]) by fonon.realnet (8.13.6/8.13.6) with ESMTP id k5KCbR4h057116 for ; Tue, 20 Jun 2006 16:40:45 +0400 (MSD) (envelope-from stas@fonon.realnet) Received: (from stas@localhost) by fonon.realnet (8.13.6/8.13.6/Submit) id k5KCbQBX057115; Tue, 20 Jun 2006 16:37:26 +0400 (MSD) (envelope-from stas) Date: Tue, 20 Jun 2006 16:37:18 +0400 From: Stanislav Sedov To: freebsd-fs@freebsd.org Message-Id: <20060620163718.26387be9.stas@310.ru> In-Reply-To: References: Organization: MBSD labs, Inc. X-Mailer: carrier-pigeon X-Operating-System: FreeBSD Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA1"; boundary="Signature=_Tue__20_Jun_2006_16_37_18_+0400_9zT__xfgwWD+j4rE" X-Spam-Flag: SKIP X-Spam-Yversion: Spamooborona 1.5.2 Subject: Re: ext2fs marked dirty when mounted read-only X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 13:30:25 -0000 --Signature=_Tue__20_Jun_2006_16_37_18_+0400_9zT__xfgwWD+j4rE Content-Type: text/plain; charset=KOI8-R Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, 19 Jun 2006 18:11:38 -0400 (EDT) hardware@ee.ccny.cuny.edu wrote: > Dear FreeBSD Filesystem List, >=20 > When I uncleanly unmount an ext2fs filesystem that is mounted read-only > it needs to be fsck'ed before the kernel will allow it to remounted. >=20 > kernel: 6.1-RC1 FreeBSD 6.1-RC1 > disk: da0 at umass-sim0 bus 0 target 0 lun 0 > da0: Fixed Direct Access SCSI-0 device > da0: 40.000MB/s transfers > da0: 476940MB (976773168 512 byte sectors: 255H 63S/T 60801C) >=20 > When I try to mount the ext2fs filesystem in read-only mode, I get this: >=20 > WARNING: R/W mount denied. Filesystem is not clean - run fsck >=20 > This problem can be reproduced on my system by the following steps >=20 > 1) mount -t ext2fs -o ro /dev/da0 /mnt # da0 is external USB > 2) ls /mnt > 3) UNPLUG THE EXTERNAL USB DISK > 4) reboot > 5) mount -t ext2fs -o ro /dev/da0 /mnt >=20 There is a small bug in ext2fs code. I'd experienced the same problem some time ago. Unfortunately, I have lost the patch. Try to change the line "if (fs->s_rd_only && !vfs_flagopt(opts, "ro", NULL, 0)) {" in ext2_vfsops.c to "if (fs->s_rd_only && !(mp->mnt_flag & MNT_RDONLY)) {" I suppose it will help. --=20 Stanislav Sedov MBSD labs, Inc. =F2=CF=D3=D3=C9=D1, =ED=CF=D3=CB=D7=C1 http://mbsd.msk.ru -------------------------------------------------------------------- If the facts don't fit the theory, change the facts. -- A. Einstein -------------------------------------------------------------------- PGP fingerprint: F21E D6CC 5626 9609 6CE2 A385 2BF5 5993 EB26 9581 --Signature=_Tue__20_Jun_2006_16_37_18_+0400_9zT__xfgwWD+j4rE Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEl+wGK/VZk+smlYERAkbYAJwJdh8bHPx+tJmGOz2yQa0Xn01k6gCfQiYi bGCKZ5yrO+jEozsjqIYjJQU= =Ohuh -----END PGP SIGNATURE----- --Signature=_Tue__20_Jun_2006_16_37_18_+0400_9zT__xfgwWD+j4rE-- From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 14:34:13 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F0EDA16A482 for ; Tue, 20 Jun 2006 14:34:13 +0000 (UTC) (envelope-from matt@genesi-usa.com) Received: from mail.genesippc.com (mithrandir.softwarenexus.net [66.98.186.96]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8A7B343D46 for ; Tue, 20 Jun 2006 14:34:13 +0000 (GMT) (envelope-from matt@genesi-usa.com) Received: from p54b0ffec.dip.t-dialin.net ([84.176.255.236] helo=yukito) by mail.genesippc.com with esmtpa (Exim 4.62 (FreeBSD)) (envelope-from ) id 1FshFM-0000LB-GL for freebsd-fs@freebsd.org; Tue, 20 Jun 2006 14:30:29 +0000 From: "Matt Sealey" To: Date: Tue, 20 Jun 2006 09:34:09 -0500 Organization: Genesi Message-ID: <00b101c69476$9a6686a0$99dfdfdf@bakuhatsu.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2869 Thread-Index: AcaUdpaaug5/duZMRBKrbssl/kADGA== Subject: On-disk format of UFS/UFS2 (for firmware implementation) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: matt@genesi-usa.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 14:34:14 -0000 Hello, I was wondering if there is any definitive, authoritative or even just comprehensive documentation on the on-disk format (superblock, nodes, file linking, extents..) of the UFS and UFS2 filesystems. We would like to implement it in firmware for booting FreeBSD (and NetBSD and eventually Solaris which uses some variant). I have the source code from sys/fs/ufs already but it is not the preferred method of implementing filesystem code. It only needs read support for now so it shouldn't be that complicated, but... Anyway. Any hints here? Thanks for any advice you can give, -- Matt Sealey Manager, Genesi, Developer Relations From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 14:48:58 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2BB9E16A479 for ; Tue, 20 Jun 2006 14:48:58 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30304.mail.mud.yahoo.com (web30304.mail.mud.yahoo.com [68.142.200.97]) by mx1.FreeBSD.org (Postfix) with SMTP id 0CAAF43D64 for ; Tue, 20 Jun 2006 14:48:39 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 93601 invoked by uid 60001); 20 Jun 2006 14:48:38 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=vPrCozFrXQkF3CyRgIncPGJA9n4efSf2LDJHBL7vVGk8iA7rD+CEd2yOWwCKzRV0gwgMOSXP/BN8EMCzO3FEDuilGc5j0yCEEKwmTmPOjkqVHgB9EmMoy4raNfd55DKvft/7cah6Wtgnw+RDDrbyCfDwcVJhkO54oq5wrh5ZHnw= ; Message-ID: <20060620144838.93599.qmail@web30304.mail.mud.yahoo.com> Received: from [213.54.95.153] by web30304.mail.mud.yahoo.com via HTTP; Tue, 20 Jun 2006 07:48:38 PDT Date: Tue, 20 Jun 2006 07:48:38 -0700 (PDT) From: "R. B. Riddick" To: matt@genesi-usa.com, freebsd-fs@freebsd.org In-Reply-To: <00b101c69476$9a6686a0$99dfdfdf@bakuhatsu.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: On-disk format of UFS/UFS2 (for firmware implementation) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 14:48:58 -0000 --- Matt Sealey wrote: > Solaris which uses some variant). I have the source code > from sys/fs/ufs already but it is not the preferred method > of implementing filesystem code. It only needs read support > for now so it shouldn't be that complicated, but... > > Anyway. Any hints here? > I say, does it sound like u r looking for src/sys/boot/common/ufsread.c? -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 15:14:09 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2FD1616A479 for ; Tue, 20 Jun 2006 15:14:09 +0000 (UTC) (envelope-from matt@genesi-usa.com) Received: from mail.genesippc.com (mithrandir.softwarenexus.net [66.98.186.96]) by mx1.FreeBSD.org (Postfix) with ESMTP id D957343D46 for ; Tue, 20 Jun 2006 15:14:08 +0000 (GMT) (envelope-from matt@genesi-usa.com) Received: from p54b0ffec.dip.t-dialin.net ([84.176.255.236] helo=yukito) by mail.genesippc.com with esmtpa (Exim 4.62 (FreeBSD)) (envelope-from ) id 1Fshry-0001Ew-Ps; Tue, 20 Jun 2006 15:10:24 +0000 From: "Matt Sealey" To: "'R. B. Riddick'" , Date: Tue, 20 Jun 2006 10:14:02 -0500 Organization: Genesi Message-ID: <00b401c6947c$2def2260$99dfdfdf@bakuhatsu.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2869 Thread-Index: AcaUeBqbzC36GoclQt2slj4LkJow6QAA5B2Q In-Reply-To: <20060620144838.93599.qmail@web30304.mail.mud.yahoo.com> Cc: Subject: RE: On-disk format of UFS/UFS2 (for firmware implementation) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: matt@genesi-usa.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 15:14:09 -0000 I'd prefer of course some design documentation (I refuse to believe the UFS2 disk format was designed entirely in source code form without paper/html reference documentation) What I have is src/sys/ufs/* src/lib/libstand/ufs.c and now src/sys/boot/common/ufsread.c This is a very good start but is there really not any paper doc or design reference or even an email thread on a mailing list when people were designing UFS2? My second question (since now we can poke around perhaps) is if there is a defined way to detect a "dangerously dedicated" disk in order to support these system configurations..? -- Matt Sealey Manager, Genesi, Developer Relations > -----Original Message----- > From: R. B. Riddick [mailto:arne_woerner@yahoo.com] > Sent: Tuesday, June 20, 2006 9:49 AM > To: matt@genesi-usa.com; freebsd-fs@freebsd.org > Subject: Re: On-disk format of UFS/UFS2 (for firmware implementation) > > --- Matt Sealey wrote: > > Solaris which uses some variant). I have the source code from > > sys/fs/ufs already but it is not the preferred method of > implementing > > filesystem code. It only needs read support for now so it > shouldn't be > > that complicated, but... > > > > Anyway. Any hints here? > > > I say, does it sound like u r looking for > src/sys/boot/common/ufsread.c? > > -Arne > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection > around http://mail.yahoo.com > From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 15:14:51 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E067D16A474 for ; Tue, 20 Jun 2006 15:14:51 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5558443D5F for ; Tue, 20 Jun 2006 15:14:48 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id D507D46BE6; Tue, 20 Jun 2006 11:14:45 -0400 (EDT) Date: Tue, 20 Jun 2006 16:14:45 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Matt Sealey In-Reply-To: <00b101c69476$9a6686a0$99dfdfdf@bakuhatsu.net> Message-ID: <20060620161319.N18295@fledge.watson.org> References: <00b101c69476$9a6686a0$99dfdfdf@bakuhatsu.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: On-disk format of UFS/UFS2 (for firmware implementation) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 15:14:52 -0000 On Tue, 20 Jun 2006, Matt Sealey wrote: > I was wondering if there is any definitive, authoritative or even just > comprehensive documentation on the on-disk format (superblock, nodes, file > linking, extents..) of the UFS and UFS2 filesystems. > > We would like to implement it in firmware for booting FreeBSD (and NetBSD > and eventually Solaris which uses some variant). I have the source code from > sys/fs/ufs already but it is not the preferred method of implementing > filesystem code. It only needs read support for now so it shouldn't be that > complicated, but... This isn't quite what you're asking for, but you might take a look at src/sys/boot/common/ufsread.c, which is used by the FreeBSD boot loader to load files from UFS. It uses include files from sys/ufs. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 15:42:35 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 48DC716A47C for ; Tue, 20 Jun 2006 15:42:35 +0000 (UTC) (envelope-from simon_freebsd_fs@bigblue.demon.co.uk) Received: from anchor-post-35.mail.demon.net (anchor-post-35.mail.demon.net [194.217.242.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id DAB9643D48 for ; Tue, 20 Jun 2006 15:42:34 +0000 (GMT) (envelope-from simon_freebsd_fs@bigblue.demon.co.uk) Received: from demon-gw.ledwoodtechnology.co.uk ([80.177.235.49] helo=[192.168.25.19]) by anchor-post-35.mail.demon.net with esmtp (Exim 4.42) id 1FsiMx-000Kxg-Hk for freebsd-fs@freebsd.org; Tue, 20 Jun 2006 15:42:29 +0000 Message-ID: <449817BE.50104@bigblue.demon.co.uk> Date: Tue, 20 Jun 2006 16:43:58 +0100 From: Simon Truss User-Agent: Thunderbird 1.5.0.4 (Windows/20060516) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <00b101c69476$9a6686a0$99dfdfdf@bakuhatsu.net> In-Reply-To: <00b101c69476$9a6686a0$99dfdfdf@bakuhatsu.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: On-disk format of UFS/UFS2 (for firmware implementation) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 15:42:35 -0000 Matt Sealey wrote: > Hello, > > I was wondering if there is any definitive, authoritative or even just comprehensive > documentation on the on-disk format (superblock, nodes, file linking, extents..) of > the UFS and UFS2 filesystems. You probably are looking for this paper: A Fast File System for UNIX (1984) http://citeseer.ist.psu.edu/12920.html There are several books on BSD of which Kirk McKusick has written some. I would recommend 'Design and Implementation of xxxBSD'. I have the 4.3BSD book however there are newer releases and a quick google turned up FreeBSD. Sleuthkit contains code to read many file systems and its code may assist in extracting the design of FFS. http://www.sleuthkit.org/ > We would like to implement it in firmware for booting FreeBSD (and NetBSD and eventually > Solaris which uses some variant). I have the source code from sys/fs/ufs already but it > is not the preferred method of implementing filesystem code. It only needs read support > for now so it shouldn't be that complicated, but... There has been work on flash file systems for BSD, a quick search may turn up something useful. Simon From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 15:51:06 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE68216A47B for ; Tue, 20 Jun 2006 15:51:06 +0000 (UTC) (envelope-from matt@genesi-usa.com) Received: from mail.genesippc.com (mithrandir.softwarenexus.net [66.98.186.96]) by mx1.FreeBSD.org (Postfix) with ESMTP id 326B843D45 for ; Tue, 20 Jun 2006 15:51:06 +0000 (GMT) (envelope-from matt@genesi-usa.com) Received: from p54b0ffec.dip.t-dialin.net ([84.176.255.236] helo=yukito) by mail.genesippc.com with esmtpa (Exim 4.62 (FreeBSD)) (envelope-from ) id 1FsiRl-00026Y-7Y; Tue, 20 Jun 2006 15:47:21 +0000 From: "Matt Sealey" To: "'Simon Truss'" , Date: Tue, 20 Jun 2006 10:51:00 -0500 Organization: Genesi Message-ID: <00bf01c69481$57a793d0$99dfdfdf@bakuhatsu.net> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2869 Thread-Index: AcaUf7hGXMcuzdaZSAa9DGmhTv2gtwAAMh4w In-Reply-To: <449817BE.50104@bigblue.demon.co.uk> Cc: Subject: RE: On-disk format of UFS/UFS2 (for firmware implementation) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: matt@genesi-usa.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 15:51:07 -0000 > You probably are looking for this paper: > > A Fast File System for UNIX (1984) > http://citeseer.ist.psu.edu/12920.html We have this implemented already in the firmware. However this does not help reading UFS2 (64-bit numbers, extents etc. and myriad other things to look out for). > Sleuthkit contains code to read many file systems and its > code may assist in extracting the design of FFS. > http://www.sleuthkit.org/ > > There has been work on flash file systems for BSD, a quick > search may turn up something useful. Why a flash filesystem? Our OpenFirmware has disk support, it pushes them into the device tree and you can load/boot any file on any supported filesystem (this includes IDE, SCSI, USB disks..) We just want to parse and use UFS filesystems, i.e. the modern ones that OpenBSD (UFS) and FreeBSD and NetBSD (UFS2) use. FFS isn't the same thing, as I am told by hundreds of articles... at least there are some differences which I have found hard to find the documentation on. In theory we should be able to implement the entire gamut in one filesystem support package, the same way the Linux ufs driver does, by adapting the "1984 FFS" driver. For that we need to know what we are doing.. :) The Sleuthkit guy seems to have written a book which may be very useful to us (if we order it..) http://www.digital-evidence.org/fsfa/ and this is more like what we are looking for. Source code as documentation is really the last resort when it comes to supporting things as it makes for very hard and expensive work. -- Matt Sealey Manager, Genesi, Developer Relations From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 16:07:17 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2417716A479 for ; Tue, 20 Jun 2006 16:07:17 +0000 (UTC) (envelope-from hardware@ee.ccny.cuny.edu) Received: from ees1s0.engr.ccny.cuny.edu (ees1s0.engr.ccny.cuny.edu [134.74.16.1]) by mx1.FreeBSD.org (Postfix) with SMTP id 5332943D45 for ; Tue, 20 Jun 2006 16:07:16 +0000 (GMT) (envelope-from hardware@ee.ccny.cuny.edu) Received: (qmail 14148 invoked from network); 20 Jun 2006 16:16:01 -0000 Received: from unknown (HELO eesfc20) (134.74.228.20) by ees1s0.engr.ccny.cuny.edu with SMTP; 20 Jun 2006 16:16:01 -0000 Date: Tue, 20 Jun 2006 12:06:45 -0400 (EDT) From: hardware@ee.ccny.cuny.edu X-X-Sender: hardware@eesfc20.engr.ccny.cuny.edu To: freebsd-fs@freebsd.org In-Reply-To: <20060620163718.26387be9.stas@310.ru> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: Re: ext2fs marked dirty when mounted read-only X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 16:07:17 -0000 I tried that modification. Still the same problem. -kr On Tue, 20 Jun 2006, Stanislav Sedov wrote: > There is a small bug in ext2fs code. I'd experienced the same problem > some time ago. > > Unfortunately, I have lost the patch. > > Try to change the line > "if (fs->s_rd_only && !vfs_flagopt(opts, "ro", NULL, 0)) {" > > in ext2_vfsops.c > > to > "if (fs->s_rd_only && !(mp->mnt_flag & MNT_RDONLY)) {" > > I suppose it will help. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 19:21:00 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1792B16A481 for ; Tue, 20 Jun 2006 19:21:00 +0000 (UTC) (envelope-from mikej@rogers.com) Received: from smtp101.rog.mail.re2.yahoo.com (smtp101.rog.mail.re2.yahoo.com [206.190.36.79]) by mx1.FreeBSD.org (Postfix) with SMTP id C1E4043D7C for ; Tue, 20 Jun 2006 19:20:39 +0000 (GMT) (envelope-from mikej@rogers.com) Received: (qmail 30511 invoked from network); 20 Jun 2006 19:20:38 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rogers.com; h=Received:Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=hj305CjEvHVsU5NZ25RJw755sYFbJTSuAgNveYBJs9ZAEIqUpdhNtIChn/GxZu6OepJraxKcBefQX3Hvbgm5vNNg7YZJOoUCT9GcdOQhkvQXha55TXbnleYcflpfXN1L6a45gG6KheCecXbkOx+wnZjwcOW7u7BsjeaLIsIdbZo= ; Received: from unknown (HELO ?70.31.50.218?) (mikej@rogers.com@70.31.50.218 with plain) by smtp101.rog.mail.re2.yahoo.com with SMTP; 20 Jun 2006 19:20:38 -0000 Message-ID: <44984A91.8040805@rogers.com> Date: Tue, 20 Jun 2006 15:20:49 -0400 From: Mike Jakubik User-Agent: Thunderbird 1.5.0.4 (Windows/20060516) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20060619131101.GD1130@garage.freebsd.pl> In-Reply-To: <20060619131101.GD1130@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 19:21:00 -0000 Pawel Jakub Dawidek wrote: > Copying one large file: > UFS: 8s > UFS+SU: 8s > gjournal(1): 16s > gjournal(2): 14s > > Copying eight large files in parallel: > UFS: 120s > UFS+SU: 120s > gjournal(1): 184s > gjournal(2): 165s > > Untaring eight src.tgz in parallel: > UFS: 791s > UFS+SU: 650s > gjournal(1): 333s > gjournal(2): 309s > > Reading. grep -r on two src/ directories in parallel: > UFS: 84s > UFS+SU: 138s > gjournal(1): 102s > gjournal(2): 89s > Not to sound ungrateful for the work, which i am, this is great! But the performance impact seems rather large to me. Does the presence of journaling mean that we could perhaps mount the filesystems async? Does it eliminate the need for softupdates? From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 19:39:14 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E9D9F16A474; Tue, 20 Jun 2006 19:39:13 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4D8F743D46; Tue, 20 Jun 2006 19:39:12 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 9351D51388; Tue, 20 Jun 2006 21:39:11 +0200 (CEST) Received: from localhost (dlc33.neoplus.adsl.tpnet.pl [83.24.32.33]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 24C8451307; Tue, 20 Jun 2006 21:39:06 +0200 (CEST) Date: Tue, 20 Jun 2006 21:36:30 +0200 From: Pawel Jakub Dawidek To: Mike Jakubik Message-ID: <20060620193630.GA8007@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <44984A91.8040805@rogers.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0F1p//8PRICkK4MW" Content-Disposition: inline In-Reply-To: <44984A91.8040805@rogers.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL,RCVD_IN_SORBS_WEB autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 19:39:14 -0000 --0F1p//8PRICkK4MW Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 20, 2006 at 03:20:49PM -0400, Mike Jakubik wrote: +> Pawel Jakub Dawidek wrote: +> >Copying one large file: +> >UFS: 8s +> >UFS+SU: 8s +> >gjournal(1): 16s +> >gjournal(2): 14s +> > +> >Copying eight large files in parallel: +> >UFS: 120s +> >UFS+SU: 120s +> >gjournal(1): 184s +> >gjournal(2): 165s +> > +> >Untaring eight src.tgz in parallel: +> >UFS: 791s +> >UFS+SU: 650s +> >gjournal(1): 333s +> >gjournal(2): 309s +> > +> >Reading. grep -r on two src/ directories in parallel: +> >UFS: 84s +> >UFS+SU: 138s +> >gjournal(1): 102s +> >gjournal(2): 89s +> > =20 +>=20 +> Not to sound ungrateful for the work, which i am, this is great! But the= performance impact seems rather large to me. Does the presence of journali= ng mean that we could=20 +> perhaps mount the filesystems async? Does it eliminate the need for soft= updates? The performance impact is big for large files, because in theory we have to write the data twice. Yes, it eliminates need for SU, but there are reasons, that you still want to use SU, eg. for snapshots. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --0F1p//8PRICkK4MW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEmE4+ForvXbEpPzQRAjSQAJ0e6afsrswjGwoJhPut8ECFSwWpwwCgp+gl dEU8PUrMPRznRZEOSYn1v5g= =NVNr -----END PGP SIGNATURE----- --0F1p//8PRICkK4MW-- From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 20:00:01 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 595D416A47B; Tue, 20 Jun 2006 20:00:01 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [210.51.165.229]) by mx1.FreeBSD.org (Postfix) with ESMTP id D42FA43D6E; Tue, 20 Jun 2006 19:59:54 +0000 (GMT) (envelope-from delphij@delphij.net) Received: from localhost (tarsier.geekcn.org [210.51.165.229]) by tarsier.geekcn.org (Postfix) with ESMTP id 83BD3EB2D6D; Wed, 21 Jun 2006 03:59:53 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([210.51.165.229]) by localhost (mail.geekcn.org [210.51.165.229]) (amavisd-new, port 10024) with ESMTP id TI1VS5fYHhkY; Wed, 21 Jun 2006 03:59:48 +0800 (CST) Received: from [192.168.1.9] (unknown [221.217.210.14]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTP id 8125EEB2D68; Wed, 21 Jun 2006 03:59:47 +0800 (CST) From: Xin LI To: Pawel Jakub Dawidek In-Reply-To: <20060620193630.GA8007@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <44984A91.8040805@rogers.com> <20060620193630.GA8007@garage.freebsd.pl> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-p+Mvcxbfi7F/bq0HN1Lm" Organization: The FreeBSD Project Date: Wed, 21 Jun 2006 03:59:46 +0800 Message-Id: <1150833586.24301.1.camel@spirit> Mime-Version: 1.0 X-Mailer: Evolution 2.6.2 FreeBSD GNOME Team Port Cc: freebsd-fs@FreeBSD.org, Mike Jakubik , freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 20:00:01 -0000 --=-p+Mvcxbfi7F/bq0HN1Lm Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable =E5=9C=A8 2006-06-20=E4=BA=8C=E7=9A=84 21:36 +0200=EF=BC=8CPawel Jakub Dawi= dek=E5=86=99=E9=81=93=EF=BC=9A > The performance impact is big for large files, because in theory we have > to write the data twice. > Yes, it eliminates need for SU, but there are reasons, that you still > want to use SU, eg. for snapshots. Em... IIRC SU and snapshots are independent, no? Cheers, --=20 Xin LI http://www.delphij.net/ --=-p+Mvcxbfi7F/bq0HN1Lm Content-Type: application/pgp-signature; name=signature.asc Content-Description: =?UTF-8?Q?=E8=BF=99=E6=98=AF=E4=BF=A1=E4=BB=B6=E7=9A=84=E6=95=B0?= =?UTF-8?Q?=E5=AD=97=E7=AD=BE=E5=90=8D=E9=83=A8=E5=88=86?= -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQBEmFOyhcUczkLqiksRAvvIAKDL6Of+43Ocr7FgJWkvEyxAfDLl+QCgtAkb wQ02zSKSY4O94UM7Uu6F15M= =4olk -----END PGP SIGNATURE----- --=-p+Mvcxbfi7F/bq0HN1Lm-- From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 20:07:25 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9A7C116A474 for ; Tue, 20 Jun 2006 20:07:25 +0000 (UTC) (envelope-from mikej@rogers.com) Received: from smtp104.rog.mail.re2.yahoo.com (smtp104.rog.mail.re2.yahoo.com [206.190.36.82]) by mx1.FreeBSD.org (Postfix) with SMTP id 4AA7F43D48 for ; Tue, 20 Jun 2006 20:07:24 +0000 (GMT) (envelope-from mikej@rogers.com) Received: (qmail 66799 invoked from network); 20 Jun 2006 20:07:23 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rogers.com; h=Received:Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject:References:In-Reply-To:Content-Type:Content-Transfer-Encoding; b=2GQun7jY0/SnK0/NVjCsnWRW1e7wa5mJhxbiTt6O12797Lr1G00QUpk88YcZFVrURHTxfNxjuSqnFPCk6dqZACM5ko3qMrO88CEGeoYEYS/xINcBON96ZMCS7ojhBF/uJ8gZhlEsxeO1NEurszUYn39qaXGIh9IBOPt44nZFmhE= ; Received: from unknown (HELO ?70.31.50.218?) (mikej@rogers.com@70.31.50.218 with plain) by smtp104.rog.mail.re2.yahoo.com with SMTP; 20 Jun 2006 20:07:23 -0000 Message-ID: <44985586.2090504@rogers.com> Date: Tue, 20 Jun 2006 16:07:34 -0400 From: Mike Jakubik User-Agent: Thunderbird 1.5.0.4 (Windows/20060516) MIME-Version: 1.0 To: Xin LI References: <20060619131101.GD1130@garage.freebsd.pl> <44984A91.8040805@rogers.com> <20060620193630.GA8007@garage.freebsd.pl> <1150833586.24301.1.camel@spirit> In-Reply-To: <1150833586.24301.1.camel@spirit> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, Pawel Jakub Dawidek , freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 20:07:25 -0000 Xin LI wrote: > 在 2006-06-20二的 21:36 +0200，Pawel Jakub Dawidek写道： > >> The performance impact is big for large files, because in theory we have >> to write the data twice. >> Yes, it eliminates need for SU, but there are reasons, that you still >> want to use SU, eg. for snapshots. >> > > Em... IIRC SU and snapshots are independent, no? > > Cheers, > What about mounting the filesystem async though? It was my understanding that the Linux filesystems were much faster in benchmarks because they were mounted async by default, however the presence of journaling allowed this safely. Is this the case here too? From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 20:07:55 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7FEE016A502 for ; Tue, 20 Jun 2006 20:07:55 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3977843D45 for ; Tue, 20 Jun 2006 20:07:54 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id 5BF863131D; Tue, 20 Jun 2006 16:07:44 -0400 (EDT) Date: Tue, 20 Jun 2006 16:07:44 -0400 (EDT) From: Ensel Sharon To: Scott Long In-Reply-To: <4496D0D8.8040705@samsco.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-fs@freebsd.org Subject: Adaptec 2820sa redux, and possible problems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 20:07:55 -0000 Scott, et. al, > As others suggested, you need to experiment with simplier > configurations. This will help us identify the cause and hopefully > implement a fix. No one is asking you to throw away money or resources. > Since you've already done the simple test with a single drive, could you > do the following two tests: > > 1. RAID-5, full size (whatever >2TB value you were talking about). > 2. RAID-6, <2TB. Unfortunately, I was at a remote site and needed to get back on a plane. Therefore, I was forced to take the 8 disks, create a mirror with the first two, and a raid-6 array with the remaining 6. It's not a great solution, but I only lost 3/8 for raid overhead instead of 4/8. As far as your tests, this shows that a <2TB raid 6 does indeed work, and that a non-raid-6 array also works. Here is the bad news: - the system survived raid creation (both arrays show optimal) (although the cards kernel _did_ crash out at 2% ... I just rebooted and it picked up where it left off until build/verify was complete) - the system survived freebsd installation and my own OS installs, port installs, etc. - the system survived some very large, very long rsyncs (200+ GB, with hundreds of thousands of inodes) HOWEVER: - if I do any kind of massive data move between the two arrays, the screen will fill up with aac0 command timeouts, and eventually will just crash and burn with: Warning! Controller is no longer running! code=0xbcef0100 I am running the latest stable firmware on this card, which I believe is 9117. Large array to array copies _are not_ something I need to do on this system ... and if it can survive a pretty brutal rsync ... I guess what I am asking is, if I am willing to accept possible system instability in rare occurances, am I in danger of data loss if I just keep running on it and wait for a better firmware (or whatever fix is developed) ? Comments ? From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 20:29:49 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 781AC16A474; Tue, 20 Jun 2006 20:29:49 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (bitblocks.com [209.204.185.216]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33FC543D46; Tue, 20 Jun 2006 20:29:49 +0000 (GMT) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 933F2294C1; Tue, 20 Jun 2006 13:29:48 -0700 (PDT) To: Pawel Jakub Dawidek In-reply-to: Your message of "Mon, 19 Jun 2006 15:11:01 +0200." <20060619131101.GD1130@garage.freebsd.pl> Date: Tue, 20 Jun 2006 13:29:48 -0700 From: Bakul Shah Message-Id: <20060620202948.933F2294C1@mail.bitblocks.com> Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 20:29:49 -0000 This is great! We have sorely needed this for quite a while what with terabyte size filesystems getting into common use. > How it works (in short). You may define one or two providers which > gjournal will use. If one provider is given, it will be used for both - > data and journal. If two providers are given, one will be used for data > and one for journal. > Every few seconds (you may define how many) journal is terminated and > marked as consistent and gjournal starts to copy data from it to the > data provider. In the same time new data are stored in new journal. Some random comments: Would it make sense to treat the journal as a circular buffer? Then commit to the underlying provider starts when the buffer has $hiwater blocks or the upper layer wants to sync. The commit stops when the buffer has $lowater blocks or in case of sync the buffer is empty. This will allow parallel writes to the provider and the journal, thereby reducing latency. I don't understand why you need FS synchronization. Once the journal is written, the data is safe. A "redo" may be needed after a crash to sync the filesystem but that is about it. Redo should be idempotent. Each journal write block may need some flags. For instance mark a block as a "sync point" -- when this block is on the disk, the FS will be in a consistent state. In case of redo after crash you have to throw away all the journal blocks after the last sync point. It seems to me if you write a serial number with each data block, in the worst case redo has to do a binary search to find the first block to write but normal writes to journal and reads from journal (for commiting to the provider) can be completely sequential. Since redo will be much much faster than fsck you can afford to slow it down a bit if the normal case can be speeded up. Presumably you disallow opening any file in /.deleted. Can you gjournal the journal disk? Recursion is good:-) -- bakul From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 20:33:11 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 35D1316A492; Tue, 20 Jun 2006 20:33:11 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6187C43D72; Tue, 20 Jun 2006 20:33:03 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [10.10.3.185] ([69.15.205.254]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k5KKWZMA004370; Tue, 20 Jun 2006 14:32:40 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <44985B5C.7090201@samsco.org> Date: Tue, 20 Jun 2006 14:32:28 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mike Jakubik References: <20060619131101.GD1130@garage.freebsd.pl> <44984A91.8040805@rogers.com> <20060620193630.GA8007@garage.freebsd.pl> <1150833586.24301.1.camel@spirit> <44985586.2090504@rogers.com> In-Reply-To: <44985586.2090504@rogers.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.0 required=3.8 tests=none autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: freebsd-fs@freebsd.org, Pawel Jakub Dawidek , freebsd-current@freebsd.org, Xin LI , freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 20:33:11 -0000 Mike Jakubik wrote: > Xin LI wrote: > >> 在 2006-06-20二的 21:36 +0200，Pawel Jakub Dawidek写道： >> >> >>> The performance impact is big for large files, because in theory we have >>> to write the data twice. >>> Yes, it eliminates need for SU, but there are reasons, that you still >>> want to use SU, eg. for snapshots. >>> >> >> >> Em... IIRC SU and snapshots are independent, no? >> >> Cheers, >> > > > What about mounting the filesystem async though? It was my understanding > that the Linux filesystems were much faster in benchmarks because they > were mounted async by default, however the presence of journaling > allowed this safely. Is this the case here too? > Yes, async mounting is much faster that sync mounting, and slightly faster than SU, except when SU is dealing with huge data sets. Then async is significantly faster. Scott From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 20:50:20 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 12F5B16A47F for ; Tue, 20 Jun 2006 20:50:20 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.184]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4A3BB43D55 for ; Tue, 20 Jun 2006 20:50:11 +0000 (GMT) (envelope-from uspoerlein@gmail.com) Received: by nf-out-0910.google.com with SMTP id h2so1248nfe for ; Tue, 20 Jun 2006 13:50:10 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:date:from:to:cc:subject:message-id:mail-followup-to:references:mime-version:content-type:content-disposition:in-reply-to; b=V+SfkU1yMDE8CPKJ5Y7Ojk8uWGueDBuZgYFi754VxAJ6rinWPIH98lNLhimlu0XAuaTVNgQ7VSJVlM86l0R84Wo1jsiALZlFW+QOn49VG5kQSOVxvVPTY6j0JkBGd6yu9HO5nu0zGLcqNx9/7+nDaLrJRbk+Rs/2Q5q/Wo/VEv0= Received: by 10.49.60.12 with SMTP id n12mr6009379nfk; Tue, 20 Jun 2006 13:43:54 -0700 (PDT) Received: from roadrunner.q.local ( [217.185.119.244]) by mx.gmail.com with ESMTP id a24sm7139686nfc.2006.06.20.13.43.52; Tue, 20 Jun 2006 13:43:53 -0700 (PDT) Received: from roadrunner.q.local (localhost [127.0.0.1]) by roadrunner.q.local (8.13.6/8.13.6) with ESMTP id k5KKi37R003488; Tue, 20 Jun 2006 22:44:04 +0200 (CEST) (envelope-from uspoerlein@gmail.com) Received: (from q@localhost) by roadrunner.q.local (8.13.6/8.13.6/Submit) id k5KHXdv3002364; Tue, 20 Jun 2006 19:33:39 +0200 (CEST) (envelope-from uspoerlein@gmail.com) Date: Tue, 20 Jun 2006 19:33:39 +0200 From: Ulrich Spoerlein To: Pawel Jakub Dawidek Message-ID: <20060620173339.GA1638@roadrunner.informatik.uni-wuerzburg.de> Mail-Followup-To: Pawel Jakub Dawidek , freebsd-current@FreeBSD.org, freebsd-fs@FreeBSD.org, freebsd-geom@FreeBSD.org References: <20060619131101.GD1130@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wRRV7LY7NUeQGEoC" Content-Disposition: inline In-Reply-To: <20060619131101.GD1130@garage.freebsd.pl> Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 20:50:20 -0000 --wRRV7LY7NUeQGEoC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Pawel Jakub Dawidek wrote: > Hello. >=20 > For the last few months I have been working on gjournal project. Cool Stuff! > Reading. grep -r on two src/ directories in parallel: > UFS: 84s > UFS+SU: 138s > gjournal(1): 102s > gjournal(2): 89s >=20 > As you can see, even on one disk, untaring eight src.tgz is two times > faster than UFS+SU. I've no idea why gjournal is faster in reading. The UFS+SU score doesn't seem right. Why do SU have a negative impact on read performance? Is it solely because of the atime updates? Ulrich Spoerlein --=20 PGP Key ID: 20FEE9DD Encrypted mail welcome! Fingerprint: AEC9 AF5E 01AC 4EE1 8F70 6CBD E76E 2227 20FE E9DD Which is worse: ignorance or apathy? Don't know. Don't care. --wRRV7LY7NUeQGEoC Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEmDFy524iJyD+6d0RAgcyAJoDlV8lNXEyU0AdGTc9XJtCSpYbLACfYEI3 yjo78oywHF0CfLTt9aq5IzI= =0lpD -----END PGP SIGNATURE----- --wRRV7LY7NUeQGEoC-- From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 20:53:12 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9DCDA16A47A; Tue, 20 Jun 2006 20:53:12 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id D05D243D53; Tue, 20 Jun 2006 20:53:11 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id D3D4B51814; Tue, 20 Jun 2006 22:53:08 +0200 (CEST) Received: from localhost (dkb36.neoplus.adsl.tpnet.pl [83.24.5.36]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 24F9F50EA7; Tue, 20 Jun 2006 22:53:00 +0200 (CEST) Date: Tue, 20 Jun 2006 22:50:22 +0200 From: Pawel Jakub Dawidek To: Phil Regnauld Message-ID: <20060620205022.GB8007@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <20060620102910.GG27055@tetard.starbsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qlTNgmc+xy1dBmNv" Content-Disposition: inline In-Reply-To: <20060620102910.GG27055@tetard.starbsd.org> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 20:53:12 -0000 --qlTNgmc+xy1dBmNv Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 20, 2006 at 12:29:10PM +0200, Phil Regnauld wrote: +> On Mon, Jun 19, 2006 at 03:11:01PM +0200, Pawel Jakub Dawidek wrote: +> >=20 +> > Copying one large file: +> > UFS: 8s +> > UFS+SU: 8s +> > gjournal(1): 16s +> > gjournal(2): 14s +>=20 +> This is very very interesting work! +>=20 +> I am definitely going to test this. +>=20 +> I know this is too early to ask considering the optimizations +> that can be done, but do you have any idea how this would perform +> compared to ReiserFS on similar operations as the ones you +> benchmarked ? No idea. I think ReiserFS is using only metadata journaling, but I don't know for sure. +> PS: is it me or is the patch missing a gjournal command, as invoked +> in your examples ? It is you:) gjournal(8) is implemented as shared library for geom(8) command, just like gconcat(8), gstripe(8), gmirror(8), graid3(8), geli(8), gnop(8), glabel(8) and gshsec(8). --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --qlTNgmc+xy1dBmNv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEmF+OForvXbEpPzQRAqUeAKCN+3d/HrYXz731kPvkf39DX9NX9ACeOwI2 Dc2MSZ4STzuYpZYc9lrLM50= =LvSB -----END PGP SIGNATURE----- --qlTNgmc+xy1dBmNv-- From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 21:00:12 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2C93716A505 for ; Tue, 20 Jun 2006 21:00:11 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 37CFF43D7D for ; Tue, 20 Jun 2006 21:00:04 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id A5C2531324; Tue, 20 Jun 2006 17:00:01 -0400 (EDT) Date: Tue, 20 Jun 2006 17:00:01 -0400 (EDT) From: Ensel Sharon To: freebsd-fs@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: RE: Adaptec 2820sa redux, and possible problems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 21:00:12 -0000 Additional tests reveal that an rsync from array0 -> array1 also causes the kernel controller to crash, etc. Both arrays can survive a brutal (big, long, lots of inodes) rsync, and they can even survive it when they are done simultaneously. They just can't survive it to each other (although I was surprised that an rsync over ssh behaved the same as a 'mv' or a 'cp'...) The good news is, it doesn't seem to be a bad card or corrupt memory or anything, since the crash is predictable and non-randomm. All disk caching is turned off in the controller, and the arrays were created with no read or write caching. There is no hardware (disk or controller) caching going on of any kind. Any other information I can provide ? Thanks. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 21:52:25 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C128B16A474 for ; Tue, 20 Jun 2006 21:52:25 +0000 (UTC) (envelope-from stas@310.ru) Received: from com1.ht-systems.ru (com1.ht-systems.ru [83.97.104.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2454243D79 for ; Tue, 20 Jun 2006 21:52:18 +0000 (GMT) (envelope-from stas@310.ru) Received: from [213.87.86.28] (helo=fonon.realnet) by com1.ht-systems.ru with esmtpa (Exim 4.62) (envelope-from ) id 1Fso7V-0008Q3-U7 for freebsd-fs@freebsd.org; Wed, 21 Jun 2006 01:50:52 +0400 Received: from fonon.realnet (localhost [127.0.0.1]) by fonon.realnet (8.13.6/8.13.6) with ESMTP id k5KLpfSo020404 for ; Wed, 21 Jun 2006 01:51:41 +0400 (MSD) (envelope-from stas@fonon.realnet) Received: (from stas@localhost) by fonon.realnet (8.13.6/8.13.6/Submit) id k5KLpdYp020403; Wed, 21 Jun 2006 01:51:39 +0400 (MSD) (envelope-from stas) Date: Wed, 21 Jun 2006 01:51:39 +0400 From: Stanislav Sedov To: freebsd-fs@freebsd.org Message-Id: <20060621015139.50da7a26.stas@310.ru> In-Reply-To: References: <20060620163718.26387be9.stas@310.ru> Organization: MBSD labs, Inc. X-Mailer: carrier-pigeon X-Operating-System: FreeBSD Mime-Version: 1.0 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable X-Spam-Flag: SKIP X-Spam-Yversion: Spamooborona 1.5.2 Subject: Re: ext2fs marked dirty when mounted read-only X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 21:52:25 -0000 On Tue, 20 Jun 2006 12:06:45 -0400 (EDT) hardware@ee.ccny.cuny.edu wrote: > I tried that modification. Still the same problem. -kr >=20 Ok, it was my fault. This should work (actually, have no time to test) --- /sys/gnu/fs/ext2fs/ext2_vfsops.c.orig Wed Jun 21 01:49:08 2006 +++ /sys/gnu/fs/ext2fs/ext2_vfsops.c Wed Jun 21 01:48:57 2006 @@ -599,7 +599,7 @@ int error; int ronly; =20 - ronly =3D vfs_flagopt(mp->mnt_optnew, "ro", NULL, 0); + ronly =3D (mp->mnt_flag & MNT_RDONLY) !=3D 0; /* XXX: use VOP_ACESS to check FS perms */ DROP_GIANT(); g_topology_lock(); --=20 Stanislav Sedov MBSD labs, Inc. =F2=CF=D3=D3=C9=D1, =ED=CF=D3=CB=D7=C1 http://mbsd.msk.ru -------------------------------------------------------------------- If the facts don't fit the theory, change the facts. -- A. Einstein -------------------------------------------------------------------- PGP fingerprint: F21E D6CC 5626 9609 6CE2 A385 2BF5 5993 EB26 9581 From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 23:18:40 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A2BB116A47A for ; Tue, 20 Jun 2006 23:18:40 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from multiplay.co.uk (core6.multiplay.co.uk [85.236.96.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id 05BB943D49 for ; Tue, 20 Jun 2006 23:18:39 +0000 (GMT) (envelope-from killing@multiplay.co.uk) Received: from vader ([212.135.219.179]) by multiplay.co.uk (multiplay.co.uk [85.236.96.23]) (MDaemon PRO v9.0.1) with ESMTP id md50002672049.msg for ; Wed, 21 Jun 2006 00:18:39 +0100 Message-ID: <009e01c694bf$d777b9d0$b3db87d4@multiplay.co.uk> From: "Steven Hartland" To: "Ensel Sharon" , References: Date: Wed, 21 Jun 2006 00:18:20 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2869 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2869 X-Spam-Processed: multiplay.co.uk, Wed, 21 Jun 2006 00:18:39 +0100 (not processed: message from valid local sender) X-MDRemoteIP: 212.135.219.179 X-Return-Path: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-fs@freebsd.org X-MDAV-Processed: multiplay.co.uk, Wed, 21 Jun 2006 00:18:40 +0100 Cc: Subject: Re: Adaptec 2820sa redux, and possible problems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 23:18:40 -0000 Ensel Sharon wrote: > Additional tests reveal that an rsync from array0 -> array1 also > causes the kernel controller to crash, etc. > > Both arrays can survive a brutal (big, long, lots of inodes) rsync, > and they can even survive it when they are done simultaneously. They > just can't survive it to each other (although I was surprised that an > rsync over ssh behaved the same as a 'mv' or a 'cp'...) > > The good news is, it doesn't seem to be a bad card or corrupt memory > or anything, since the crash is predictable and non-randomm. > > All disk caching is turned off in the controller, and the arrays were > created with no read or write caching. There is no hardware (disk or > controller) caching going on of any kind. Any other information I can > provide ? Out of interest what disks are u using. This sounds very much like a bug that I ran into with the hpt driver turned out to be that Seagate 400GB drives dont handle 28-bit command on sector 0xfffffff correctly. Driver must use a 48-bit command to access that sector. Drives from other vendors (Maxtor, WD) don't seem to have this issue. Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 20 23:53:49 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 65B6316A479 for ; Tue, 20 Jun 2006 23:53:49 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id E868B43D49 for ; Tue, 20 Jun 2006 23:53:48 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id D87543131C; Tue, 20 Jun 2006 19:53:47 -0400 (EDT) Date: Tue, 20 Jun 2006 19:53:47 -0400 (EDT) From: Ensel Sharon To: Steven Hartland In-Reply-To: <009e01c694bf$d777b9d0$b3db87d4@multiplay.co.uk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-fs@freebsd.org Subject: Re: Adaptec 2820sa redux, and possible problems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Jun 2006 23:53:49 -0000 On Wed, 21 Jun 2006, Steven Hartland wrote: > Out of interest what disks are u using. This sounds very much like > a bug that I ran into with the hpt driver turned out to be that Seagate > 400GB drives dont handle 28-bit command on sector 0xfffffff correctly. > Driver must use a 48-bit command to access that sector. > > Drives from other vendors (Maxtor, WD) don't seem to have this issue. The drives are Hitachi 500 GB 7200 RPM 16meg: HDS725050KLA360 Adaptec told me today that these drives are verified or approved or something with that card... So in your case, you made multiple arrays with those drives, and then large data moves between arrays caused crashes, and it turned out to be the drives fault ? I would think the problem you describe would manifest itself locally to a single array ... From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 00:09:41 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9941916A47E for ; Wed, 21 Jun 2006 00:09:41 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from multiplay.co.uk (core6.multiplay.co.uk [85.236.96.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id BCDDA43D7C for ; Wed, 21 Jun 2006 00:09:34 +0000 (GMT) (envelope-from killing@multiplay.co.uk) Received: from vader ([212.135.219.179]) by multiplay.co.uk (multiplay.co.uk [85.236.96.23]) (MDaemon PRO v9.0.1) with ESMTP id md50002672146.msg for ; Wed, 21 Jun 2006 01:09:20 +0100 Message-ID: <00cf01c694c6$ec19b120$b3db87d4@multiplay.co.uk> From: "Steven Hartland" To: "Ensel Sharon" References: Date: Wed, 21 Jun 2006 01:09:03 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2869 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2869 X-Spam-Processed: multiplay.co.uk, Wed, 21 Jun 2006 01:09:20 +0100 (not processed: message from valid local sender) X-MDRemoteIP: 212.135.219.179 X-Return-Path: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-fs@freebsd.org X-MDAV-Processed: multiplay.co.uk, Wed, 21 Jun 2006 01:09:21 +0100 Cc: freebsd-fs@freebsd.org Subject: Re: Adaptec 2820sa redux, and possible problems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 00:09:41 -0000 Ensel Sharon wrote: > On Wed, 21 Jun 2006, Steven Hartland wrote: > > The drives are Hitachi 500 GB 7200 RPM 16meg: > > HDS725050KLA360 > > Adaptec told me today that these drives are verified or approved or > something with that card... > > So in your case, you made multiple arrays with those drives, and then > large data moves between arrays caused crashes, and it turned out to > be the drives fault ? I would think the problem you describe would > manifest itself locally to a single array ... Thats correct. Changing the driver to always use high bit addressing if supported fixed the issue. ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 00:49:04 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3CE5916A482 for ; Wed, 21 Jun 2006 00:49:04 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 19CDC43D5C for ; Wed, 21 Jun 2006 00:49:01 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id 63FC131305; Tue, 20 Jun 2006 20:49:00 -0400 (EDT) Date: Tue, 20 Jun 2006 20:49:00 -0400 (EDT) From: Ensel Sharon To: Steven Hartland In-Reply-To: <00cf01c694c6$ec19b120$b3db87d4@multiplay.co.uk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-fs@freebsd.org Subject: Re: Adaptec 2820sa redux, and possible problems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 00:49:04 -0000 On Wed, 21 Jun 2006, Steven Hartland wrote: > > The drives are Hitachi 500 GB 7200 RPM 16meg: > > > > HDS725050KLA360 > > > > Adaptec told me today that these drives are verified or approved or > > something with that card... > > > > So in your case, you made multiple arrays with those drives, and then > > large data moves between arrays caused crashes, and it turned out to > > be the drives fault ? I would think the problem you describe would > > manifest itself locally to a single array ... > > Thats correct. Changing the driver to always use high bit addressing > if supported fixed the issue. Cool. And there was no data loss involved in the bad behavior ? That is, if you hadn't fixed it, it would have been merely annoying and unreliable, but always recoverable ? To be honest, it looks and smells just like the snapshot problems. Obviously it isn't, since I am not running any snapshots, but the point is, all of the snapshot bugs could be manifested simply by copying all of /usr/ports from one partition to another, and this looks and smells just like that. But what do I know. Can anyone help me with aac specifically, and this 2820sa ? I can provide many more details if necessary ... I'd _love it_ if I could just patch my driver and go off into the sunset... From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 03:08:21 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 72F7616A47C for ; Wed, 21 Jun 2006 03:08:21 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 88F2D43D48 for ; Wed, 21 Jun 2006 03:08:20 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.22] (andersonbox2.centtech.com [192.168.42.22]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k5L38HsI093164; Tue, 20 Jun 2006 22:08:18 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4498B82C.2010600@centtech.com> Date: Tue, 20 Jun 2006 22:08:28 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: matt@genesi-usa.com References: <00bf01c69481$57a793d0$99dfdfdf@bakuhatsu.net> In-Reply-To: <00bf01c69481$57a793d0$99dfdfdf@bakuhatsu.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1555/Tue Jun 20 17:21:12 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: On-disk format of UFS/UFS2 (for firmware implementation) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 03:08:21 -0000 Matt Sealey wrote: >> You probably are looking for this paper: >> >> A Fast File System for UNIX (1984) >> http://citeseer.ist.psu.edu/12920.html > > We have this implemented already in the firmware. > > However this does not help reading UFS2 (64-bit numbers, extents etc. > and myriad other things to look out for). I like the "Design and Implementation of the FreeBSD 5.2..." book. It covers what you need, plus more that you won't need. The best documentation is the .h files I think, but you aren't interested in those. >> Sleuthkit contains code to read many file systems and its >> code may assist in extracting the design of FFS. >> http://www.sleuthkit.org/ >> >> There has been work on flash file systems for BSD, a quick >> search may turn up something useful. > > Why a flash filesystem? For different reasons - mostly for embedded systems builders who use a real OS (like FreeBSD) but want to make their flash last a while. > Our OpenFirmware has disk support, it pushes them into the device > tree and you can load/boot any file on any supported filesystem (this > includes IDE, SCSI, USB disks..) > > We just want to parse and use UFS filesystems, i.e. the modern ones > that OpenBSD (UFS) and FreeBSD and NetBSD (UFS2) use. FFS isn't the > same thing, as I am told by hundreds of articles... at least there are > some differences which I have found hard to find the documentation on. > > In theory we should be able to implement the entire gamut in one > filesystem support package, the same way the Linux ufs driver does, > by adapting the "1984 FFS" driver. > > For that we need to know what we are doing.. :) > > The Sleuthkit guy seems to have written a book which may be very > useful to us (if we order it..) http://www.digital-evidence.org/fsfa/ > and this is more like what we are looking for. Source code as > documentation is really the last resort when it comes to supporting > things as it makes for very hard and expensive work. > Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 03:20:32 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 59E9C16A474; Wed, 21 Jun 2006 03:20:32 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id A072943D46; Wed, 21 Jun 2006 03:20:31 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.22] (andersonbox2.centtech.com [192.168.42.22]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k5L3KRp3013262; Tue, 20 Jun 2006 22:20:29 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4498BB05.9020006@centtech.com> Date: Tue, 20 Jun 2006 22:20:37 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: Scott Long References: <449606B8.5080106@centtech.com> <4496CEBB.4020200@samsco.org> In-Reply-To: <4496CEBB.4020200@samsco.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1555/Tue Jun 20 17:21:12 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org, Ensel Sharon , freebsd-questions@freebsd.org Subject: Re: Please help - adaptec 2820sa not ... RESOLVED X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 03:20:32 -0000 Scott Long wrote: > Eric Anderson wrote: >> Ensel Sharon wrote: >> >>> >>> On Mon, 19 Jun 2006, Jahilliya wrote: >>> >>>>> Ok, aac is in the dmesg. >>>>> >>>>> I can see both 2820sa cards in the dmesg, and I see no errors, etc. - >>>>> there are just no drives listed in dmesg. >>>>> >>>>> My setup is 8 500 Gb drives in a single raid-6 array, size ~2.8TB. >>>>> Any >>>>> problems with that ? Perhaps sysinstall cannot deal with a >2TB drive >>>>> ? There are no other drives in the system besides the single 2.8TB >>>>> raid 6 >>>>> array ... >>>>> >>>> Have you got any other drives you can attach to the raid? >>>> >>>> If so, disconnect the 8 drives connected, connect up a couple that are >>>> not part of the raid and configure them as a simple raid 1 and see if >>>> the installers sees that raid. Or try any combination in drives to >>>> bring the raid size down below 2TB (I'm sure this limitation has been >>>> fixed.) >>> >>> >>> >>> Ok, the answer is that it has not been fixed. >>> >>> 6.1 sysinstall does in fact see both 2820sa controllers, and when I >>> put in >>> a single 160GB sata drive, it does see that single drive and I can >>> install >>> onto it, etc. >>> >>> Sysinstall does _not_ see my 2.7TB raid6 array. I suspect that if it >>> were >>> smaller than 2TB, it would see it correctly. >>> >>> I have a number of options with which to deal with this, all of which >>> involve either wasting money or wasting disk space. Fantastic. >> >> >> Right - FreeBSD doesn't recognize >2TB LUNs. > > Wrong on several counts. First, the AAC driver does not present arrays > to the system as SCSI LUNs. The traditional 2TB limit with 12 byte CDB > issue simply doesn't exist with this driver. Second, the FreeBSD SCSI > layer knows how to issue 16 byte CDBs to access >2TB, assuming that the > target understands the 16-byte protocol. So no, there is no 2TB limit > inherent to FreeBSD. The only limit is with individual drivers and with > hardware. > > Scott Scott, thanks for correcting me on this. I was under the (false) impression that this was a FreeBSD SCSI issue, not a driver issue. I had tried this with QLogic fiber (fabric) storage in the past, and had issues, so I incorrectly thought that the information I had was true. Is it difficult to make drivers (such as the qlogic isp driver) handle >2TB devices? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 15:30:25 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1FA0216A47C; Wed, 21 Jun 2006 15:30:25 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2.pacific.net.au [61.8.0.115]) by mx1.FreeBSD.org (Postfix) with ESMTP id 32E8E43D5A; Wed, 21 Jun 2006 15:30:23 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.2.162]) by mailout2.pacific.net.au (Postfix) with ESMTP id 5B36C10D098; Thu, 22 Jun 2006 01:30:21 +1000 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy1.pacific.net.au (8.13.4/8.13.4/Debian-3sarge1) with ESMTP id k5LFUIqT018161; Thu, 22 Jun 2006 01:30:19 +1000 Date: Thu, 22 Jun 2006 01:30:18 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Ulrich Spoerlein In-Reply-To: <20060620173339.GA1638@roadrunner.informatik.uni-wuerzburg.de> Message-ID: <20060622003036.M52310@delplex.bde.org> References: <20060619131101.GD1130@garage.freebsd.pl> <20060620173339.GA1638@roadrunner.informatik.uni-wuerzburg.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, Pawel Jakub Dawidek , freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 15:30:25 -0000 On Tue, 20 Jun 2006, Ulrich Spoerlein wrote: > Pawel Jakub Dawidek wrote: >> Hello. >> >> For the last few months I have been working on gjournal project. > > Cool Stuff! > >> Reading. grep -r on two src/ directories in parallel: >> UFS: 84s >> UFS+SU: 138s >> gjournal(1): 102s >> gjournal(2): 89s >> >> As you can see, even on one disk, untaring eight src.tgz is two times >> faster than UFS+SU. I've no idea why gjournal is faster in reading. > > The UFS+SU score doesn't seem right. Why do SU have a negative impact on > read performance? Is it solely because of the atime updates? ffs+SU is only 1-10% slower than ffs in my benchmarks of reading back a copy of most of src/ written to a new file system by the same filesystem (code) that does the readback. The speed only depends on which file system wrote the data. I use tar for reading. Maybe concurrent greps on separate directories amplify the problem. A tiny subset of saved benchmarked output: %%% Jan 29 2004 real-current writing to WD 1200JB h: 26683965 73593765 --- srcs = "contrib crypto lib sys" in /usr/src ffs-16384-02048-1: tarcp /f srcs: 43.23 real 0.65 user 6.85 sys tar cf /dev/zero srcs: 15.58 real 0.19 user 2.13 sys ffs-16384-02048-2: tarcp /f srcs: 41.26 real 0.50 user 7.06 sys tar cf /dev/zero srcs: 15.80 real 0.25 user 2.10 sys ffs-16384-02048-as-1: tarcp /f srcs: 22.17 real 0.49 user 6.47 sys tar cf /dev/zero srcs: 15.52 real 0.22 user 2.13 sys ffs-16384-02048-as-2: tarcp /f srcs: 21.67 real 0.45 user 6.61 sys tar cf /dev/zero srcs: 15.65 real 0.19 user 2.16 sys ffs-16384-02048-su-1: tarcp /f srcs: 60.35 real 0.49 user 7.02 sys tar cf /dev/zero srcs: 17.32 real 0.20 user 2.15 sys ffs-16384-02048-su-2: tarcp /f srcs: 61.82 real 0.50 user 7.14 sys tar cf /dev/zero srcs: 17.56 real 0.21 user 2.17 sys %%% Notation: 16384-02048 is the block-frag size; /""/as/su/ are /default/async mounts/soft updates/; -[12] is ffs[12]. The source tree is prefetched into VMIO so that the copy part of the benchmark is mostly a write benchmark and is not affected by any slowness in the physical source file system. The above shows soft updates being about 2 seconds or 10% slower for read-back. It also shows that soft updates is about 3 times as slow as async mounts and about 1.5 times as slow as the default (sync metadata and async data). Soft updates was faster than the default when it was first implemented, but became slower at least for writing a copy of src/. This seems to be due to soft updates interacting badly with bufdaemon. This may be fixed now (I have later runs of the benchmark showing soft updates having about the same speed as the default, but none for -realcurrent). I never found the exact cause of the slower readback. My theory is that block allocation is more delayed in the soft updates case, and soft updates uses this to perfectly pessimize some aspects of the allocation. My version of ffs allocates the first indirect block between the NDADDR-1'th and NDADDR'th data blocks. This seems to help generally, and reduces the disadvantage of soft updates. IIRC, the default normally puts this block not very far away but not necessarily between the data blocks, but soft updates pessimizes it by moving it a long way away. It's still surprising that this makes nearly a 10% difference for src/, since most files in src/ are too small to have even 1 indirect block. I always disable atime updates on ffs file systems and don't have comparitive benchmarks for the difference from this. Bruce From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 19:21:40 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3347516A474 for ; Wed, 21 Jun 2006 19:21:40 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8D4E343D58 for ; Wed, 21 Jun 2006 19:21:38 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id 115253132A; Wed, 21 Jun 2006 15:21:37 -0400 (EDT) Date: Wed, 21 Jun 2006 15:21:37 -0400 (EDT) From: Ensel Sharon To: freebsd-fs@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: 2820sa crashes were quotas the whole time (!) Some questions... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 19:21:40 -0000 I was having trouble this weekend with a 2820sa on 6.1-RELEASE. Although those problems went away when I downsized my largest partitions below 2TB, the system was still crashing _a lot_, always with aac0 timeouts, and eventually a message: Warning! Controller is no longer running! code=0xbcef0100 I turned off quotas and all is well. ----- So ... is it possible that the quota problems in 6.1 would cause a raid controller to crash ? (it seems to be the case) What are the details of the 6.1 quota problems (I am running them fine in 6.0, with a 1610sa adaptec sata raid card) and is there some kind of tuning or workaround that would allow them to be run ? Is there a forthcoming patch that I can apply to 6.1-RELEASE without reloading the whole OS ? Thanks. From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 20:06:43 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0A15A16A481 for ; Wed, 21 Jun 2006 20:06:43 +0000 (UTC) (envelope-from sven.hazejager@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.181]) by mx1.FreeBSD.org (Postfix) with ESMTP id E5DC743D45 for ; Wed, 21 Jun 2006 20:06:41 +0000 (GMT) (envelope-from sven.hazejager@gmail.com) Received: by py-out-1112.google.com with SMTP id d80so63908pyd for ; Wed, 21 Jun 2006 13:06:41 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type; b=i8BjVs2x37DEz4cE7BRkCcoCtIbLgF2WUt6lOGrvfGSdnWjifYub3Hu8jYKQfmmUgZ970xZc54g7ebenXDSzJYl6QJIKyU0A1SZ2yIw1u8ZUD4rS0on387/Nl7QFeIBmMr7IIq6PTrNVtTyEfG7TORxLvNqJrytVAIqZ4LGfGV4= Received: by 10.35.84.16 with SMTP id m16mr232625pyl; Wed, 21 Jun 2006 13:06:39 -0700 (PDT) Received: by 10.35.123.3 with HTTP; Wed, 21 Jun 2006 13:06:38 -0700 (PDT) Message-ID: Date: Wed, 21 Jun 2006 22:06:38 +0200 From: "Sven Hazejager" To: freebsd-questions@freebsd.org, freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: HELP! Filesystem EMPTY after upgrade from 4.11 to 6.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 20:06:43 -0000 Hi all! I have quite a big problem here.... I've upgraded my FreeBSD 4.11 to 6.1. Basically, I did a newfs of / and /usr and reinstalled from scratch. That worked. Rebooted in 6.1 single user mode, mounted /, /usr and /usr/home. The latter I did not touch and I need to save that partition. /usr/home had all the files it needed to have. PROBLEM: mount said soft-updates were not set on /usr/home. So I did umount /usr/home, tunefs -n enable /usr/home, mount /usr/home again... EMPTY!!!! What has gone wrong? I did fsck (in read-only), no problems, rebooted the system, did tunefs -n disable, still no luck. I really need those files guys... Please help urgently. Many thanks, Sven Hazejager From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 20:24:44 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1686916A479 for ; Wed, 21 Jun 2006 20:24:44 +0000 (UTC) (envelope-from sven.hazejager@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.182]) by mx1.FreeBSD.org (Postfix) with ESMTP id 97A3843D48 for ; Wed, 21 Jun 2006 20:24:43 +0000 (GMT) (envelope-from sven.hazejager@gmail.com) Received: by py-out-1112.google.com with SMTP id e30so67366pya for ; Wed, 21 Jun 2006 13:24:43 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type; b=uLXL+LnDmnHVbcIreU5xoiqgomLkvCrJG1nNSXZocSz0/tGxTJ8BDMrK0AOQcP/PxlBrZIv3xKLe/C2L4p4AoSBg+jDScopvQZazM50zXcUi7z2Xbc7BAJ7KKOLIdWwJc2f4aG/4AaIT4XHO3fbiIZiBWgUPGGanq6N2tkwUMyI= Received: by 10.35.20.14 with SMTP id x14mr251250pyi; Wed, 21 Jun 2006 13:24:42 -0700 (PDT) Received: by 10.35.123.3 with HTTP; Wed, 21 Jun 2006 13:24:42 -0700 (PDT) Message-ID: Date: Wed, 21 Jun 2006 22:24:42 +0200 From: "Sven Hazejager" To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Help! EMPTY filesystem after 4.11 to 6.1 upgrade X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 20:24:44 -0000 Hi all! [please ignore double-post; haven't received a confirmation after 30 mins] I have quite a big problem here.... I've upgraded my FreeBSD 4.11 to 6.1. Basically, I did a newfs of / and /usr and reinstalled from scratch. That worked. Rebooted in 6.1 single user mode, mounted /, /usr and /usr/home. The latter I did not touch and I need to save that partition. /usr/home had all the files it needed to have. PROBLEM: mount said soft-updates were not set on /usr/home. So I did umount /usr/home, tunefs -n enable /usr/home, mount /usr/home again... EMPTY!!!! What has gone wrong? I did fsck (in read-only), no problems, rebooted the system, did tunefs -n disable, still no luck. I really need those files guys... Please help urgently. Many thanks, Sven Hazejager From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 20:46:02 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 61E3516A47A for ; Wed, 21 Jun 2006 20:46:02 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id F3BB843D46 for ; Wed, 21 Jun 2006 20:45:58 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k5LKjvQA078630; Wed, 21 Jun 2006 15:45:57 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4499B011.4010408@centtech.com> Date: Wed, 21 Jun 2006 15:46:09 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: Sven Hazejager References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1559/Wed Jun 21 09:23:13 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: Help! EMPTY filesystem after 4.11 to 6.1 upgrade X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 20:46:02 -0000 Sven Hazejager wrote: > Hi all! [please ignore double-post; haven't received a confirmation > after 30 > mins] > > I have quite a big problem here.... > > I've upgraded my FreeBSD 4.11 to 6.1. Basically, I did a newfs of / and > /usr > and reinstalled from scratch. That worked. > > Rebooted in 6.1 single user mode, mounted /, /usr and /usr/home. The latter > I did not touch and I need to save that partition. /usr/home had all the > files it needed to have. PROBLEM: mount said soft-updates were not set on > /usr/home. So I did umount /usr/home, tunefs -n enable /usr/home, mount > /usr/home again... EMPTY!!!! > > What has gone wrong? I did fsck (in read-only), no problems, rebooted the > system, did tunefs -n disable, still no luck. I really need those files > guys... So the fsck passes? Can you send some information like how you upgraded (binary, cvs, etc), and possibly some df's from before and after you mount the partition? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 20:50:05 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DDFB516A47C for ; Wed, 21 Jun 2006 20:50:05 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30312.mail.mud.yahoo.com (web30312.mail.mud.yahoo.com [68.142.201.230]) by mx1.FreeBSD.org (Postfix) with SMTP id 3A92143D48 for ; Wed, 21 Jun 2006 20:50:05 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 69119 invoked by uid 60001); 21 Jun 2006 20:50:03 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=dHaODgtmFmQP6JRg5R8VAqaJeDBzYkDFQz8Wm72eB6CEED2R3g6ikNIeSdJZ6verTHdaSp613tkncyY+52hK5ILS4wUfK9woAetHsnGv4DEgH92u86L986Bcz5ppzr9glY5u5J6wQ9Wf97XT43Rv7EqhB15CQqGB+S6/GOdllxE= ; Message-ID: <20060621205003.69117.qmail@web30312.mail.mud.yahoo.com> Received: from [213.54.89.167] by web30312.mail.mud.yahoo.com via HTTP; Wed, 21 Jun 2006 13:50:03 PDT Date: Wed, 21 Jun 2006 13:50:03 -0700 (PDT) From: "R. B. Riddick" To: Sven Hazejager , freebsd-fs@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Help! EMPTY filesystem after 4.11 to 6.1 upgrade X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 20:50:06 -0000 --- Sven Hazejager wrote: > What has gone wrong? I did fsck (in read-only), no problems, rebooted the > system, did tunefs -n disable, still no luck. I really need those files > guys... > > Please help urgently. > The technical part: Hmm... I would try to do something palliative: E. g.: 1. Make a sector-by-sector copy of the device that contains the /usr/home file system. Then u can experiment without damaging something. 2. Or installing an old backup. Then you could try fsdb on that copy... Maybe your data still hides somewhere... ;-)) The psychological part: Oh my! :-) I know that feeling... And that is why, I do many backups (every 10 minutes an incremental backup from my RAID1 home file system to a simple file system. And from there to DVD-RW... A full backup to DVD-RW every 50 days... My history teacher complained for years about his missing Hitler-LPs and somehow it always felt like he thinks somebody will just pull them out of his brief case... I am sorry, that I do not know something more curative... Maybe somebody else here knows something against that syndrome... But (experienced) history (teachers) teach(es) us, that backups r a good idea... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Wed Jun 21 20:57:16 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8303016A47A for ; Wed, 21 Jun 2006 20:57:16 +0000 (UTC) (envelope-from sven.hazejager@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.177]) by mx1.FreeBSD.org (Postfix) with ESMTP id D11B543D45 for ; Wed, 21 Jun 2006 20:57:10 +0000 (GMT) (envelope-from sven.hazejager@gmail.com) Received: by py-out-1112.google.com with SMTP id x31so72274pye for ; Wed, 21 Jun 2006 13:57:10 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=J+1avrSuzsxHKd3hz1k8pw8Ujr8qOFtyPQ/eOcbYKA48H5uOQ3osToMBRnTichWcXhkTYDtboVxLh4kxWCJnCE+kQT4+O5m7JSTZInQLGDw7J/zDTScXbbf5nb5IHzTmxkS6AV4bJu/vlkkp//abUAbcdeg5GkTHkOXbfO/v/DQ= Received: by 10.35.106.15 with SMTP id i15mr288327pym; Wed, 21 Jun 2006 13:57:10 -0700 (PDT) Received: by 10.35.123.3 with HTTP; Wed, 21 Jun 2006 13:57:09 -0700 (PDT) Message-ID: Date: Wed, 21 Jun 2006 22:57:09 +0200 From: "Sven Hazejager" To: "Eric Anderson" , freebsd-fs@freebsd.org In-Reply-To: <4499B011.4010408@centtech.com> MIME-Version: 1.0 References: <4499B011.4010408@centtech.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: Help! EMPTY filesystem after 4.11 to 6.1 upgrade X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jun 2006 20:57:16 -0000 On 6/21/06, Eric Anderson wrote: > > Sven Hazejager wrote: > > I've upgraded my FreeBSD 4.11 to 6.1. Basically, I did a newfs of / and > > /usr > > and reinstalled from scratch. That worked. > > > > Rebooted in 6.1 single user mode, mounted /, /usr and /usr/home. The > latter > > I did not touch and I need to save that partition. /usr/home had all the > > files it needed to have. PROBLEM: mount said soft-updates were not set > on > > /usr/home. So I did umount /usr/home, tunefs -n enable /usr/home, mount > > /usr/home again... EMPTY!!!! > > > > What has gone wrong? I did fsck (in read-only), no problems, rebooted > the > > system, did tunefs -n disable, still no luck. I really need those files > > guys... > > > So the fsck passes? Can you send some information like how you upgraded > (binary, cvs, etc), and possibly some df's from before and after you > mount the partition? Found the problem! User error! I edited /etc/fstab on 6.1 manually and made a typo... instead of /usr/home I was mounting a spare, empty partition that I did not remember was on the disk... ;-) Thank you all for reading! Sven From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 09:09:03 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A6E6516A479 for ; Thu, 22 Jun 2006 09:09:03 +0000 (UTC) (envelope-from carpetsmoker@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.181]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2B57F43D53 for ; Thu, 22 Jun 2006 09:09:03 +0000 (GMT) (envelope-from carpetsmoker@gmail.com) Received: by py-out-1112.google.com with SMTP id c59so260521pyc for ; Thu, 22 Jun 2006 02:09:02 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:date:to:subject:content-type:mime-version:references:content-transfer-encoding:message-id:in-reply-to:user-agent:from; b=r253TJCz1gW8RRJq7fsepc0Du+1OBe58pPpfvfjhgvSXaq0i1f6atgt0u2AD0B4gOQwqlZkZ7NWFM3RgpHeX4ahsMVe84pxS9P2GrzJQa9EN5DEQurBrnB5AKRNtdfg3Z7VLhRRaNZDfqaI5O2eNR4j0UCBRK3sNSOUJF+YdCfQ= Received: by 10.35.111.14 with SMTP id o14mr855997pym; Thu, 22 Jun 2006 02:09:02 -0700 (PDT) Received: from carpetsmoker.ictwerkplaats.org ( [80.126.94.163]) by mx.gmail.com with ESMTP id y78sm716333pyg.2006.06.22.02.09.00; Thu, 22 Jun 2006 02:09:01 -0700 (PDT) Date: Thu, 22 Jun 2006 11:06:48 -0000 To: "Sven Hazejager" , freebsd-questions@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; format=flowed; delsp=yes; charset=us-ascii MIME-Version: 1.0 References: Content-Transfer-Encoding: Quoted-Printable Message-ID: In-Reply-To: User-Agent: Opera Mail/9.00 (FreeBSD) From: Martin Tournoy Cc: Subject: Re: HELP! Filesystem EMPTY after upgrade from 4.11 to 6.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 09:09:03 -0000 On Wed, 21 Jun 2006 20:06:38 -0000, Sven Hazejager = wrote: > Hi all! > > I have quite a big problem here.... > > I've upgraded my FreeBSD 4.11 to 6.1. Basically, I did a newfs of / an= d = > /usr > and reinstalled from scratch. That worked. > > Rebooted in 6.1 single user mode, mounted /, /usr and /usr/home. The = > latter > I did not touch and I need to save that partition. /usr/home had all t= he > files it needed to have. PROBLEM: mount said soft-updates were not set= on > /usr/home. So I did umount /usr/home, tunefs -n enable /usr/home, moun= t > /usr/home again... EMPTY!!!! > > What has gone wrong? I did fsck (in read-only), no problems, rebooted = the > system, did tunefs -n disable, still no luck. I really need those file= s > guys... > > Please help urgently. > > Many thanks, > > Sven Hazejager 4.11 uses UFS1, 6.1 UFS2, there might be some conflict there... Try using stellar phoenix BSD, it can recover data, before you start = messing around and destroy stuff... It's not free, but there's a free trial... http://www.stellarinfo.com/download/download_form.php?sid=3D35 From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 09:49:06 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0DAD416A67D; Thu, 22 Jun 2006 09:49:06 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 082CA441AF; Thu, 22 Jun 2006 09:48:35 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 7583551853; Thu, 22 Jun 2006 11:48:33 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id A7BEA51849; Thu, 22 Jun 2006 11:48:29 +0200 (CEST) Date: Thu, 22 Jun 2006 11:45:52 +0200 From: Pawel Jakub Dawidek To: freebsd-current@FreeBSD.org, freebsd-fs@FreeBSD.org, freebsd-geom@FreeBSD.org Message-ID: <20060622094552.GC30568@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <20060620173339.GA1638@roadrunner.informatik.uni-wuerzburg.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DIOMP1UsTsWJauNi" Content-Disposition: inline In-Reply-To: <20060620173339.GA1638@roadrunner.informatik.uni-wuerzburg.de> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 09:49:06 -0000 --DIOMP1UsTsWJauNi Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 20, 2006 at 07:33:39PM +0200, Ulrich Spoerlein wrote: +> Pawel Jakub Dawidek wrote: +> > Reading. grep -r on two src/ directories in parallel: +> > UFS: 84s +> > UFS+SU: 138s +> > gjournal(1): 102s +> > gjournal(2): 89s +> >=20 +> > As you can see, even on one disk, untaring eight src.tgz is two times +> > faster than UFS+SU. I've no idea why gjournal is faster in reading. +>=20 +> The UFS+SU score doesn't seem right. Why do SU have a negative impact on +> read performance? Is it solely because of the atime updates? As I said, I've not idea. You may simply ignore my benchmarks and try them on your own:) --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --DIOMP1UsTsWJauNi Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEmmbQForvXbEpPzQRAjOhAJ9HYmJq9Mnk9C7okc6eR5sxED6o5wCfW+MI /SRcuCebXaP1hRN/30tbW+Q= =ktIy -----END PGP SIGNATURE----- --DIOMP1UsTsWJauNi-- From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 09:55:34 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1DF8216A47A; Thu, 22 Jun 2006 09:55:34 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D8F144035; Thu, 22 Jun 2006 09:55:33 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 0DB7D51884; Thu, 22 Jun 2006 11:55:32 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id C7ACB51853; Thu, 22 Jun 2006 11:55:27 +0200 (CEST) Date: Thu, 22 Jun 2006 11:52:50 +0200 From: Pawel Jakub Dawidek To: Xin LI Message-ID: <20060622095250.GD30568@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <44984A91.8040805@rogers.com> <20060620193630.GA8007@garage.freebsd.pl> <1150833586.24301.1.camel@spirit> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jL2BoiuKMElzg3CS" Content-Disposition: inline In-Reply-To: <1150833586.24301.1.camel@spirit> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, Mike Jakubik , freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 09:55:34 -0000 --jL2BoiuKMElzg3CS Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jun 21, 2006 at 03:59:46AM +0800, Xin LI wrote: +> ??? 2006-06-20?????? 21:36 +0200???Pawel Jakub Dawidek????????? +> > The performance impact is big for large files, because in theory we ha= ve +> > to write the data twice. +> > Yes, it eliminates need for SU, but there are reasons, that you still +> > want to use SU, eg. for snapshots. +>=20 +> Em... IIRC SU and snapshots are independent, no? Oops. Yes, you are right. bgfsck depends on SU, not snapshots. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --jL2BoiuKMElzg3CS Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEmmhyForvXbEpPzQRAohQAJ9cSI5QpRK5ghxwucJ91+fshMSIigCfeDjD UHVHqWF9yKOf+SsGJViLP5c= =nTBz -----END PGP SIGNATURE----- --jL2BoiuKMElzg3CS-- From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 10:00:16 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2C2BC16A47B for ; Thu, 22 Jun 2006 10:00:16 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6807F43D8B for ; Thu, 22 Jun 2006 10:00:15 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 5936851814; Thu, 22 Jun 2006 12:00:14 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id A2EA550E81; Thu, 22 Jun 2006 12:00:10 +0200 (CEST) Date: Thu, 22 Jun 2006 11:57:32 +0200 From: Pawel Jakub Dawidek To: Rong-en Fan Message-ID: <20060622095732.GE30568@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <6eb82e0606201246n49474ea0of423bec93b13c293@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="SWTRyWv/ijrBap1m" Content-Disposition: inline In-Reply-To: <6eb82e0606201246n49474ea0of423bec93b13c293@mail.gmail.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 10:00:16 -0000 --SWTRyWv/ijrBap1m Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 20, 2006 at 03:46:17PM -0400, Rong-en Fan wrote: +> Hi, +>=20 +> This sounds very exciting to me! +>=20 +> I think I must miss something in this announcement. How +> much space will be used for journal? For example, if +> the provider is 80G, how much I will lost due to gjournal? The size of the journal don't depend on file system size. It more depend on your disk speed and file system load. For example your disk can write at 60MB/s. Journal switch time is 10 seconds. The journal provider has to have place to keep two journals (active and inactive). So bascially you need 60*10*2MB + gjournal headers. I think 2GB is a safe default. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --SWTRyWv/ijrBap1m Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEmmmMForvXbEpPzQRAsfQAJ9KBVPcs0YzYFpdB4KYnowL2THGAwCgrs3Z JDQGDV+fIrOY7wov1OkJRko= =qPbz -----END PGP SIGNATURE----- --SWTRyWv/ijrBap1m-- From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 10:08:59 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1E0A616A479; Thu, 22 Jun 2006 10:08:59 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6821B43D88; Thu, 22 Jun 2006 10:08:58 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 7920851814; Thu, 22 Jun 2006 12:08:57 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id DB27750EA7; Thu, 22 Jun 2006 12:08:51 +0200 (CEST) Date: Thu, 22 Jun 2006 12:06:14 +0200 From: Pawel Jakub Dawidek To: Bakul Shah Message-ID: <20060622100614.GF30568@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <20060620202948.933F2294C1@mail.bitblocks.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="VuQYccsttdhdIfIP" Content-Disposition: inline In-Reply-To: <20060620202948.933F2294C1@mail.bitblocks.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 10:08:59 -0000 --VuQYccsttdhdIfIP Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jun 20, 2006 at 01:29:48PM -0700, Bakul Shah wrote: +> This is great! We have sorely needed this for quite a while +> what with terabyte size filesystems getting into common use. +>=20 +> > How it works (in short). You may define one or two providers which +> > gjournal will use. If one provider is given, it will be used for both - +> > data and journal. If two providers are given, one will be used for data +> > and one for journal. +> > Every few seconds (you may define how many) journal is terminated and +> > marked as consistent and gjournal starts to copy data from it to the +> > data provider. In the same time new data are stored in new journal. +>=20 +> Some random comments: +>=20 +> Would it make sense to treat the journal as a circular +> buffer? Then commit to the underlying provider starts when +> the buffer has $hiwater blocks or the upper layer wants to +> sync. The commit stops when the buffer has $lowater blocks +> or in case of sync the buffer is empty. This will allow +> parallel writes to the provider and the journal, thereby +> reducing latency. This is bascially what is done now. There are always two journal - active and inactive. New data are written to the active journal. When journal switch time arrives (timeout occurs or cache is full), the active journal is terminated and new active journal is started right after this one. The previous active journal becomes inactive and the data is copied to the destination (data) provider in parallel to new requests which are stored in the active journal. Writes are suspended only on synchronize file system and terminate the active journal. Copying data from the inactive journal is done in parallel to normal operations. +> I don't understand why you need FS synchronization. Once the +> journal is written, the data is safe. [...] Which data? When you for example delete a file, you need to perform those operations: - remove name from a directory - mark inode as free - mark blocks as free Synchronizing file system gives me certainty that all those operations reached gjournal, so I can safely mark file system as clean and terminate the journal. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --VuQYccsttdhdIfIP Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEmmuWForvXbEpPzQRAkx5AKDwCF5E7AnzKAMcIrzidCm3f841GACfbc4W TbyLYq/4XoccF4Zb4qfRNac= =OMuu -----END PGP SIGNATURE----- --VuQYccsttdhdIfIP-- From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 13:59:29 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB1D716AA19; Thu, 22 Jun 2006 13:59:29 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 028BC444F7; Thu, 22 Jun 2006 13:37:18 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (qbqjyl@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k5MDbCHO071725; Thu, 22 Jun 2006 15:37:17 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k5MDbCBm071724; Thu, 22 Jun 2006 15:37:12 +0200 (CEST) (envelope-from olli) Date: Thu, 22 Jun 2006 15:37:12 +0200 (CEST) Message-Id: <200606221337.k5MDbCBm071724@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG, pjd@FreeBSD.ORG In-Reply-To: <20060622095732.GE30568@garage.freebsd.pl> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.0-20051224 ("Ronay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Thu, 22 Jun 2006 15:37:18 +0200 (CEST) Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-fs@FreeBSD.ORG, pjd@FreeBSD.ORG List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 13:59:30 -0000 Pawel Jakub Dawidek wrote: > Rong-en Fan wrote: > +> I think I must miss something in this announcement. How > +> much space will be used for journal? For example, if > +> the provider is 80G, how much I will lost due to gjournal? > > The size of the journal don't depend on file system size. It more depend > on your disk speed and file system load. For example your disk can write > at 60MB/s. Journal switch time is 10 seconds. The journal provider has > to have place to keep two journals (active and inactive). So bascially > you need 60*10*2MB + gjournal headers. I think 2GB is a safe default. But what happens in the case when you have only one provider? You wrote that you can specify only one provider (e.g. one partition) to gjournal, which will be used for both journal and file system data. In that case, how do you tell how much space is used for the journals and how much space is left for the file system? It didn't become clear from the examples in your initial mail. Another question: What happens if you (accidentally) make the journals provider too small, so it hits the end before the next regular switch? Best regards Oliver PS: I've also sent a follow-up to the -fs mailing list a few days ago, containing a few questions ... did you overlook it? PPS: Thanks again for all your work on journaling! It looks very cool and promising. Any plans to commit it? A wider audience will speed up the process of polishing the rough edges, I think. -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. > Can the denizens of this group enlighten me about what the > advantages of Python are, versus Perl ? "python" is more likely to pass unharmed through your spelling checker than "perl". -- An unknown poster and Fredrik Lundh From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 14:13:48 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B859416A47C for ; Thu, 22 Jun 2006 14:13:48 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 942B243D58 for ; Thu, 22 Jun 2006 14:13:46 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 889B351814; Thu, 22 Jun 2006 16:13:45 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 8B84D51307 for ; Thu, 22 Jun 2006 16:13:40 +0200 (CEST) Date: Thu, 22 Jun 2006 16:11:01 +0200 From: Pawel Jakub Dawidek To: freebsd-fs@FreeBSD.ORG Message-ID: <20060622141101.GI30568@garage.freebsd.pl> References: <20060622095732.GE30568@garage.freebsd.pl> <200606221337.k5MDbCBm071724@lurza.secnetix.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="CNK/L7dwKXQ4Ub8J" Content-Disposition: inline In-Reply-To: <200606221337.k5MDbCBm071724@lurza.secnetix.de> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 14:13:48 -0000 --CNK/L7dwKXQ4Ub8J Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 22, 2006 at 03:37:12PM +0200, Oliver Fromme wrote: +> Pawel Jakub Dawidek wrote: +> > Rong-en Fan wrote: +> > +> I think I must miss something in this announcement. How +> > +> much space will be used for journal? For example, if +> > +> the provider is 80G, how much I will lost due to gjournal? +> >=20 +> > The size of the journal don't depend on file system size. It more dep= end +> > on your disk speed and file system load. For example your disk can wr= ite +> > at 60MB/s. Journal switch time is 10 seconds. The journal provider has +> > to have place to keep two journals (active and inactive). So bascially +> > you need 60*10*2MB + gjournal headers. I think 2GB is a safe default. +>=20 +> But what happens in the case when you have only one provider? +> You wrote that you can specify only one provider (e.g. one +> partition) to gjournal, which will be used for both journal +> and file system data. In that case, how do you tell how much +> space is used for the journals and how much space is left for +> the file system? It didn't become clear from the examples in +> your initial mail. There is no manual page for gjournal(8) yet. There is an option you can use to specify journal size: # gjournal label -s 2147483648 ad0 This will create 2GB journal and the rest will be for the data. +> Another question: What happens if you (accidentally) make +> the journals provider too small, so it hits the end before +> the next regular switch? Currently it will panic, but I'm working on it. +> PS: I've also sent a follow-up to the -fs mailing list a few +> days ago, containing a few questions ... did you overlook it? If you haven't CCed me, I probably did. +> PPS: Thanks again for all your work on journaling! It +> looks very cool and promising. Any plans to commit it? +> A wider audience will speed up the process of polishing +> the rough edges, I think. There are some decisions I need to make before it will go to HEAD. Like 'gjournal' mount option. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --CNK/L7dwKXQ4Ub8J Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEmqT1ForvXbEpPzQRAn0RAKCbLysV/EiK6fdluB+/zdR6DwJHtACfYJ90 OBNsnhHDk+tq24rRDxhySm8= =B8H0 -----END PGP SIGNATURE----- --CNK/L7dwKXQ4Ub8J-- From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 14:45:06 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AF6E716A482; Thu, 22 Jun 2006 14:45:06 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6A7DC43D58; Thu, 22 Jun 2006 14:45:05 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (wxklmr@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k5MEiwgp074697; Thu, 22 Jun 2006 16:45:03 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k5MEiwHu074696; Thu, 22 Jun 2006 16:44:58 +0200 (CEST) (envelope-from olli) Date: Thu, 22 Jun 2006 16:44:58 +0200 (CEST) Message-Id: <200606221444.k5MEiwHu074696@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG, pjd@FreeBSD.ORG In-Reply-To: <20060622141101.GI30568@garage.freebsd.pl> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.0-20051224 ("Ronay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Thu, 22 Jun 2006 16:45:04 +0200 (CEST) Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-fs@FreeBSD.ORG, pjd@FreeBSD.ORG List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 14:45:06 -0000 Pawel Jakub Dawidek wrote: > There is no manual page for gjournal(8) yet. There is an option you can > use to specify journal size: > > # gjournal label -s 2147483648 ad0 > > This will create 2GB journal and the rest will be for the data. Cool. I guess you can also specify the journal switch interval as an option, right? Oh, by the way: Are both UFS1 and UFS2 supported? > +> Another question: What happens if you (accidentally) make > +> the journals provider too small, so it hits the end before > +> the next regular switch? > > Currently it will panic, but I'm working on it. OK. Actually I already guessed that it would panic. :-) It probably would be better to block all write requests and wait until the other journal has been written out, so it can be re-used. And print(9) a message so the admin knows what kind of problem there is. However, if the size of the journals provider is twice the maximum raw transfer rate of the disk multiplied by the switch interval time (as you recommended), it should be impossible to get a journal overrun. > +> PS: I've also sent a follow-up to the -fs mailing list a few > +> days ago, containing a few questions ... did you overlook it? > > If you haven't CCed me, I probably did. I see. I'm sorry for that. I'll forward you a copy if you don't mind. > +> PPS: Thanks again for all your work on journaling! It > +> looks very cool and promising. Any plans to commit it? > +> A wider audience will speed up the process of polishing > +> the rough edges, I think. > > There are some decisions I need to make before it will go to HEAD. Like > 'gjournal' mount option. I see. I hope there won't be problems with that. Bringing journaling support to UFS is a significant milestone in FreeBSD's history. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. With Perl you can manipulate text, interact with programs, talk over networks, drive Web pages, perform arbitrary precision arithmetic, and write programs that look like Snoopy swearing. From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 15:22:37 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF1A016A7C3; Thu, 22 Jun 2006 15:22:37 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3507344596; Thu, 22 Jun 2006 14:59:29 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k5MExToe049632; Thu, 22 Jun 2006 09:59:29 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <449AB05D.3030402@centtech.com> Date: Thu, 22 Jun 2006 09:59:41 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: freebsd-fs@freebsd.org, pjd@freebsd.org References: <200606221444.k5MEiwHu074696@lurza.secnetix.de> In-Reply-To: <200606221444.k5MEiwHu074696@lurza.secnetix.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1560/Thu Jun 22 04:07:47 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 15:22:37 -0000 Oliver Fromme wrote: > Pawel Jakub Dawidek wrote: > > There is no manual page for gjournal(8) yet. There is an option you can > > use to specify journal size: > > > > # gjournal label -s 2147483648 ad0 > > > > This will create 2GB journal and the rest will be for the data. > > Cool. I guess you can also specify the journal switch > interval as an option, right? > > Oh, by the way: Are both UFS1 and UFS2 supported? > > > +> Another question: What happens if you (accidentally) make > > +> the journals provider too small, so it hits the end before > > +> the next regular switch? > > > > Currently it will panic, but I'm working on it. > > OK. Actually I already guessed that it would panic. :-) > It probably would be better to block all write requests > and wait until the other journal has been written out, so > it can be re-used. And print(9) a message so the admin > knows what kind of problem there is. > > However, if the size of the journals provider is twice the > maximum raw transfer rate of the disk multiplied by the > switch interval time (as you recommended), it should be > impossible to get a journal overrun. What if the journal had to write out little bits across the entire journaled area, so the throughput was greatly reduced, while the journaled area had massive heavy random reads keeping the disk heads very busy? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 15:30:08 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 156EE16A47C for ; Thu, 22 Jun 2006 15:30:08 +0000 (UTC) (envelope-from pedro@ambientworks.net) Received: from protection.cx (protection.cx [209.242.20.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 157F243D49 for ; Thu, 22 Jun 2006 15:29:59 +0000 (GMT) (envelope-from pedro@ambientworks.net) Received: by protection.cx (Postfix, from userid 1001) id 9AC034C; Thu, 22 Jun 2006 10:35:04 -0500 (CDT) Date: Thu, 22 Jun 2006 12:35:04 -0300 From: Pedro Martelletto To: freebsd-fs@freebsd.org Message-ID: <20060622153504.GB835@static.protection.cx> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Subject: plug memory leaks and fix nested loops in udf_find_partmaps() X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 15:30:08 -0000 currently, there are two nested 'for' loops in udf_find_partmaps() which use the same control variable (i), as well as memory leaks in two error paths, which the following diff should fix. -p. Index: udf_vfsops.c =================================================================== RCS file: /home/ncvs/src/sys/fs/udf/udf_vfsops.c,v retrieving revision 1.41 diff -u -p -r1.41 udf_vfsops.c --- udf_vfsops.c 26 May 2006 01:21:51 -0000 1.41 +++ udf_vfsops.c 22 Jun 2006 15:08:25 -0000 @@ -728,7 +728,7 @@ udf_find_partmaps(struct udf_mnt *udfmp, struct regid *pmap_id; struct buf *bp; unsigned char regid_id[UDF_REGID_ID_SIZE + 1]; - int i, ptype, psize, error; + int i, k, ptype, psize, error; for (i = 0; i < le32toh(lvd->n_pm); i++) { pmap = (union udf_pmap *)&lvd->maps[i * UDF_PMAP_SIZE]; @@ -776,6 +776,7 @@ udf_find_partmaps(struct udf_mnt *udfmp, brelse(bp); printf("Failed to read Sparing Table at sector %d\n", le32toh(pms->st_loc[0])); + FREE(udfmp->s_table, M_UDFMOUNT); return (error); } bcopy(bp->b_data, udfmp->s_table, le32toh(pms->st_size)); @@ -783,15 +784,16 @@ udf_find_partmaps(struct udf_mnt *udfmp, if (udf_checktag(&udfmp->s_table->tag, 0)) { printf("Invalid sparing table found\n"); + FREE(udfmp->s_table, M_UDFMOUNT); return (EINVAL); } /* See how many valid entries there are here. The list is * supposed to be sorted. 0xfffffff0 and higher are not valid */ - for (i = 0; i < le16toh(udfmp->s_table->rt_l); i++) { - udfmp->s_table_entries = i; - if (le32toh(udfmp->s_table->entries[i].org) >= + for (k = 0; k < le16toh(udfmp->s_table->rt_l); k++) { + udfmp->s_table_entries = k; + if (le32toh(udfmp->s_table->entries[k].org) >= 0xfffffff0) break; } From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 15:46:31 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 96F1B16A47B; Thu, 22 Jun 2006 15:46:31 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE4CC43D53; Thu, 22 Jun 2006 15:46:30 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (chobmt@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k5MFkOMW076906; Thu, 22 Jun 2006 17:46:29 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k5MFkO59076905; Thu, 22 Jun 2006 17:46:24 +0200 (CEST) (envelope-from olli) Date: Thu, 22 Jun 2006 17:46:24 +0200 (CEST) Message-Id: <200606221546.k5MFkO59076905@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG, pjd@FreeBSD.ORG In-Reply-To: <449AB05D.3030402@centtech.com> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.0-20051224 ("Ronay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Thu, 22 Jun 2006 17:46:29 +0200 (CEST) Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-fs@FreeBSD.ORG, pjd@FreeBSD.ORG List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 15:46:31 -0000 Eric Anderson wrote: > Oliver Fromme wrote: > > [...] > > However, if the size of the journals provider is twice the > > maximum raw transfer rate of the disk multiplied by the I should have been more specific: "of the disk containing the _journal_ provider", of course. > > switch interval time (as you recommended), it should be > > impossible to get a journal overrun. > > What if the journal had to write out little bits across the entire > journaled area, so the throughput was greatly reduced, while the > journaled area had massive heavy random reads keeping the disk heads > very busy? In that case there is even less danger of an overrun, because the many seek operations of the disk will reduce the throughput, so it will be far from reaching the end of the journal space before then end of the journal switch interval. Pawel, please correct me if I'm wrong. The maximum raw transfer rate of the journal provider is a hard limit which cannot be exceeded. If you configure the size of the journal provider to be twice that maximum rate multiplied by the switch interval, there is simply no way to hit the end of the journal space before the end of the switch interval. The worst case would happen when there have been no write operations for a while, so the journal is empty, and the inactive journal does not need to be read (i.e. the disk is idle). Now a huge amount of write operations start, causing to fill the active journal sequentially with the maximum possible throughput (almost no seek operations of the disk heads). The recommendation for the size of the journal provider should account for that worst case. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "One of the main causes of the fall of the Roman Empire was that, lacking zero, they had no way to indicate successful termination of their C programs." -- Robert Firth From owner-freebsd-fs@FreeBSD.ORG Thu Jun 22 21:45:33 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2F0C516A492 for ; Thu, 22 Jun 2006 21:45:33 +0000 (UTC) (envelope-from maxsec@gmail.com) Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.228]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2E56743E31 for ; Thu, 22 Jun 2006 21:12:29 +0000 (GMT) (envelope-from maxsec@gmail.com) Received: by wr-out-0506.google.com with SMTP id 50so490055wri for ; Thu, 22 Jun 2006 14:12:29 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:subject:cc:in-reply-to:mime-version:content-type:references; b=ZgK0NmDL0VkabBqMH0KPNcQosyfIis1rhQzdd5X4FDApPTAmO0Al3O0Th2jZjT7+Q8wF4Q4EmBnQRmERl0KpqPiZHBQJUOoRPcu72RrnH7Tat1u+cWaeaI66FCVbu4gv9ash33B32uw4ABgEY7PtyAS5K9R03mKEbMLZVs1q3CA= Received: by 10.54.76.3 with SMTP id y3mr2534607wra; Thu, 22 Jun 2006 14:12:29 -0700 (PDT) Received: by 10.54.107.12 with HTTP; Thu, 22 Jun 2006 14:12:29 -0700 (PDT) Message-ID: <72cf361e0606221412l72dde88co74082023f13cf437@mail.gmail.com> Date: Thu, 22 Jun 2006 22:12:29 +0100 From: "Martin Hepworth" In-Reply-To: MIME-Version: 1.0 References: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, freebsd-questions@freebsd.org Subject: Re: HELP! Filesystem EMPTY after upgrade from 4.11 to 6.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2006 21:45:34 -0000 Sven ok so no backup beforehand then... Have you tried moving the drive to a 4.11 system and mounting the partition from there, and then backing-up the data! Might be some way of forcing UFS1 over UFS2 on the mount command, but I don't see any mention of this in the man page (which still says 4.11 on the online version!) -- Martin On 6/21/06, Sven Hazejager wrote: > > Hi all! > > I have quite a big problem here.... > > I've upgraded my FreeBSD 4.11 to 6.1. Basically, I did a newfs of / and > /usr > and reinstalled from scratch. That worked. > > Rebooted in 6.1 single user mode, mounted /, /usr and /usr/home. The > latter > I did not touch and I need to save that partition. /usr/home had all the > files it needed to have. PROBLEM: mount said soft-updates were not set on > /usr/home. So I did umount /usr/home, tunefs -n enable /usr/home, mount > /usr/home again... EMPTY!!!! > > What has gone wrong? I did fsck (in read-only), no problems, rebooted the > system, did tunefs -n disable, still no luck. I really need those files > guys... > > Please help urgently. > > Many thanks, > > Sven Hazejager > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to " > freebsd-questions-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 08:22:39 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BA78D16A492; Fri, 23 Jun 2006 08:22:39 +0000 (UTC) (envelope-from never@kurush.osdn.org.ua) Received: from kurush.osdn.org.ua (external.osdn.org.ua [212.40.34.156]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9C06C43D46; Fri, 23 Jun 2006 08:22:22 +0000 (GMT) (envelope-from never@kurush.osdn.org.ua) Received: from kurush.osdn.org.ua (never@localhost [127.0.0.1]) by kurush.osdn.org.ua (8.12.11/8.12.11) with ESMTP id k5N8MAlM073999; Fri, 23 Jun 2006 11:22:11 +0300 (EEST) (envelope-from never@kurush.osdn.org.ua) Received: (from never@localhost) by kurush.osdn.org.ua (8.12.11/8.12.11/Submit) id k5N8M9vB073996; Fri, 23 Jun 2006 11:22:09 +0300 (EEST) (envelope-from never) Date: Fri, 23 Jun 2006 11:22:09 +0300 From: Alexandr Kovalenko To: Pawel Jakub Dawidek Message-ID: <20060623082209.GD13474@nevermind.kiev.ua> References: <20060619131101.GD1130@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060619131101.GD1130@garage.freebsd.pl> User-Agent: Mutt/1.5.4i Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 08:22:39 -0000 Hello, Pawel Jakub Dawidek! On Mon, Jun 19, 2006 at 03:11:01PM +0200, you wrote: > For the last few months I have been working on gjournal project. > To stop confusion right here, I want to note, that this project is not > related to gjournal project on which Ivan Voras was working on the > last SoC (2005). [dd] > Quick start: > > # gjournal label /dev/ad0 > # gjournal load > # newfs /dev/ad0.journal > # mount -o async,gjournal /dev/ad0.journal /mnt > (yes, with gjournal 'async' is safe) > > Now, after a power failure or system crash no fsck is needed (yay!). Is it safe to do so on existing filesystem (if I'm using 2nd partition for journal)? i.e.: $ grep ad0s1f /etc/fstab /dev/ad0s1f /usr ufs rw,noatime 2 2 $ grep ad0s1b /etc/fstab #/dev/ad0s1b none swap sw 0 0 # gjournal label ad0s1f ad0s1b # gjournal load # fsck -y /dev/ad0s1f.journal # sed -i -e 's|ad0s1f|ad0s1f.journal|' /etc/fstab # sed -i -e 's|noatime|noatime,async,gjournal|' /etc/fstab # mount /usr -- NEVE-RIPE, will build world for food Ukrainian FreeBSD User Group http://uafug.org.ua/ From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 08:38:39 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EA38A16A4A0 for ; Fri, 23 Jun 2006 08:38:39 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30308.mail.mud.yahoo.com (web30308.mail.mud.yahoo.com [68.142.200.101]) by mx1.FreeBSD.org (Postfix) with SMTP id A697443D49 for ; Fri, 23 Jun 2006 08:38:38 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 86541 invoked by uid 60001); 23 Jun 2006 08:38:38 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=F3rZ2dXJ6OklEpWvhqIZ/euBNIWOJXDu3T/EMhEpd0s2JcjwrQPSfF6BUbAkJ/PtxcDyK9l/Ss6/FFXwxXwyd97wx/GwUy4Le5mJrat+qqGnpHxGkMqEi3ID4qeR+Cni6jXZ0dvb5husRPpXvNJO465S63ldbwJu5CtBda7i7oY= ; Message-ID: <20060623083838.86539.qmail@web30308.mail.mud.yahoo.com> Received: from [213.54.80.34] by web30308.mail.mud.yahoo.com via HTTP; Fri, 23 Jun 2006 01:38:38 PDT Date: Fri, 23 Jun 2006 01:38:38 -0700 (PDT) From: "R. B. Riddick" To: Alexandr Kovalenko In-Reply-To: <20060623082209.GD13474@nevermind.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 08:38:40 -0000 --- Alexandr Kovalenko wrote: > Is it safe to do so on existing filesystem (if I'm using 2nd partition for > journal)? > Hmm... Depends: If your existing file system needs its last sector, then it wont work. If it does not need it, then it might work (although fsck does not check for a raw-device shrinkage - I think)... I say, can you make the size of ad0s1f one sector bigger with bsdlabel(8) without changing the start sector? I mean: Is there at least one free sector after ad0s1f? -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 08:53:33 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4520816A494; Fri, 23 Jun 2006 08:53:33 +0000 (UTC) (envelope-from never@kurush.osdn.org.ua) Received: from kurush.osdn.org.ua (external.osdn.org.ua [212.40.34.156]) by mx1.FreeBSD.org (Postfix) with ESMTP id 61F3743D45; Fri, 23 Jun 2006 08:52:49 +0000 (GMT) (envelope-from never@kurush.osdn.org.ua) Received: from kurush.osdn.org.ua (never@localhost [127.0.0.1]) by kurush.osdn.org.ua (8.12.11/8.12.11) with ESMTP id k5N8qbal076002; Fri, 23 Jun 2006 11:52:37 +0300 (EEST) (envelope-from never@kurush.osdn.org.ua) Received: (from never@localhost) by kurush.osdn.org.ua (8.12.11/8.12.11/Submit) id k5N8qbof075999; Fri, 23 Jun 2006 11:52:37 +0300 (EEST) (envelope-from never) Date: Fri, 23 Jun 2006 11:52:37 +0300 From: Alexandr Kovalenko To: "R. B. Riddick" Message-ID: <20060623085236.GE13474@nevermind.kiev.ua> References: <20060623082209.GD13474@nevermind.kiev.ua> <20060623083838.86539.qmail@web30308.mail.mud.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060623083838.86539.qmail@web30308.mail.mud.yahoo.com> User-Agent: Mutt/1.5.4i Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 08:53:33 -0000 Hello, R. B. Riddick! On Fri, Jun 23, 2006 at 01:38:38AM -0700, you wrote: > --- Alexandr Kovalenko wrote: > > Is it safe to do so on existing filesystem (if I'm using 2nd partition for > > journal)? > > > Hmm... > > Depends: > If your existing file system needs its last sector, then it wont work. If it > does not need it, then it might work (although fsck does not check for a > raw-device shrinkage - I think)... > > I say, can you make the size of ad0s1f one sector bigger with bsdlabel(8) > without changing the start sector? > I mean: Is there at least one free sector after ad0s1f? Unfortunately - no :( -- NEVE-RIPE, will build world for food Ukrainian FreeBSD User Group http://uafug.org.ua/ From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 11:36:36 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 71D1E16A49A; Fri, 23 Jun 2006 11:36:36 +0000 (UTC) (envelope-from rubenl@bloemgarten.demon.nl) Received: from post-25.mail.nl.demon.net (post-25.mail.nl.demon.net [194.159.73.195]) by mx1.FreeBSD.org (Postfix) with ESMTP id C4A9D43D49; Fri, 23 Jun 2006 11:36:35 +0000 (GMT) (envelope-from rubenl@bloemgarten.demon.nl) Received: from axeldev.demon.nl ([83.160.42.116]:53421 helo=abubbletprpdda) by post-25.mail.nl.demon.net with esmtp (Exim 4.51) id 1Ftjxi-000C5h-Hw; Fri, 23 Jun 2006 11:36:34 +0000 From: "Ruben Bloemgarten" To: "'Sven Hazejager'" , , Date: Fri, 23 Jun 2006 13:37:14 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1807 Thread-Index: AcaVbnGDqABZERGVStetswNs6E1LcwBSnreQ Message-Id: <20060623113635.C4A9D43D49@mx1.FreeBSD.org> Cc: Subject: RE: HELP! Filesystem EMPTY after upgrade from 4.11 to 6.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ruben@bloemgarten.demon.nl List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 11:36:36 -0000 Hi Sven, How was the disk labeled originally? Is it possible that /usr/home did not reside on a dedicated slice and when you newfs'd /usr you wiped /usr/home ? Regards, Ruben -----Original Message----- From: owner-freebsd-questions@freebsd.org [mailto:owner-freebsd-questions@freebsd.org] On Behalf Of Sven Hazejager Sent: June 21, 2006 10:07 PM To: freebsd-questions@freebsd.org; freebsd-fs@freebsd.org Subject: HELP! Filesystem EMPTY after upgrade from 4.11 to 6.1 Hi all! I have quite a big problem here.... I've upgraded my FreeBSD 4.11 to 6.1. Basically, I did a newfs of / and /usr and reinstalled from scratch. That worked. Rebooted in 6.1 single user mode, mounted /, /usr and /usr/home. The latter I did not touch and I need to save that partition. /usr/home had all the files it needed to have. PROBLEM: mount said soft-updates were not set on /usr/home. So I did umount /usr/home, tunefs -n enable /usr/home, mount /usr/home again... EMPTY!!!! What has gone wrong? I did fsck (in read-only), no problems, rebooted the system, did tunefs -n disable, still no luck. I really need those files guys... Please help urgently. Many thanks, Sven Hazejager _______________________________________________ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org" -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.1.394 / Virus Database: 268.9.2/370 - Release Date: 06/20/2006 -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.1.394 / Virus Database: 268.9.2/370 - Release Date: 06/20/2006 From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 15:20:28 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5D82116A47E; Fri, 23 Jun 2006 15:20:28 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 00E0543D46; Fri, 23 Jun 2006 15:20:27 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k5NFKQtN079480; Fri, 23 Jun 2006 10:20:27 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <449C06C6.9070801@centtech.com> Date: Fri, 23 Jun 2006 10:20:38 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20060619131101.GD1130@garage.freebsd.pl> In-Reply-To: <20060619131101.GD1130@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1562/Fri Jun 23 02:50:07 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 15:20:28 -0000 Pawel Jakub Dawidek wrote: > Hello. > > For the last few months I have been working on gjournal project. > To stop confusion right here, I want to note, that this project is not > related to gjournal project on which Ivan Voras was working on the > last SoC (2005). > > The lack of journaled file system in FreeBSD was a tendon of achilles > for many years. We do have many file systems, but none with journaling: > - ext2fs (journaling is in ext3fs), > - XFS (read-only), > - ReiserFS (read-only), > - HFS+ (read-write, but without journaling), > - NTFS (read-only). > > GJournal was designed to journal GEOM providers, so it actually works > below file system layer, but it has hooks which allow to work with > file systems. In other words, gjournal is not file system-depended, > it can work probably with any file system with minimum knowledge > about it. I implemented only UFS support. > > The patches are here: > > http://people.freebsd.org/~pjd/patches/gjournal.patch (for HEAD) > http://people.freebsd.org/~pjd/patches/gjournal6.patch (for RELENG_6) > > To patch your sources you need to: > > # cd /usr/src > # mkdir sbin/geom/class/journal sys/geom/journal sys/modules/geom/geom_journal > # patch < /path/to/gjournal.patch > > Add 'options UFS_GJOURNAL' to your kernel configuration file and > recompile kernel and world. > > How it works (in short). You may define one or two providers which > gjournal will use. If one provider is given, it will be used for both - > data and journal. If two providers are given, one will be used for data > and one for journal. > Every few seconds (you may define how many) journal is terminated and > marked as consistent and gjournal starts to copy data from it to the > data provider. In the same time new data are stored in new journal. I'm not sure this is happening the way you describe exactly. On my laptop, while rsyncing my /home partition to a newly created external disk (400G), I see 20MB/s writing to the journaled UFS2 device (/dev/label/backup.journal) passing through to the journal device (/dev/label/journal), then it switches to no writes to the journaled UFS2 device (/dev/label/backup.journal) (my rsync pauses) while the journaled device (/dev/label/backup) writes at 20MB/s for about 3-10 seconds. > Let's call the moment in which journal is terminated as "journal switch". > Journal switch looks as follows: > 1. Start journal switch if we have timeout or if we run out of cache. > Don't perform journal switch if there were no write requests. > 2. If we have file system, synchronize it. > 3. Mark file system as clean. > 4. Block all write requests to the file system. > 5. Terminate the journal. > 6. Eventually wait if copying of the previous journal is not yet > finished. Seems like this is the point we are busy in. > 7. Send BIO_FLUSH request (if the given provider supports it). > 8. Mark new journal position on the journal provider. > 9. Unblock write requests. > 10. Start copying data from the terminated journal to the data provider. And it seems that 10 is happening earlier on.. Is this all expected behaviour? Thanks for the great work, and fantastic GEOM tools! Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 16:53:00 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5513216A47B for ; Fri, 23 Jun 2006 16:53:00 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 791AF43D58 for ; Fri, 23 Jun 2006 16:52:57 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (ufwdix@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k5NGqoKl023671 for ; Fri, 23 Jun 2006 18:52:56 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k5NGqoQ0023670; Fri, 23 Jun 2006 18:52:50 +0200 (CEST) (envelope-from olli) Date: Fri, 23 Jun 2006 18:52:50 +0200 (CEST) Message-Id: <200606231652.k5NGqoQ0023670@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG In-Reply-To: <20060623083838.86539.qmail@web30308.mail.mud.yahoo.com> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.0-20051224 ("Ronay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Fri, 23 Jun 2006 18:52:56 +0200 (CEST) Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-fs@FreeBSD.ORG List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 16:53:00 -0000 R. B. Riddick wrote: > Alexandr Kovalenko wrote: > > Is it safe to do so on existing filesystem (if I'm using 2nd partition for > > journal)? > > Depends: > If your existing file system needs its last sector, then it wont work. If it > does not need it, then it might work (although fsck does not check for a > raw-device shrinkage - I think)... It has no way to check it. If the last sector of the partition happens to be part of file data, overwriting it with gjournal meta data will lead to a corrupted file, and fsck(8) has no way to notice that, of course. If that sector happens to contain UFS meta data, fsck(8) might detect the corruption and try to correct it, which will destroy the gjournal meta data. I guess that both cases are very, very bad. :-) It's not difficult to check if the last sector is in use or not. Just repeat the newfs(8) with the -N flag, so it prints out the values without doing anything (you can even do this as normal user, not root). For example: $ bsdlabel /dev/ad0s1 | grep a: a: 488397105 0 4.2BSD 2048 16384 106 # (Cyl. 0 - 484520*) $ newfs -N /dev/ad0s1a Warning: Block size and bytes per inode restrict cylinders per group to 89. Warning: 1744 sector(s) in last cylinder unallocated /dev/ad0s1a: 488397104 sectors in 119238 cylinders of 1 tracks, 4096 sectors 238475.1MB in 1340 cyl groups (89 c/g, 178.00MB/g, 22528 i/g) In that case, the last sector is not used by the file system. (Of course, if you created the FS with special options, e.g. different cylinder group size, you must specify those options here, too, or you might get wrong output.) FreeBSD does have growfs(8), but unfortunately it still doesn't have shrinkfs(8), which other operating systems have (e.g. Solaris). It might be a nice project for a junior FS hacker ... ;-) Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "... there are two ways of constructing a software design: One way is to make it so simple that there are _obviously_ no deficiencies and the other way is to make it so complicated that there are no _obvious_ deficiencies." -- C.A.R. Hoare, ACM Turing Award Lecture, 1980 From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 17:13:09 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 909FB16A492 for ; Fri, 23 Jun 2006 17:13:09 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30305.mail.mud.yahoo.com (web30305.mail.mud.yahoo.com [68.142.200.98]) by mx1.FreeBSD.org (Postfix) with SMTP id E010343D45 for ; Fri, 23 Jun 2006 17:13:08 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 87890 invoked by uid 60001); 23 Jun 2006 17:13:07 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=c1P/hIBBT7bdElyW8kQvCH4W2hX8uVLdxUUwlFn4bsiT7+sWjkGoZbrqvHMl3zE/sUehxrdFno8tUAP9Tgvt3LysB2OjKEAaG7q9DcR0DUJn0G9vnvy8ZmPOTegnvRnZGQhOdIwKn0ohRQSAw3WwH6c9QNopwQvfbWuTFj9IItY= ; Message-ID: <20060623171307.87888.qmail@web30305.mail.mud.yahoo.com> Received: from [213.54.81.91] by web30305.mail.mud.yahoo.com via HTTP; Fri, 23 Jun 2006 10:13:07 PDT Date: Fri, 23 Jun 2006 10:13:07 -0700 (PDT) From: "R. B. Riddick" To: freebsd-fs@FreeBSD.ORG In-Reply-To: <200606231652.k5NGqoQ0023670@lurza.secnetix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 17:13:09 -0000 --- Oliver Fromme wrote: > R. B. Riddick wrote: > > Depends: > > If your existing file system needs its last sector, > > then it wont work. If it does not need it, then it > > might work (although fsck does not check for a > > raw-device shrinkage - I think)... > > It has no way to check it. > Hmm... Maybe it is not so interesting, but I wish fsck could say, if the partition in question is as big as it was when newfs built the file system... > file, and fsck(8) has no way to notice that, of course. > fsck could not even read the last sector of the former file system, because gjournal took it away... Or doesn't it? > If that sector happens to contain UFS meta data, fsck(8) > might detect the corruption and try to correct it, which > will destroy the gjournal meta data. > I think write access to that sector is not possible. Furthermore the device that is handed to fsck does not contain that meta data sector... At least I hope, that it is like that with my geom_mirror and geom_stripe discs... ;-) -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 17:42:15 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0839E16A47B for ; Fri, 23 Jun 2006 17:42:15 +0000 (UTC) (envelope-from user@dhp.com) Received: from shell.dhp.com (shell.dhp.com [199.245.105.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B06F43D45 for ; Fri, 23 Jun 2006 17:42:14 +0000 (GMT) (envelope-from user@dhp.com) Received: by shell.dhp.com (Postfix, from userid 896) id A16C431315; Fri, 23 Jun 2006 13:42:08 -0400 (EDT) Date: Fri, 23 Jun 2006 13:42:08 -0400 (EDT) From: Ensel Sharon To: freebsd-fs@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: 6.1 quota bugs cause adaptec 2820sa kernel to crash ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 17:42:15 -0000 FreeBSD 6.1-RELEASE, p4 xeon system, adaptec 2820sa SATA raid controller with all 8 disks in use. Two arrays are present - a mirror to boot from, and a 6-disk raid6 array of size 1.8TB. I am aware that there are problems with quotas in 6.1, but I am successfully using them with 6.0-RELEASE and an adaptec 1610sa. I figured it couldn't be any worse... ----- After loading, the system frequently (multi-daily) crashed with the error: Warning! Controller is no longer running! code=0xbcef0100 (after a page or so of aac0 timeout messages) So I disabled quotas on the system, and it has been completely stable ever since. ----- Does anyone understand the mechanism wherein the 6.1 quota problems could cause a raid controller kernel to crash ? Is that possible ? (it seems to be) Are there workaround for the 6.1 quota problems, as in "they don't work, but if you sysctl this and set this, they will" ? How close are we to working quota code on 6.1, and will I be able to just patch some files in /usr/src and recompile my kernel, or will it require a full rebuild of the OS ? Thank you. From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 19:14:14 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E2E3716A4A7 for ; Fri, 23 Jun 2006 19:14:14 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6B7CA43D49 for ; Fri, 23 Jun 2006 19:14:12 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id E78B05131F; Fri, 23 Jun 2006 21:14:10 +0200 (CEST) Received: from localhost (dlk232.neoplus.adsl.tpnet.pl [83.24.40.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 5D05B50EA7 for ; Fri, 23 Jun 2006 21:14:05 +0200 (CEST) Date: Fri, 23 Jun 2006 21:11:25 +0200 From: Pawel Jakub Dawidek To: freebsd-fs@FreeBSD.ORG Message-ID: <20060623191125.GA40269@garage.freebsd.pl> References: <20060622141101.GI30568@garage.freebsd.pl> <200606221444.k5MEiwHu074696@lurza.secnetix.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="J2SCkAp4GZ/dPZZf" Content-Disposition: inline In-Reply-To: <200606221444.k5MEiwHu074696@lurza.secnetix.de> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 19:14:15 -0000 --J2SCkAp4GZ/dPZZf Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 22, 2006 at 04:44:58PM +0200, Oliver Fromme wrote: +> Pawel Jakub Dawidek wrote: +> > There is no manual page for gjournal(8) yet. There is an option you c= an +> > use to specify journal size: +> >=20 +> > # gjournal label -s 2147483648 ad0 +> >=20 +> > This will create 2GB journal and the rest will be for the data. +>=20 +> Cool. I guess you can also specify the journal switch +> interval as an option, right? You can do it globally by setting kern.geom.journal.switch_time sysctl. +> Oh, by the way: Are both UFS1 and UFS2 supported? I tested only UFS2, but it should work for both. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --J2SCkAp4GZ/dPZZf Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEnDzdForvXbEpPzQRAus+AKDvGXiZc5m91GtfNIAbuA4d9B5iEgCfQTrB clOApXIXlPgqjbCZTkq8lk8= =zZ2J -----END PGP SIGNATURE----- --J2SCkAp4GZ/dPZZf-- From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 19:23:07 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A495916A492 for ; Fri, 23 Jun 2006 19:23:07 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id F39D343D48 for ; Fri, 23 Jun 2006 19:23:06 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id EEF3F51814; Fri, 23 Jun 2006 21:23:05 +0200 (CEST) Received: from localhost (dlk232.neoplus.adsl.tpnet.pl [83.24.40.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 20EE950EA7 for ; Fri, 23 Jun 2006 21:23:01 +0200 (CEST) Date: Fri, 23 Jun 2006 21:20:21 +0200 From: Pawel Jakub Dawidek To: freebsd-fs@freebsd.org Message-ID: <20060623192021.GB40269@garage.freebsd.pl> References: <200606221500.k5MF038R074998@lurza.secnetix.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/WwmFnJnmDyWGHa4" Content-Disposition: inline In-Reply-To: <200606221500.k5MF038R074998@lurza.secnetix.de> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 19:23:07 -0000 --/WwmFnJnmDyWGHa4 Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 22, 2006 at 05:00:03PM +0200, Oliver Fromme wrote: +> Pawel Jakub Dawidek wrote: +> > GJournal was designed to journal GEOM providers, so it actually works +> > below file system layer, but it has hooks which allow to work with +> > file systems. In other words, gjournal is not file system-depended, +> > it can work probably with any file system with minimum knowledge +> > about it. I implemented only UFS support. +>=20 +> Very cool. Thanks for providing journaling to FreeBSD. +>=20 +> How "stable" should your code be considered? I assume it +> hasn't been subject to testing by a wider audience yet, +> so it should be considered "alpha" quality, right? It works currently on about 100 heavy loaded servers in production and it works well. There is still one issue I'm trying to track down, but it is very hard to trigger. It doesn't currupt the data, but can hang the system. But. GJournal was tested only on a specific workload, so it should be considered experimental! +> First of all, is it possible to add journaling to an +> existing file system, i.e. without having to dump, newfs +> and restore? If you have an additional partition for journal you may try, but gjournal will steal the last sector from the provider holding your file system. I can't tell you it is safe or not. +> > When last +> > name is deleted, the file/directory is moved to the .deleted/ +> > directory and removed from there on last close. +>=20 +> Is that directory located in the root of the file system, +> similar to the .snap directory for snapshots? Yes. +> > [...] +> > Reading. grep -r on two src/ directories in parallel: +> > UFS: 84s +> > UFS+SU: 138s +> > gjournal(1): 102s +> > gjournal(2): 89s +> >=20 +> > As you can see, even on one disk, untaring eight src.tgz is two times +> > faster than UFS+SU. I've no idea why gjournal is faster in reading. +>=20 +> Maybe it is because of the atime updates. You'll get a lot +> of them when grepping recursively on the src tree. AFAIR file system was mounted with noatime, but I'm not sure. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --/WwmFnJnmDyWGHa4 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD4DBQFEnD71ForvXbEpPzQRAj0QAJiE7eY3lcf6nA9WgryawCQfxfLQAKCheFnQ WJ2B6Me3/EG2yu5IZr4VNQ== =z0hi -----END PGP SIGNATURE----- --/WwmFnJnmDyWGHa4-- From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 19:42:24 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 90E2F16A494; Fri, 23 Jun 2006 19:42:24 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7C39D43D8A; Fri, 23 Jun 2006 19:41:44 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 8F4C55131F; Fri, 23 Jun 2006 21:41:42 +0200 (CEST) Received: from localhost (dlk232.neoplus.adsl.tpnet.pl [83.24.40.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 0634E50EA7; Fri, 23 Jun 2006 21:41:36 +0200 (CEST) Date: Fri, 23 Jun 2006 21:38:57 +0200 From: Pawel Jakub Dawidek To: Eric Anderson Message-ID: <20060623193857.GC40269@garage.freebsd.pl> References: <20060619131101.GD1130@garage.freebsd.pl> <449C06C6.9070801@centtech.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XWOWbaMNXpFDWE00" Content-Disposition: inline In-Reply-To: <449C06C6.9070801@centtech.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 19:42:24 -0000 --XWOWbaMNXpFDWE00 Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jun 23, 2006 at 10:20:38AM -0500, Eric Anderson wrote: +> Pawel Jakub Dawidek wrote: +> >Hello. +> >For the last few months I have been working on gjournal project. +> >To stop confusion right here, I want to note, that this project is not +> >related to gjournal project on which Ivan Voras was working on the +> >last SoC (2005). +> >The lack of journaled file system in FreeBSD was a tendon of achilles +> >for many years. We do have many file systems, but none with journaling: +> >- ext2fs (journaling is in ext3fs), +> >- XFS (read-only), +> >- ReiserFS (read-only), +> >- HFS+ (read-write, but without journaling), +> >- NTFS (read-only). +> >GJournal was designed to journal GEOM providers, so it actually works +> >below file system layer, but it has hooks which allow to work with +> >file systems. In other words, gjournal is not file system-depended, +> >it can work probably with any file system with minimum knowledge +> >about it. I implemented only UFS support. +> >The patches are here: +> > http://people.freebsd.org/~pjd/patches/gjournal.patch (for HEAD) +> > http://people.freebsd.org/~pjd/patches/gjournal6.patch (for RELENG_6) +> >To patch your sources you need to: +> > # cd /usr/src +> > # mkdir sbin/geom/class/journal sys/geom/journal sys/modules/geom/geom= _journal +> > # patch < /path/to/gjournal.patch +> >Add 'options UFS_GJOURNAL' to your kernel configuration file and +> >recompile kernel and world. +> >How it works (in short). You may define one or two providers which +> >gjournal will use. If one provider is given, it will be used for both - +> >data and journal. If two providers are given, one will be used for data +> >and one for journal. +> >Every few seconds (you may define how many) journal is terminated and +> >marked as consistent and gjournal starts to copy data from it to the +> >data provider. In the same time new data are stored in new journal. +>=20 +> I'm not sure this is happening the way you describe exactly. On my lapt= op, while rsyncing my /home partition to a newly created external disk (400= G), I see 20MB/s writing=20 +> to the journaled UFS2 device (/dev/label/backup.journal) passing through= to the journal device (/dev/label/journal), then it switches to no writes = to the journaled UFS2=20 +> device (/dev/label/backup.journal) (my rsync pauses) while the journaled= device (/dev/label/backup) writes at 20MB/s for about 3-10 seconds. When it is time for journal switch, we cannot switch the journals if we still copy data from the inactive journal, so we wait then. You can tune it a bit using those two sysctls: kern.geom.journal.parallel_flushes - Number of flush I/O requests send in parallel kern.geom.journal.parallel_copies - Number of copy I/O requests send in parallel By default those are equal, you may increase the second one or decrease the first one to tell gjournal to focus more on copying the data from the inactive journal, so when journal switch time arrives, it doesn't have to wait. Before you do it, please consult kern.geom.journal.stats.wait_for_copy sysctl variable, which will tell you how many times journal switch was delayed because of inactive journal not beeing fully copied. More waiting is because a lot of data is only in memory and when I call file system synchronization all the data go to gjournal provider. All modes in which UFS can operate are not optimal for gjournal - I mean here sync, async and SU. The most optimal mode for gjournal will be something like: send write request immediatelly and don't wait for an answer. GJournal will take care of reordering write request to get optimal throughput and this will allow for more balanced load. For example SU send write requests in picks, which is bad for gjournal. +> >Let's call the moment in which journal is terminated as "journal switch= ". +> >Journal switch looks as follows: +> >1. Start journal switch if we have timeout or if we run out of cache. +> > Don't perform journal switch if there were no write requests. +> >2. If we have file system, synchronize it. +> >3. Mark file system as clean. +> >4. Block all write requests to the file system. +> >5. Terminate the journal. +> >6. Eventually wait if copying of the previous journal is not yet +> > finished. +>=20 +> Seems like this is the point we are busy in. +>=20 +> >7. Send BIO_FLUSH request (if the given provider supports it). +> >8. Mark new journal position on the journal provider. +> >9. Unblock write requests. +> >10. Start copying data from the terminated journal to the data provider. +>=20 +> And it seems that 10 is happening earlier on.. The point number 10 is actually after the journal switch. It is when the active journal was turned into an inactive journal and the copy starts. Don't take this order to strict, I more wanted to show what steps are performed. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --XWOWbaMNXpFDWE00 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) iD8DBQFEnENRForvXbEpPzQRAl+vAKC1c+ophEYLProOjQ1373BDyoaFKwCdH15u Gb6918+pzKh34atzxPrxhnQ= =opPZ -----END PGP SIGNATURE----- --XWOWbaMNXpFDWE00-- From owner-freebsd-fs@FreeBSD.ORG Fri Jun 23 19:49:26 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F1B7316A47B; Fri, 23 Jun 2006 19:49:25 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail05.syd.optusnet.com.au (mail05.syd.optusnet.com.au [211.29.132.186]) by mx1.FreeBSD.org (Postfix) with ESMTP id 43AC943D4C; Fri, 23 Jun 2006 19:49:24 +0000 (GMT) (envelope-from peterjeremy@optushome.com.au) Received: from turion.vk2pj.dyndns.org (c220-239-19-236.belrs4.nsw.optusnet.com.au [220.239.19.236]) by mail05.syd.optusnet.com.au (8.12.11/8.12.11) with ESMTP id k5NJnJU0018458 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Sat, 24 Jun 2006 05:49:19 +1000 Received: from turion.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by turion.vk2pj.dyndns.org (8.13.6/8.13.6) with ESMTP id k5NJnJqi003289; Sat, 24 Jun 2006 05:49:19 +1000 (EST) (envelope-from peter@turion.vk2pj.dyndns.org) Received: (from peter@localhost) by turion.vk2pj.dyndns.org (8.13.6/8.13.6/Submit) id k5NJnIIQ003288; Sat, 24 Jun 2006 05:49:18 +1000 (EST) (envelope-from peter) Date: Sat, 24 Jun 2006 05:49:18 +1000 From: Peter Jeremy To: "R. B. Riddick" Message-ID: <20060623194917.GB747@turion.vk2pj.dyndns.org> References: <20060623082209.GD13474@nevermind.kiev.ua> <20060623083838.86539.qmail@web30308.mail.mud.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="HlL+5n6rz5pIUxbD" Content-Disposition: inline In-Reply-To: <20060623083838.86539.qmail@web30308.mail.mud.yahoo.com> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.11 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Journaling UFS with gjournal. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jun 2006 19:49:26 -0000 --HlL+5n6rz5pIUxbD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, 2006-Jun-23 01:38:38 -0700, R. B. Riddick wrote: >--- Alexandr Kovalenko wrote: >> Is it safe to do so on existing filesystem (if I'm using 2nd partition f= or >> journal)? =2E.. >If your existing file system needs its last sector, then it wont work. If = it >does not need it, then it might work (although fsck does not check for a >raw-device shrinkage - I think)... In my experience, the last partition in a disk slice normally has an odd number of sectors and UFS definitely can't handle anything smaller than a fragment (which defaults to 2K) - and I suspect that UFS can't handle a trailing fragment. In this case, the last sector is definitely unused. I may be wrong but I don't think it's possible for the last sector of a partition to be FS metadata because the metadata is always at the beginning of a CG and newfs won't create a CG unless there's some space for data in the CG. If there are an integral number of fragments (or maybe blocks), then marking the last fragment as 'bad' would seem an adequate solution - the FS will ignore that block but anything below the filesystem won't see the "bad block" marker. --=20 Peter Jeremy --HlL+5n6rz5pIUxbD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEnEW8/opHv/APuIcRAjoPAJ0f90leQiv+V3Xu4VpYvnBMZT+XwQCfZIQc A3v0WxoDBaIt5pM5omNJN58= =aq1w -----END PGP SIGNATURE----- --HlL+5n6rz5pIUxbD-- From owner-freebsd-fs@FreeBSD.ORG Sat Jun 24 22:26:44 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7CEA416A516 for ; Sat, 24 Jun 2006 22:26:44 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 07CD543D45 for ; Sat, 24 Jun 2006 22:26:43 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 5D9CB46BD2; Sat, 24 Jun 2006 18:26:43 -0400 (EDT) Date: Sat, 24 Jun 2006 23:26:43 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Ensel Sharon In-Reply-To: Message-ID: <20060624232457.D8526@fledge.watson.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: 6.1 quota bugs cause adaptec 2820sa kernel to crash ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Jun 2006 22:26:44 -0000 On Fri, 23 Jun 2006, Ensel Sharon wrote: > ----- > > After loading, the system frequently (multi-daily) crashed with the error: > > Warning! Controller is no longer running! code=0xbcef0100 > > (after a page or so of aac0 timeout messages) > > So I disabled quotas on the system, and it has been completely stable ever > since. > > ----- The above sounds a lot like a problem with {Adaptect driver, controller, disks}, rather than a quota problem. So it might be that there's an I/O load change with quotas running that triggers the problem. Alternatively, there's a memory corruption bug (or the like) in quotas that corrupts data structures for the adaptec driver, but hopefully not. I believe Scott Long follows this list, but if you don't hear back in a bit, you might forward that description to him and see if he has thoughts. In the past, he's maintained the Adaptec drivers, but I'm not sure what his level of involvement with them is at this point. Robert N M Watson Computer Laboratory University of Cambridge