From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 14:16:40 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 166EC16A406 for ; Mon, 10 Apr 2006 14:16:40 +0000 (UTC) (envelope-from Nicolas.Kowalski@imag.fr) Received: from imag.imag.fr (imag.imag.fr [129.88.30.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 79C8443D7B for ; Mon, 10 Apr 2006 14:16:35 +0000 (GMT) (envelope-from Nicolas.Kowalski@imag.fr) Received: from mail-veri.imag.fr (mail-veri.imag.fr [129.88.43.52]) by imag.imag.fr (8.13.6/8.13.6) with ESMTP id k3AEGRQY004034 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 10 Apr 2006 16:16:27 +0200 (CEST) Received: from corbeau.imag.fr ([129.88.43.162]) by mail-veri.imag.fr with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1FSxBr-0003wS-HO for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 16:16:27 +0200 Received: from kowalski by corbeau.imag.fr with local (Exim 4.50) id 1FSxBr-0001jc-0g for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 16:16:27 +0200 To: freebsd-fs@FreeBSD.org References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> From: Nicolas KOWALSKI Date: Mon, 10 Apr 2006 16:16:27 +0200 In-Reply-To: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: Nicolas Kowalski X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (imag.imag.fr [129.88.30.1]); Mon, 10 Apr 2006 16:16:27 +0200 (CEST) X-IMAG-MailScanner-Information: Please contact IMAG DMI for more information X-IMAG-MailScanner: Found to be clean X-MailScanner-From: kowalski@imag.fr Cc: Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 14:16:40 -0000 Hello, Kostik Belousov writes: > I already mailed about my development of the patch that > allows for UFS with quotas to operate without Giant. Sorry if the > repeat would be annoying. Does this patch improve the performance of a file server, using multiple disks, controlled by quotas, exported by NFS/Samba ? If so, I would be really interested: our file server (4.11, but perhaps 6.x soon), has some major slowdowns when one or multiple user/s exceed her/his quota ; this impact every user, even those working on another disk. Thanks, -- Nicolas From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 14:49:15 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3673516A404 for ; Mon, 10 Apr 2006 14:49:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from fw.zoral.com.ua (ll-227.216.82.212.sovam.net.ua [212.82.216.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id B605043D58 for ; Mon, 10 Apr 2006 14:49:11 +0000 (GMT) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by fw.zoral.com.ua (8.13.4/8.13.3) with ESMTP id k3AEn5ue092052 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 10 Apr 2006 17:49:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.13.6/8.13.6) with ESMTP id k3AEn5K6009060; Mon, 10 Apr 2006 17:49:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.13.6/8.13.6/Submit) id k3AEn5WW009059; Mon, 10 Apr 2006 17:49:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 10 Apr 2006 17:49:04 +0300 From: Kostik Belousov To: Nicolas KOWALSKI Message-ID: <20060410144904.GC1408@deviant.kiev.zoral.com.ua> References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="5G06lTa6Jq83wMTw" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.1i X-Virus-Scanned: ClamAV version 0.88, clamav-milter version 0.87 on fw.zoral.com.ua X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 14:49:15 -0000 --5G06lTa6Jq83wMTw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 10, 2006 at 04:16:27PM +0200, Nicolas KOWALSKI wrote: > Hello, >=20 > Kostik Belousov writes: >=20 > > I already mailed about my development of the patch that > > allows for UFS with quotas to operate without Giant. Sorry if the > > repeat would be annoying. >=20 > Does this patch improve the performance of a file server, using > multiple disks, controlled by quotas, exported by NFS/Samba ? >=20 > If so, I would be really interested: our file server (4.11, but > perhaps 6.x soon), has some major slowdowns when one or multiple > user/s exceed her/his quota ; this impact every user, even those > working on another disk. I don't think that patch shall have effect on this situation (quota exceede= d). Probably, you have some other issues, esp. for 4.x, where only one process can progress in kernel mode anyway. Just guessing: do you have slow (serial) console ? Yes, I expect patch to improve overall system performance for 6.x/7 when quotas are compiled in the kernel compared with the same kernel config without patch. This was the reason for developing the change. I do not have a numerical measurement of improvement, though. I will be very glad for testing/stress testing/performance measurement for the patch. BUT, PLEASE BEWARE. Don't apply the patch for non-test machines. Kris Kennaway said the system deadlocks with patch applied. I still cannot reproduce it (and debug). In my defense I could say that I did found two issues since Kris' report. Both are fixed. Best regards, Kostik Belousov. --5G06lTa6Jq83wMTw Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (FreeBSD) iD8DBQFEOnBgC3+MBN1Mb4gRAsjkAJ9B+/z7Vbh35dj+Mh8HNcD6Y56srACg1HeU yAxUdsu7R9JRUbyej4u3XWE= =XEMk -----END PGP SIGNATURE----- --5G06lTa6Jq83wMTw-- From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 15:37:46 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DD02016A409 for ; Mon, 10 Apr 2006 15:37:46 +0000 (UTC) (envelope-from Nicolas.Kowalski@imag.fr) Received: from imag.imag.fr (imag.imag.fr [129.88.30.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 51CDC43D5F for ; Mon, 10 Apr 2006 15:36:57 +0000 (GMT) (envelope-from Nicolas.Kowalski@imag.fr) Received: from mail-veri.imag.fr (mail-veri.imag.fr [129.88.43.52]) by imag.imag.fr (8.13.6/8.13.6) with ESMTP id k3AFasoF020464 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 10 Apr 2006 17:36:54 +0200 (CEST) Received: from corbeau.imag.fr ([129.88.43.162]) by mail-veri.imag.fr with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1FSyRi-0006Xl-At for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 17:36:54 +0200 Received: from kowalski by corbeau.imag.fr with local (Exim 4.50) id 1FSyRi-00012z-3A for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 17:36:54 +0200 To: freebsd-fs@FreeBSD.org References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> From: Nicolas KOWALSKI Date: Mon, 10 Apr 2006 17:36:54 +0200 In-Reply-To: <20060410144904.GC1408@deviant.kiev.zoral.com.ua> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: Nicolas Kowalski X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (imag.imag.fr [129.88.30.1]); Mon, 10 Apr 2006 17:36:54 +0200 (CEST) X-IMAG-MailScanner-Information: Please contact IMAG DMI for more information X-IMAG-MailScanner: Found to be clean X-MailScanner-From: kowalski@imag.fr Cc: Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 15:37:47 -0000 Kostik Belousov writes: > On Mon, Apr 10, 2006 at 04:16:27PM +0200, Nicolas KOWALSKI wrote: >> Hello, >> >> Kostik Belousov writes: >> >> > I already mailed about my development of the patch that >> > allows for UFS with quotas to operate without Giant. Sorry if the >> > repeat would be annoying. >> >> Does this patch improve the performance of a file server, using >> multiple disks, controlled by quotas, exported by NFS/Samba ? >> >> If so, I would be really interested: our file server (4.11, but >> perhaps 6.x soon), has some major slowdowns when one or multiple >> user/s exceed her/his quota ; this impact every user, even those >> working on another disk. > I don't think that patch shall have effect on this situation (quota > exceeded). :-( , but Thanks for your response. > Probably, you have some other issues, esp. for 4.x, where > only one process can progress in kernel mode anyway. > Just guessing: do you have slow (serial) console ? No serial console here, the server is connected to a standard vga/ps2 KVM switch. > Yes, I expect patch to improve overall system performance for 6.x/7 when > quotas are compiled in the kernel compared with the same kernel > config without patch. This was the reason for developing the change. > I do not have a numerical measurement of improvement, though. > I will be very glad for testing/stress testing/performance measurement > for the patch. > > BUT, PLEASE BEWARE. Don't apply the patch for non-test machines. Ok, I will install a test machine and try to stress it. Best regards, -- Nicolas From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 15:41:16 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D81C16A402 for ; Mon, 10 Apr 2006 15:41:16 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 30BC443D45 for ; Mon, 10 Apr 2006 15:41:15 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k3AFfEcA022998; Mon, 10 Apr 2006 10:41:14 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <443A7C8E.4020203@centtech.com> Date: Mon, 10 Apr 2006 10:41:02 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5 (X11/20060402) MIME-Version: 1.0 To: Nicolas KOWALSKI References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1389/Mon Apr 10 07:58:55 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 15:41:16 -0000 Nicolas KOWALSKI wrote: > Kostik Belousov writes: > >> On Mon, Apr 10, 2006 at 04:16:27PM +0200, Nicolas KOWALSKI wrote: >>> Hello, >>> >>> Kostik Belousov writes: >>> >>>> I already mailed about my development of the patch that >>>> allows for UFS with quotas to operate without Giant. Sorry if the >>>> repeat would be annoying. >>> Does this patch improve the performance of a file server, using >>> multiple disks, controlled by quotas, exported by NFS/Samba ? >>> >>> If so, I would be really interested: our file server (4.11, but >>> perhaps 6.x soon), has some major slowdowns when one or multiple >>> user/s exceed her/his quota ; this impact every user, even those >>> working on another disk. >> I don't think that patch shall have effect on this situation (quota >> exceeded). > If you watch a tcpdump while this is happening, I'll bet you see the client (over NFS) repeatedly trying to allocate blocks, and the server returning ENOSPACE. The load and cpu utilization on the server will skyrocket while it gets hammered with requests for more space. Not sure what the solution is to this. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 16:10:22 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 31DEA16A412 for ; Mon, 10 Apr 2006 16:10:22 +0000 (UTC) (envelope-from Nicolas.Kowalski@imag.fr) Received: from imag.imag.fr (imag.imag.fr [129.88.30.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id DDF9D43D73 for ; Mon, 10 Apr 2006 16:10:11 +0000 (GMT) (envelope-from Nicolas.Kowalski@imag.fr) Received: from mail-veri.imag.fr (mail-veri.imag.fr [129.88.43.52]) by imag.imag.fr (8.13.6/8.13.6) with ESMTP id k3AGA5je026577 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 10 Apr 2006 18:10:05 +0200 (CEST) Received: from corbeau.imag.fr ([129.88.43.162]) by mail-veri.imag.fr with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1FSyxp-0007SM-J0 for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 18:10:05 +0200 Received: from kowalski by corbeau.imag.fr with local (Exim 4.50) id 1FSyxp-00016g-ES for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 18:10:05 +0200 To: freebsd-fs@FreeBSD.org References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> From: Nicolas KOWALSKI Date: Mon, 10 Apr 2006 18:10:05 +0200 In-Reply-To: <443A7C8E.4020203@centtech.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: Nicolas Kowalski X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (imag.imag.fr [129.88.30.1]); Mon, 10 Apr 2006 18:10:05 +0200 (CEST) X-IMAG-MailScanner-Information: Please contact IMAG DMI for more information X-IMAG-MailScanner: Found to be clean X-MailScanner-From: kowalski@imag.fr Cc: Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 16:10:22 -0000 Eric Anderson writes: > Nicolas KOWALSKI wrote: >> Kostik Belousov writes: >> >>> On Mon, Apr 10, 2006 at 04:16:27PM +0200, Nicolas KOWALSKI wrote: >>>> Hello, >>>> >>>> Kostik Belousov writes: >>>> >>>>> I already mailed about my development of the patch that >>>>> allows for UFS with quotas to operate without Giant. Sorry if the >>>>> repeat would be annoying. >>>> Does this patch improve the performance of a file server, using >>>> multiple disks, controlled by quotas, exported by NFS/Samba ? >>>> >>>> If so, I would be really interested: our file server (4.11, but >>>> perhaps 6.x soon), has some major slowdowns when one or multiple >>>> user/s exceed her/his quota ; this impact every user, even those >>>> working on another disk. >>> I don't think that patch shall have effect on this situation (quota >>> exceeded). > > If you watch a tcpdump while this is happening, I'll bet you see the > client (over NFS) repeatedly trying to allocate blocks, and the server > returning ENOSPACE. The load and cpu utilization on the server will > skyrocket while it gets hammered with requests for more space. Not > sure what the solution is to this. Yes, this is exactly what is happening. To add some precision, some students here use calculation applications that allocate a lot of disk space, ususally more than their allowed home quotas; when by error they launch these apps in their home directories, instead of their workstation dedicated space, it makes the server go to its knees on the NFS client side. -- Nicolas From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 16:31:12 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4843716A402 for ; Mon, 10 Apr 2006 16:31:12 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id C609743D46 for ; Mon, 10 Apr 2006 16:31:11 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k3AGVAY0025244; Mon, 10 Apr 2006 11:31:11 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <443A8842.6060802@centtech.com> Date: Mon, 10 Apr 2006 11:30:58 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5 (X11/20060402) MIME-Version: 1.0 To: Nicolas KOWALSKI References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1389/Mon Apr 10 07:58:55 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 16:31:12 -0000 Nicolas KOWALSKI wrote: > Eric Anderson writes: > >> Nicolas KOWALSKI wrote: >>> Kostik Belousov writes: >>> >>>> On Mon, Apr 10, 2006 at 04:16:27PM +0200, Nicolas KOWALSKI wrote: >>>>> Hello, >>>>> >>>>> Kostik Belousov writes: >>>>> >>>>>> I already mailed about my development of the patch that >>>>>> allows for UFS with quotas to operate without Giant. Sorry if the >>>>>> repeat would be annoying. >>>>> Does this patch improve the performance of a file server, using >>>>> multiple disks, controlled by quotas, exported by NFS/Samba ? >>>>> >>>>> If so, I would be really interested: our file server (4.11, but >>>>> perhaps 6.x soon), has some major slowdowns when one or multiple >>>>> user/s exceed her/his quota ; this impact every user, even those >>>>> working on another disk. >>>> I don't think that patch shall have effect on this situation (quota >>>> exceeded). >> If you watch a tcpdump while this is happening, I'll bet you see the >> client (over NFS) repeatedly trying to allocate blocks, and the server >> returning ENOSPACE. The load and cpu utilization on the server will >> skyrocket while it gets hammered with requests for more space. Not >> sure what the solution is to this. > > Yes, this is exactly what is happening. > > To add some precision, some students here use calculation applications > that allocate a lot of disk space, ususally more than their allowed > home quotas; when by error they launch these apps in their home > directories, instead of their workstation dedicated space, it makes > the server go to its knees on the NFS client side. When you say 'to it's knees' - what do you mean exactly? How many clients do you have, how much memory is on the server, and how many nfsd threads are you using? What kind of load average do you see during this (on the server)? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 16:45:46 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9554C16A406 for ; Mon, 10 Apr 2006 16:45:46 +0000 (UTC) (envelope-from dxw@mrecic.gov.ar) Received: from belusi.mrecic.gov.ar (belusi.mrecic.gov.ar [200.16.97.20]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0F25543D80 for ; Mon, 10 Apr 2006 16:45:43 +0000 (GMT) (envelope-from dxw@mrecic.gov.ar) Received: from localhost (localhost.mrec.ar [127.0.0.1]) by belusi.mrecic.gov.ar (Postfix) with ESMTP id E5B2ECC335 for ; Mon, 10 Apr 2006 13:45:40 -0300 (ART) Received: from belusi.mrecic.gov.ar ([127.0.0.1]) by localhost (belusi.mrecic.gov.ar [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 58786-06 for ; Mon, 10 Apr 2006 13:45:34 -0300 (ART) Received: from megadeth.mrec.ar (megadeth.mrec.ar [140.191.48.33]) by belusi.mrecic.gov.ar (Postfix) with ESMTP id 9CF49CC316 for ; Mon, 10 Apr 2006 13:45:34 -0300 (ART) Received: from localhost (localhost.mrec.ar [127.0.0.1]) by megadeth.mrec.ar (Postfix) with ESMTP id 00FB3702042 for ; Mon, 10 Apr 2006 13:45:33 -0300 (ART) Received: from megadeth.mrec.ar ([127.0.0.1]) by localhost (megadeth.mrec.ar [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 58233-09 for ; Mon, 10 Apr 2006 13:45:29 -0300 (ART) Received: from diegows (ws2285.mrec.ar [140.191.4.78]) by megadeth.mrec.ar (Postfix) with ESMTP id 78BEB70203F for ; Mon, 10 Apr 2006 13:45:29 -0300 (ART) From: Diego Woitasen To: freebsd-fs@freebsd.org Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-jSgo2ukfYgDkcDzaZxCG" Organization: Ministerio de Relaciones Exteriores, Comercio Internacional y Culto Date: Mon, 10 Apr 2006 13:43:38 -0300 Message-Id: <1144687418.11014.9.camel@diegows> Mime-Version: 1.0 X-Mailer: Evolution 2.4.1 X-Virus-Scanned: amavisd-new at mrecic.gov.ar X-Virus-Scanned: amavisd-new at mrecic.gov.ar Subject: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 16:45:46 -0000 --=-jSgo2ukfYgDkcDzaZxCG Content-Type: text/plain Content-Transfer-Encoding: quoted-printable I want to know how a file is deleted in a ufs2 filesystem, specifically what happen with the information in the inode. The information is deleted to or the inode is marked as free but the information (uid, gid, blocks, times, etc) remains there? I read the chapter 8 of 'Design and implementation of FreeBSD" and "a Fast file system for Unix", but i can't see the answer. Reading the code is an interesting choice, but is the last resource :) thanks! --=20 Diego Woitasen Ministerio de Relaciones Exteriores, Comercio Internacional y Culto --=-jSgo2ukfYgDkcDzaZxCG Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQBEOos6rtJ8Sa50a9URAr8cAJ9i4Pt0KHUvexzkd6BT2XNx6qplBACgiJ6+ iFpK4yotPvDtH8DTNyKMC7E= =YMF+ -----END PGP SIGNATURE----- --=-jSgo2ukfYgDkcDzaZxCG-- From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 16:55:09 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6ACD416A402 for ; Mon, 10 Apr 2006 16:55:09 +0000 (UTC) (envelope-from Nicolas.Kowalski@imag.fr) Received: from imag.imag.fr (imag.imag.fr [129.88.30.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8C28443D45 for ; Mon, 10 Apr 2006 16:55:07 +0000 (GMT) (envelope-from Nicolas.Kowalski@imag.fr) Received: from mail-veri.imag.fr (mail-veri.imag.fr [129.88.43.52]) by imag.imag.fr (8.13.6/8.13.6) with ESMTP id k3AGt3wX004948 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 10 Apr 2006 18:55:04 +0200 (CEST) Received: from corbeau.imag.fr ([129.88.43.162]) by mail-veri.imag.fr with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1FSzfL-0000YJ-TW for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 18:55:03 +0200 Received: from kowalski by corbeau.imag.fr with local (Exim 4.50) id 1FSzfL-0001Dz-Om for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 18:55:03 +0200 To: freebsd-fs@FreeBSD.org References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> <443A8842.6060802@centtech.com> From: Nicolas KOWALSKI Date: Mon, 10 Apr 2006 18:55:03 +0200 In-Reply-To: <443A8842.6060802@centtech.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: Nicolas Kowalski X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (imag.imag.fr [129.88.30.1]); Mon, 10 Apr 2006 18:55:04 +0200 (CEST) X-IMAG-MailScanner-Information: Please contact IMAG DMI for more information X-IMAG-MailScanner: Found to be clean X-MailScanner-From: kowalski@imag.fr Cc: Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 16:55:09 -0000 Eric Anderson writes: > Nicolas KOWALSKI wrote: >> >> Yes, this is exactly what is happening. To add some precision, some >> students here use calculation applications >> that allocate a lot of disk space, ususally more than their allowed >> home quotas; when by error they launch these apps in their home >> directories, instead of their workstation dedicated space, it makes >> the server go to its knees on the NFS client side. > > When you say 'to it's knees' - what do you mean exactly? How many > clients do you have, how much memory is on the server, and how many > nfsd threads are you using? What kind of load average do you see > during this (on the server)? Sorry for the imprecision. The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320 76GB disks and controller. It is accessed by NFS from ~100 Unix (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The network connection is GB ethernet. During slowdowns, it's only from a NFS client view that the server does not respond. For example, a simple 'ls' in my home directory is almost immediate, but when it slows down, it can take up to 2 minutes. On the server, the load average goes to 0.5, compared to a default maximum of 0.15-0.20. The nfsd processus shows them in the state "biowr" in top, but nothing is really written, because the quotas system block any further writes to the user exceeding her/his quotas. -- Nicolas From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 17:38:16 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6B75116A400 for ; Mon, 10 Apr 2006 17:38:16 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id B015543D45 for ; Mon, 10 Apr 2006 17:38:15 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k3AHcDUP028314; Mon, 10 Apr 2006 12:38:13 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <443A97F9.8090601@centtech.com> Date: Mon, 10 Apr 2006 12:38:01 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5 (X11/20060402) MIME-Version: 1.0 To: Nicolas KOWALSKI References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> <443A8842.6060802@centtech.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1389/Mon Apr 10 07:58:55 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 17:38:16 -0000 Nicolas KOWALSKI wrote: > Eric Anderson writes: > >> Nicolas KOWALSKI wrote: >>> Yes, this is exactly what is happening. To add some precision, some >>> students here use calculation applications >>> that allocate a lot of disk space, ususally more than their allowed >>> home quotas; when by error they launch these apps in their home >>> directories, instead of their workstation dedicated space, it makes >>> the server go to its knees on the NFS client side. >> When you say 'to it's knees' - what do you mean exactly? How many >> clients do you have, how much memory is on the server, and how many >> nfsd threads are you using? What kind of load average do you see >> during this (on the server)? > > Sorry for the imprecision. > > The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320 > 76GB disks and controller. It is accessed by NFS from ~100 Unix > (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The > network connection is GB ethernet. > > During slowdowns, it's only from a NFS client view that the server > does not respond. For example, a simple 'ls' in my home directory is > almost immediate, but when it slows down, it can take up to 2 minutes. > > On the server, the load average goes to 0.5, compared to a default > maximum of 0.15-0.20. The nfsd processus shows them in the state > "biowr" in top, but nothing is really written, because the quotas > system block any further writes to the user exceeding her/his quotas. > In this case (which is what I suspected), try bumping up your nfsd threads to 128. I set mine very high (I have around 1000 clients), and I can say there aren't really ill-effects besides a bit of memory usage (which you have plenty of). I suspect increasing the threads will neutralize this problem for you. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 17:56:04 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9806B16A401 for ; Mon, 10 Apr 2006 17:56:04 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id EC8EC43D48 for ; Mon, 10 Apr 2006 17:56:03 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k3AHu2Hn029139; Mon, 10 Apr 2006 12:56:03 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <443A9C26.4060103@centtech.com> Date: Mon, 10 Apr 2006 12:55:50 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5 (X11/20060402) MIME-Version: 1.0 To: Diego Woitasen References: <1144687418.11014.9.camel@diegows> In-Reply-To: <1144687418.11014.9.camel@diegows> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1389/Mon Apr 10 07:58:55 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 17:56:04 -0000 Diego Woitasen wrote: > I want to know how a file is deleted in a ufs2 filesystem, specifically > what happen with the information in the inode. The information is > deleted to or the inode is marked as free but the information (uid, gid, > blocks, times, etc) remains there? > > I read the chapter 8 of 'Design and implementation of FreeBSD" and "a > Fast file system for Unix", but i can't see the answer. > > Reading the code is an interesting choice, but is the last resource :) I'm 100% certain here, but looking at the code, I don't think much happens besides freeing the inode and clearing it for re-use. The on-disk data remains. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 18:11:03 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0103416A400 for ; Mon, 10 Apr 2006 18:11:03 +0000 (UTC) (envelope-from eksffa@freebsdbrasil.com.br) Received: from capeta.freebsdbrasil.com.br (vrrp.freebsdbrasil.com.br [200.210.70.30]) by mx1.FreeBSD.org (Postfix) with SMTP id BFF2F43D48 for ; Mon, 10 Apr 2006 18:10:58 +0000 (GMT) (envelope-from eksffa@freebsdbrasil.com.br) Received: (qmail 10297 invoked by uid 0); 10 Apr 2006 15:10:54 -0300 Received: from eksffa@freebsdbrasil.com.br by capeta.freebsdbrasil.com.br by uid 82 with qmail-scanner-1.22 (spamassassin: 2.64. Clear:RC:1(200.210.42.5):. Processed in 101.894297 secs); 10 Apr 2006 18:10:54 -0000 Received: from unknown (HELO ?10.69.69.69?) (200.210.42.5) by capeta.freebsdbrasil.com.br with SMTP; 10 Apr 2006 15:09:12 -0300 Message-ID: <443A9F3D.6090008@freebsdbrasil.com.br> Date: Mon, 10 Apr 2006 15:09:01 -0300 From: Patrick Tracanelli Organization: FreeBSD Brasil LTDA User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20051013 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <1144687418.11014.9.camel@diegows> <443A9C26.4060103@centtech.com> In-Reply-To: <443A9C26.4060103@centtech.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 18:11:03 -0000 Eric Anderson wrote: > Diego Woitasen wrote: > >> I want to know how a file is deleted in a ufs2 filesystem, specifically >> what happen with the information in the inode. The information is >> deleted to or the inode is marked as free but the information (uid, gid, >> blocks, times, etc) remains there? >> >> I read the chapter 8 of 'Design and implementation of FreeBSD" and "a >> Fast file system for Unix", but i can't see the answer. >> >> Reading the code is an interesting choice, but is the last resource :) > > > > I'm 100% certain here, but looking at the code, I don't think much > happens besides freeing the inode and clearing it for re-use. The > on-disk data remains. > > > Eric When the story ends, you have char *f unlink(f); in the code. So reading "man 2 unlink" might be a good start point. If reading McKusick's book and the syscall man page dont make it clear, you will have no better choice other than reading the code yourself :D -- Patrick Tracanelli FreeBSD Brasil LTDA. (31) 3281-9633 / 3281-3547 316601@sip.freebsdbrasil.com.br http://www.freebsdbrasil.com.br "Long live Hanin Elias, Kim Deal!" From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 18:19:45 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 62B6B16A4F3 for ; Mon, 10 Apr 2006 18:19:45 +0000 (UTC) (envelope-from Nicolas.Kowalski@imag.fr) Received: from imag.imag.fr (imag.imag.fr [129.88.30.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 02D4E43D49 for ; Mon, 10 Apr 2006 18:19:36 +0000 (GMT) (envelope-from Nicolas.Kowalski@imag.fr) Received: from mail-veri.imag.fr (mail-veri.imag.fr [129.88.43.52]) by imag.imag.fr (8.13.6/8.13.6) with ESMTP id k3AIJXAS018507 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 10 Apr 2006 20:19:33 +0200 (CEST) Received: from corbeau.imag.fr ([129.88.43.162]) by mail-veri.imag.fr with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1FT0z7-0001wn-ON for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 20:19:33 +0200 Received: from kowalski by corbeau.imag.fr with local (Exim 4.50) id 1FT0z7-0001P6-LE for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 20:19:33 +0200 To: freebsd-fs@FreeBSD.org References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> <443A8842.6060802@centtech.com> <443A97F9.8090601@centtech.com> From: Nicolas KOWALSKI Date: Mon, 10 Apr 2006 20:19:33 +0200 In-Reply-To: <443A97F9.8090601@centtech.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: Nicolas Kowalski X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (imag.imag.fr [129.88.30.1]); Mon, 10 Apr 2006 20:19:33 +0200 (CEST) X-IMAG-MailScanner-Information: Please contact IMAG DMI for more information X-IMAG-MailScanner: Found to be clean X-MailScanner-From: kowalski@imag.fr Cc: Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 18:19:45 -0000 Eric Anderson writes: > Nicolas KOWALSKI wrote: >> Eric Anderson writes: >> >>> Nicolas KOWALSKI wrote: >>>> Yes, this is exactly what is happening. To add some precision, some >>>> students here use calculation applications >>>> that allocate a lot of disk space, ususally more than their allowed >>>> home quotas; when by error they launch these apps in their home >>>> directories, instead of their workstation dedicated space, it makes >>>> the server go to its knees on the NFS client side. >>> When you say 'to it's knees' - what do you mean exactly? How many >>> clients do you have, how much memory is on the server, and how many >>> nfsd threads are you using? What kind of load average do you see >>> during this (on the server)? >> Sorry for the imprecision. >> The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320 >> 76GB disks and controller. It is accessed by NFS from ~100 Unix >> (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The >> network connection is GB ethernet. >> During slowdowns, it's only from a NFS client view that the server >> does not respond. For example, a simple 'ls' in my home directory is >> almost immediate, but when it slows down, it can take up to 2 minutes. >> On the server, the load average goes to 0.5, compared to a default >> maximum of 0.15-0.20. The nfsd processus shows them in the state >> "biowr" in top, but nothing is really written, because the quotas >> system block any further writes to the user exceeding her/his quotas. >> > > In this case (which is what I suspected), try bumping up your nfsd > threads to 128. I set mine very high (I have around 1000 clients), > and I can say there aren't really ill-effects besides a bit of memory > usage (which you have plenty of). I suspect increasing the threads > will neutralize this problem for you. Thanks for your suggestion. However, I am apparently not able to change the default. I stopped the nfsd master process (kill -USR1 as written in the manpage), then started it: pave# nfsd -t -u -n 128 nfsd: nfsd count 128; reset to 4 What am I forgetting here ? Thanks, -- Nicolas From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 18:40:11 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7BF6516A402 for ; Mon, 10 Apr 2006 18:40:11 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30306.mail.mud.yahoo.com (web30306.mail.mud.yahoo.com [68.142.200.99]) by mx1.FreeBSD.org (Postfix) with SMTP id 41FB843D5F for ; Mon, 10 Apr 2006 18:40:07 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 68440 invoked by uid 60001); 10 Apr 2006 18:40:07 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=n6AOsPJB5ao9kiRuOdwSCdscLsqvdq80a6QACX9n02GttWOXJ4ZoZzpZgTXqTL3xmwy+l6fL3/SYW/eX8lLpnwB8C7zHHXacVjvL5JnFOrYiC6P7IdrK7YQZdjeL0uhXQ8ZKJx3Uoe5xThxAfKpeJI55Lm1niwHcfBK7xL2F6mo= ; Message-ID: <20060410184007.68438.qmail@web30306.mail.mud.yahoo.com> Received: from [213.54.79.199] by web30306.mail.mud.yahoo.com via HTTP; Mon, 10 Apr 2006 11:40:07 PDT Date: Mon, 10 Apr 2006 11:40:07 -0700 (PDT) From: "R. B. Riddick" To: Nicolas KOWALSKI , freebsd-fs@FreeBSD.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 18:40:11 -0000 --- Nicolas KOWALSKI wrote: > pave# nfsd -t -u -n 128 > nfsd: nfsd count 128; reset to 4 > > > What am I forgetting here ? > I just had a look at the source code http://www.freebsd.org/cgi/cvsweb.cgi/src/usr.sbin/nfsd/nfsd.c which limits the maximum number of nfsd children to 20 in some releases R4 (I have not found the tag for R4.11, so I am not sure, if this holds true for your system...): #define MAXNFSDCNT 20 128 clients is not possible with your nfsd version. So you might want to recompile your nfsd... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 18:42:57 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4499716A409 for ; Mon, 10 Apr 2006 18:42:57 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D92A043D45 for ; Mon, 10 Apr 2006 18:42:54 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k3AIgrrg032349; Mon, 10 Apr 2006 13:42:54 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <443AA721.2080902@centtech.com> Date: Mon, 10 Apr 2006 13:42:41 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5 (X11/20060402) MIME-Version: 1.0 To: Nicolas KOWALSKI References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> <443A8842.6060802@centtech.com> <443A97F9.8090601@centtech.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1389/Mon Apr 10 07:58:55 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 18:42:57 -0000 Nicolas KOWALSKI wrote: > Eric Anderson writes: > >> Nicolas KOWALSKI wrote: >>> Eric Anderson writes: >>> >>>> Nicolas KOWALSKI wrote: >>>>> Yes, this is exactly what is happening. To add some precision, some >>>>> students here use calculation applications >>>>> that allocate a lot of disk space, ususally more than their allowed >>>>> home quotas; when by error they launch these apps in their home >>>>> directories, instead of their workstation dedicated space, it makes >>>>> the server go to its knees on the NFS client side. >>>> When you say 'to it's knees' - what do you mean exactly? How many >>>> clients do you have, how much memory is on the server, and how many >>>> nfsd threads are you using? What kind of load average do you see >>>> during this (on the server)? >>> Sorry for the imprecision. >>> The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320 >>> 76GB disks and controller. It is accessed by NFS from ~100 Unix >>> (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The >>> network connection is GB ethernet. >>> During slowdowns, it's only from a NFS client view that the server >>> does not respond. For example, a simple 'ls' in my home directory is >>> almost immediate, but when it slows down, it can take up to 2 minutes. >>> On the server, the load average goes to 0.5, compared to a default >>> maximum of 0.15-0.20. The nfsd processus shows them in the state >>> "biowr" in top, but nothing is really written, because the quotas >>> system block any further writes to the user exceeding her/his quotas. >>> >> In this case (which is what I suspected), try bumping up your nfsd >> threads to 128. I set mine very high (I have around 1000 clients), >> and I can say there aren't really ill-effects besides a bit of memory >> usage (which you have plenty of). I suspect increasing the threads >> will neutralize this problem for you. > > Thanks for your suggestion. > > However, I am apparently not able to change the default. I stopped the > nfsd master process (kill -USR1 as written in the manpage), then > started it: > > pave# nfsd -t -u -n 128 > nfsd: nfsd count 128; reset to 4 > > > What am I forgetting here ? oops - forgot you are running 5.x. Try this: cd /usr/src/usr.sbin change the line: #define MAXNFSDCNT 128 make make install now try to restart.. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon Apr 10 20:29:24 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1A43D16A400 for ; Mon, 10 Apr 2006 20:29:24 +0000 (UTC) (envelope-from Nicolas.Kowalski@imag.fr) Received: from imag.imag.fr (imag.imag.fr [129.88.30.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9FFCD43D67 for ; Mon, 10 Apr 2006 20:29:16 +0000 (GMT) (envelope-from Nicolas.Kowalski@imag.fr) Received: from mail-veri.imag.fr (mail-veri.imag.fr [129.88.43.52]) by imag.imag.fr (8.13.6/8.13.6) with ESMTP id k3AKTCbO007565 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Mon, 10 Apr 2006 22:29:13 +0200 (CEST) Received: from corbeau.imag.fr ([129.88.43.162]) by mail-veri.imag.fr with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1FT30a-00043O-Rp for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 22:29:12 +0200 Received: from kowalski by corbeau.imag.fr with local (Exim 4.50) id 1FT30a-0001Wo-Ov for freebsd-fs@FreeBSD.org; Mon, 10 Apr 2006 22:29:12 +0200 To: freebsd-fs@FreeBSD.org References: <20060410184007.68438.qmail@web30306.mail.mud.yahoo.com> From: Nicolas KOWALSKI Date: Mon, 10 Apr 2006 22:29:12 +0200 In-Reply-To: <20060410184007.68438.qmail@web30306.mail.mud.yahoo.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: Nicolas Kowalski X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (imag.imag.fr [129.88.30.1]); Mon, 10 Apr 2006 22:29:13 +0200 (CEST) X-IMAG-MailScanner-Information: Please contact IMAG DMI for more information X-IMAG-MailScanner: Found to be clean X-MailScanner-From: kowalski@imag.fr Cc: Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Apr 2006 20:29:24 -0000 "R. B. Riddick" writes: > --- Nicolas KOWALSKI wrote: >> pave# nfsd -t -u -n 128 >> nfsd: nfsd count 128; reset to 4 >> >> What am I forgetting here ? >> > I just had a look at the source code > http://www.freebsd.org/cgi/cvsweb.cgi/src/usr.sbin/nfsd/nfsd.c > which limits the maximum number of nfsd children to 20 in some > releases R4 > (I have not found the tag for R4.11, so I am not sure, if this > holds true for your system...): > #define MAXNFSDCNT 20 > > 128 clients is not possible with your nfsd version. > > So you might want to recompile your nfsd... I edited nfsd.c as you wrote, and it worked, Thanks. The server is now running 128 nfsd processes. Next, I will test/stress the server with the applications causing harm. Many thanks to all, -- Nicolas From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 00:40:48 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 289A316A401 for ; Tue, 11 Apr 2006 00:40:48 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9592243D45 for ; Tue, 11 Apr 2006 00:40:45 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [192.168.254.11] (junior.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k3B0egOo036723; Mon, 10 Apr 2006 18:40:43 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <443AFB03.6060301@samsco.org> Date: Mon, 10 Apr 2006 18:40:35 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20051230 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Diego Woitasen References: <1144687418.11014.9.camel@diegows> In-Reply-To: <1144687418.11014.9.camel@diegows> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.4 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: freebsd-fs@freebsd.org Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 00:40:48 -0000 Diego Woitasen wrote: > I want to know how a file is deleted in a ufs2 filesystem, specifically > what happen with the information in the inode. The information is > deleted to or the inode is marked as free but the information (uid, gid, > blocks, times, etc) remains there? > > I read the chapter 8 of 'Design and implementation of FreeBSD" and "a > Fast file system for Unix", but i can't see the answer. > > Reading the code is an interesting choice, but is the last resource :) > > > thanks! > > Two things happen when a file gets 'deleted'. First is that the directory entry for the filename gets cleared from the directory, and the reference count on the inode is decremented. However, a 'file' can have multiple hard links, or it might still be opened by a program. So nothing further might happen until the reference count goes to 0. When that happens, the inode is zeroed and the free block bitmaps are updated to indicate that the data blocks and any indirect blocks have been freed. Softupdates complicates this by ordering the operations, but that's a whole other discussion. But to specifically answer your question, when an inode gets freed it is also zeroed and any information in it is lost permanently. It's not like MSDOS FAT where just a bit gets set in the directory entry and the information remains valid until it is re-allocated and overwritten. IOW, there is no easy way to undelete a file. Scott From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 01:20:53 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 26F1316A404 for ; Tue, 11 Apr 2006 01:20:53 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id A5E7543D45 for ; Tue, 11 Apr 2006 01:20:52 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.21] (andersonbox1.centtech.com [192.168.42.21]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k3B1Kphq048906; Mon, 10 Apr 2006 20:20:51 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <443B0466.6040508@centtech.com> Date: Mon, 10 Apr 2006 20:20:38 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5 (X11/20060402) MIME-Version: 1.0 To: Scott Long References: <1144687418.11014.9.camel@diegows> <443AFB03.6060301@samsco.org> In-Reply-To: <443AFB03.6060301@samsco.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1389/Mon Apr 10 07:58:55 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 01:20:53 -0000 Scott Long wrote: > Diego Woitasen wrote: >> I want to know how a file is deleted in a ufs2 filesystem, specifically >> what happen with the information in the inode. The information is >> deleted to or the inode is marked as free but the information (uid, gid, >> blocks, times, etc) remains there? >> >> I read the chapter 8 of 'Design and implementation of FreeBSD" and "a >> Fast file system for Unix", but i can't see the answer. >> >> Reading the code is an interesting choice, but is the last resource :) >> >> >> thanks! >> >> > > Two things happen when a file gets 'deleted'. First is that the > directory entry for the filename gets cleared from the directory, and > the reference count on the inode is decremented. However, a 'file' can > have multiple hard links, or it might still be opened by a program. > So nothing further might happen until the reference count goes to 0. > When that happens, the inode is zeroed and the free block bitmaps > are updated to indicate that the data blocks and any indirect blocks > have been freed. Softupdates complicates this by ordering the > operations, but that's a whole other discussion. > > But to specifically answer your question, when an inode gets freed it > is also zeroed and any information in it is lost permanently. It's > not like MSDOS FAT where just a bit gets set in the directory entry > and the information remains valid until it is re-allocated and > overwritten. IOW, there is no easy way to undelete a file. Hmm.. Can you explain then how a tool could recover rm'ed files (or just point me to the code snippet)? There are a few tools that do this. I was under the understanding that the direct/indirect/* lists got blasted away, but parts of the inode were left over. Thanks! Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 02:04:15 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0214A16A409 for ; Tue, 11 Apr 2006 02:04:15 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (megan.kiwi-computer.com [63.224.10.3]) by mx1.FreeBSD.org (Postfix) with SMTP id 46BD743D46 for ; Tue, 11 Apr 2006 02:04:13 +0000 (GMT) (envelope-from rick@kiwi-computer.com) Received: (qmail 44840 invoked by uid 2001); 11 Apr 2006 02:04:12 -0000 Date: Mon, 10 Apr 2006 21:04:12 -0500 From: "Rick C. Petty" To: Eric Anderson Message-ID: <20060411020412.GA44643@megan.kiwi-computer.com> References: <1144687418.11014.9.camel@diegows> <443AFB03.6060301@samsco.org> <443B0466.6040508@centtech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <443B0466.6040508@centtech.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 02:04:15 -0000 On Mon, Apr 10, 2006 at 08:20:38PM -0500, Eric Anderson wrote: > > Hmm.. Can you explain then how a tool could recover rm'ed files (or just > point me to the code snippet)? There are a few tools that do this. I > was under the understanding that the direct/indirect/* lists got blasted > away, but parts of the inode were left over. Nope, the whole inode gets cleared. I thought this too and discovered the hard way. Maybe without softupdates, I'm not sure?? There doesn't seem to be many tools that do this, at least with UFS2. I wrote my own program to try to recover a toasted drive. My program searches the free blocks on the disk looking for indirect blocks (i.e. blocks that have certain characteristics), validates those blocks, then pieces together the blocks into unnamed files (file names are stored in the directories, not in the inode or elsewhere). This gives you all but the first 12 blocks of the file, due to the on-disk structure of inodes. It tries some heuristics to guess at these blocks, but the success rate is very low. As long as fewer blocks have been allocated & written to the disk then are available on the disk, it works well (because the allocation algorithm is highly deterministic). Writing the program was a major headache, and it didn't work as well as I was hoping. However it did allow me to spend much time into our UFS/FFS code (where I did discover some potential bugs-- haven't had time to submit patches yet). I started this program because ffsrecov (still) doesn't work with UFS2. There is a utility (sysutils/tct) which does something similar. It makes a copy of all the free blocks and tries piecing them together, using file(1) to verify & classify the files. Unfortunately this tool wasn't useful to me. My program instead analyzes the disk as a whole and makes determinations based on the block numbers (i.e. it finds indirect blocks which point only to freed blocks). Inodes get scrubbed when the files are completely unlinked. "rm -r" is even more invasive, as the directories which get deleted tend to have junk pointers to freed blocks. The freed data blocks remain intact until the FFS block allocator decides to use the blocks again. It's messy to try to recover deleted files. I've learned (many times) the hard way that there's no substitute for periodic backups, archiving, and redundant mirroring! -- Rick C. Petty From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 02:37:27 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3171316A401 for ; Tue, 11 Apr 2006 02:37:27 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D036543D49 for ; Tue, 11 Apr 2006 02:37:26 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.21] (andersonbox1.centtech.com [192.168.42.21]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k3B2bP7K053025; Mon, 10 Apr 2006 21:37:25 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <443B1658.1060706@centtech.com> Date: Mon, 10 Apr 2006 21:37:12 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5 (X11/20060402) MIME-Version: 1.0 To: "Rick C. Petty" References: <1144687418.11014.9.camel@diegows> <443AFB03.6060301@samsco.org> <443B0466.6040508@centtech.com> <20060411020412.GA44643@megan.kiwi-computer.com> In-Reply-To: <20060411020412.GA44643@megan.kiwi-computer.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1389/Mon Apr 10 07:58:55 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 02:37:27 -0000 Rick C. Petty wrote: > On Mon, Apr 10, 2006 at 08:20:38PM -0500, Eric Anderson wrote: >> Hmm.. Can you explain then how a tool could recover rm'ed files (or just >> point me to the code snippet)? There are a few tools that do this. I >> was under the understanding that the direct/indirect/* lists got blasted >> away, but parts of the inode were left over. > > Nope, the whole inode gets cleared. I thought this too and discovered the > hard way. Maybe without softupdates, I'm not sure?? > > There doesn't seem to be many tools that do this, at least with UFS2. I > wrote my own program to try to recover a toasted drive. My program > searches the free blocks on the disk looking for indirect blocks (i.e. > blocks that have certain characteristics), validates those blocks, then > pieces together the blocks into unnamed files (file names are stored in the > directories, not in the inode or elsewhere). This gives you all but the > first 12 blocks of the file, due to the on-disk structure of inodes. It > tries some heuristics to guess at these blocks, but the success rate is > very low. As long as fewer blocks have been allocated & written to the > disk then are available on the disk, it works well (because the allocation > algorithm is highly deterministic). > > Writing the program was a major headache, and it didn't work as well as I > was hoping. However it did allow me to spend much time into our UFS/FFS > code (where I did discover some potential bugs-- haven't had time to > submit patches yet). I started this program because ffsrecov (still) > doesn't work with UFS2. > > There is a utility (sysutils/tct) which does something similar. It makes a > copy of all the free blocks and tries piecing them together, using file(1) > to verify & classify the files. Unfortunately this tool wasn't useful to > me. My program instead analyzes the disk as a whole and makes > determinations based on the block numbers (i.e. it finds indirect blocks > which point only to freed blocks). > > Inodes get scrubbed when the files are completely unlinked. "rm -r" is > even more invasive, as the directories which get deleted tend to have junk > pointers to freed blocks. The freed data blocks remain intact until the > FFS block allocator decides to use the blocks again. It's messy to try to > recover deleted files. I've learned (many times) the hard way that there's > no substitute for periodic backups, archiving, and redundant mirroring! Thanks! This is great.. I appreciate all your hard work you've done, even if it is for nothing except for learning :). Thanks again for the details and corrections! Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 02:53:37 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C4EE916A400 for ; Tue, 11 Apr 2006 02:53:37 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1842A43D49 for ; Tue, 11 Apr 2006 02:53:36 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [192.168.254.11] (junior.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k3B2rWJO037403; Mon, 10 Apr 2006 20:53:34 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <443B1A24.8040103@samsco.org> Date: Mon, 10 Apr 2006 20:53:24 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20051230 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Rick C. Petty" References: <1144687418.11014.9.camel@diegows> <443AFB03.6060301@samsco.org> <443B0466.6040508@centtech.com> <20060411020412.GA44643@megan.kiwi-computer.com> In-Reply-To: <20060411020412.GA44643@megan.kiwi-computer.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.4 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: freebsd-fs@freebsd.org Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 02:53:37 -0000 Rick C. Petty wrote: > On Mon, Apr 10, 2006 at 08:20:38PM -0500, Eric Anderson wrote: > >>Hmm.. Can you explain then how a tool could recover rm'ed files (or just >>point me to the code snippet)? There are a few tools that do this. I >>was under the understanding that the direct/indirect/* lists got blasted >>away, but parts of the inode were left over. > > > Nope, the whole inode gets cleared. I thought this too and discovered the > hard way. Maybe without softupdates, I'm not sure?? > It happens the same with or without softupdates. The one difference is between UFS1 and UFS2. With UFS1, doing a newfs also scrubs the inode blocks. UFS2 newfs doesn't scrub them, and the filesystem relies on a flag to know to scrub them before using them the first time. This was done purely as an optimization for newfs, though it could technically be used to help recover a filesystem that was accidentally newfs'd. > There doesn't seem to be many tools that do this, at least with UFS2. I > wrote my own program to try to recover a toasted drive. My program > searches the free blocks on the disk looking for indirect blocks (i.e. > blocks that have certain characteristics), validates those blocks, then > pieces together the blocks into unnamed files (file names are stored in the > directories, not in the inode or elsewhere). This gives you all but the > first 12 blocks of the file, due to the on-disk structure of inodes. It > tries some heuristics to guess at these blocks, but the success rate is > very low. As long as fewer blocks have been allocated & written to the > disk then are available on the disk, it works well (because the allocation > algorithm is highly deterministic). > > Writing the program was a major headache, and it didn't work as well as I > was hoping. However it did allow me to spend much time into our UFS/FFS > code (where I did discover some potential bugs-- haven't had time to > submit patches yet). I started this program because ffsrecov (still) > doesn't work with UFS2. I've heard rumors of a new ffsrecov written in Python that works with UFS2. You might want to contact John-Mark Gurney (jmg@freebsd.org) about it. Scott From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 07:09:09 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 182ED16A402 for ; Tue, 11 Apr 2006 07:09:09 +0000 (UTC) (envelope-from Nicolas.Kowalski@imag.fr) Received: from imag.imag.fr (imag.imag.fr [129.88.30.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 849EA43D48 for ; Tue, 11 Apr 2006 07:09:08 +0000 (GMT) (envelope-from Nicolas.Kowalski@imag.fr) Received: from mail-veri.imag.fr (mail-veri.imag.fr [129.88.43.52]) by imag.imag.fr (8.13.6/8.13.6) with ESMTP id k3B792NV020677 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 11 Apr 2006 09:09:03 +0200 (CEST) Received: from corbeau.imag.fr ([129.88.43.162]) by mail-veri.imag.fr with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1FTCzm-0003IL-L0 for freebsd-fs@FreeBSD.org; Tue, 11 Apr 2006 09:09:02 +0200 Received: from kowalski by corbeau.imag.fr with local (Exim 4.50) id 1FTCzm-0002Jr-48 for freebsd-fs@FreeBSD.org; Tue, 11 Apr 2006 09:09:02 +0200 To: freebsd-fs@FreeBSD.org References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> <443A8842.6060802@centtech.com> <443A97F9.8090601@centtech.com> From: Nicolas KOWALSKI Date: Tue, 11 Apr 2006 09:09:02 +0200 In-Reply-To: <443A97F9.8090601@centtech.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: Nicolas Kowalski X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (imag.imag.fr [129.88.30.1]); Tue, 11 Apr 2006 09:09:03 +0200 (CEST) X-IMAG-MailScanner-Information: Please contact IMAG DMI for more information X-IMAG-MailScanner: Found to be clean X-MailScanner-From: kowalski@imag.fr Cc: Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 07:09:09 -0000 Eric Anderson writes: > Nicolas KOWALSKI wrote: >> Eric Anderson writes: >> >>> Nicolas KOWALSKI wrote: >>>> Yes, this is exactly what is happening. To add some precision, some >>>> students here use calculation applications >>>> that allocate a lot of disk space, ususally more than their allowed >>>> home quotas; when by error they launch these apps in their home >>>> directories, instead of their workstation dedicated space, it makes >>>> the server go to its knees on the NFS client side. >>> When you say 'to it's knees' - what do you mean exactly? How many >>> clients do you have, how much memory is on the server, and how many >>> nfsd threads are you using? What kind of load average do you see >>> during this (on the server)? >> Sorry for the imprecision. >> The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320 >> 76GB disks and controller. It is accessed by NFS from ~100 Unix >> (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The >> network connection is GB ethernet. >> During slowdowns, it's only from a NFS client view that the server >> does not respond. For example, a simple 'ls' in my home directory is >> almost immediate, but when it slows down, it can take up to 2 minutes. >> On the server, the load average goes to 0.5, compared to a default >> maximum of 0.15-0.20. The nfsd processus shows them in the state >> "biowr" in top, but nothing is really written, because the quotas >> system block any further writes to the user exceeding her/his quotas. >> > > In this case (which is what I suspected), try bumping up your nfsd > threads to 128. I set mine very high (I have around 1000 clients), > and I can say there aren't really ill-effects besides a bit of memory > usage (which you have plenty of). I suspect increasing the threads > will neutralize this problem for you. Using 128 nfsd threads, I stressed the server, by running on a NFS client a small C program, writting continuously in a file, so that the user "biguser" (account stored on /export/home2) exceeds his quota. It half-works: during the test, users working on another disk (/export/home) did not see any difference, but users working on the same disk that "biguser" (/export/home2) where almost halted. So, this is better, because before everybody was halted, but there is still a problem. Any other tips ? Thanks, -- Nicolas From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 11:11:12 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4DC1A16A404 for ; Tue, 11 Apr 2006 11:11:12 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id AD63343D49 for ; Tue, 11 Apr 2006 11:11:11 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k3BBBAhe074385; Tue, 11 Apr 2006 06:11:10 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <443B8EC1.8080004@centtech.com> Date: Tue, 11 Apr 2006 06:10:57 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5 (X11/20060402) MIME-Version: 1.0 To: Nicolas KOWALSKI References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> <443A8842.6060802@centtech.com> <443A97F9.8090601@centtech.com> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1391/Tue Apr 11 04:53:41 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 11:11:12 -0000 Nicolas KOWALSKI wrote: > Eric Anderson writes: > >> Nicolas KOWALSKI wrote: >>> Eric Anderson writes: >>> >>>> Nicolas KOWALSKI wrote: >>>>> Yes, this is exactly what is happening. To add some precision, some >>>>> students here use calculation applications >>>>> that allocate a lot of disk space, ususally more than their allowed >>>>> home quotas; when by error they launch these apps in their home >>>>> directories, instead of their workstation dedicated space, it makes >>>>> the server go to its knees on the NFS client side. >>>> When you say 'to it's knees' - what do you mean exactly? How many >>>> clients do you have, how much memory is on the server, and how many >>>> nfsd threads are you using? What kind of load average do you see >>>> during this (on the server)? >>> Sorry for the imprecision. >>> The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320 >>> 76GB disks and controller. It is accessed by NFS from ~100 Unix >>> (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The >>> network connection is GB ethernet. >>> During slowdowns, it's only from a NFS client view that the server >>> does not respond. For example, a simple 'ls' in my home directory is >>> almost immediate, but when it slows down, it can take up to 2 minutes. >>> On the server, the load average goes to 0.5, compared to a default >>> maximum of 0.15-0.20. The nfsd processus shows them in the state >>> "biowr" in top, but nothing is really written, because the quotas >>> system block any further writes to the user exceeding her/his quotas. >>> >> In this case (which is what I suspected), try bumping up your nfsd >> threads to 128. I set mine very high (I have around 1000 clients), >> and I can say there aren't really ill-effects besides a bit of memory >> usage (which you have plenty of). I suspect increasing the threads >> will neutralize this problem for you. > > Using 128 nfsd threads, I stressed the server, by running on a NFS > client a small C program, writting continuously in a file, so that the > user "biguser" (account stored on /export/home2) exceeds his quota. > > It half-works: during the test, users working on another disk > (/export/home) did not see any difference, but users working on the > same disk that "biguser" (/export/home2) where almost halted. > > So, this is better, because before everybody was halted, but there is > still a problem. > > Any other tips ? Watch gstat during the testing, and see if the disk that holds the full partition is really busy. I'm betting it's thrashing the disk continually checking for free space. I don't think there's any way to avoid that. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 12:56:37 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1518616A403 for ; Tue, 11 Apr 2006 12:56:37 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout1.pacific.net.au (mailout1.pacific.net.au [61.8.0.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4348443FDA for ; Tue, 11 Apr 2006 12:56:24 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.0.87]) by mailout1.pacific.net.au (Postfix) with ESMTP id 85C0F32FD91; Tue, 11 Apr 2006 22:56:19 +1000 (EST) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (8.13.4/8.13.4/Debian-3) with ESMTP id k3BCuBFF016023; Tue, 11 Apr 2006 22:56:16 +1000 Date: Tue, 11 Apr 2006 22:56:11 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Scott Long In-Reply-To: <443AFB03.6060301@samsco.org> Message-ID: <20060411210858.G46778@delplex.bde.org> References: <1144687418.11014.9.camel@diegows> <443AFB03.6060301@samsco.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 12:56:37 -0000 On Mon, 10 Apr 2006, Scott Long wrote: > Diego Woitasen wrote: >> I want to know how a file is deleted in a ufs2 filesystem, specifically >> what happen with the information in the inode. The information is >> deleted to or the inode is marked as free but the information (uid, gid, >> blocks, times, etc) remains there? >> >> I read the chapter 8 of 'Design and implementation of FreeBSD" and "a >> Fast file system for Unix", but i can't see the answer. >> >> Reading the code is an interesting choice, but is the last resource :) It should be nearly the first resort. > Two things happen when a file gets 'deleted'. First is that the directory > entry for the filename gets cleared from the directory, and > the reference count on the inode is decremented. However, a 'file' can > have multiple hard links, or it might still be opened by a program. > So nothing further might happen until the reference count goes to 0. > When that happens, the inode is zeroed and the free block bitmaps > are updated to indicate that the data blocks and any indirect blocks > have been freed. Softupdates complicates this by ordering the operations, > but that's a whole other discussion. No, the inode is not zeroed. When the inode count goes to zero, the file is truncated to size 0. The truncate routine (ffs_truncate() for ffs[1-2] could reasonably preserve the direct and indirect block numbers in the inode since the file is going away, but the truncate routine doesn't know that the context is special so it just clears all the block numbers. Finally, only the mode in the inode is cleared explicitly. I think this mode-claring is somewhat gratuitous and is not done in Linux's ext2fs. Perhaps Linux doesn't clear the block numbers either. Here is debugger output for a live and dead ffs1 inode: Live on-disk inode: % (gdb) p/x *ip % $1 = {di_mode = 0x81a4, di_nlink = 0x1, di_u = {oldids = {0x0, 0x0}}, % di_size = 0x27800, di_atime = 0x443b982f, di_atimensec = 0x0, % di_mtime = 0x443b982f, di_mtimensec = 0x0, di_ctime = 0x443b9839, % di_ctimensec = 0x0, di_db = {0x38, 0x40, 0x48, 0x50, 0x58, 0x60, 0x68, % 0x70, 0x78, 0x31, 0x0, 0x0}, di_ib = {0x0, 0x0, 0x0}, di_flags = 0x2, % di_blocks = 0x13c, di_gen = 0x1ba2f568, di_uid = 0xf, di_gid = 0x3e8, % di_spare = {0x0, 0x0}} Dead on-disk inode (same inode after final unlink): % (gdb) p/x *ip % $3 = {di_mode = 0x81a4, di_nlink = 0x1, di_u = {oldids = {0x0, 0x0}}, % di_size = 0x27800, di_atime = 0x443b982f, di_atimensec = 0x0, % di_mtime = 0x443b982f, di_mtimensec = 0x0, di_ctime = 0x443b9839, % di_ctimensec = 0x0, di_db = {0x38, 0x40, 0x48, 0x50, 0x58, 0x60, 0x68, % 0x70, 0x78, 0x31, 0x0, 0x0}, di_ib = {0x0, 0x0, 0x0}, di_flags = 0x2, % di_blocks = 0x13c, di_gen = 0x1ba2f568, di_uid = 0xf, di_gid = 0x3e8, % di_spare = {0x0, 0x0}} Comparing the fields: di_mode: gratuitously cleared di_nlink: had to be cleared for final unlink and to show that this inode is unreferenced di_u: unused before and after di_size: cleared as a side effect of truncate di_atime*: preserved di_mtime*: preserved di_ctime*: preserved di_db: cleared as a side effect of truncate di_ib: remains clear (would be cleared as a side effect of truncate) di_flags: preserved di_blocks: cleared as a side effect of truncate di_gen: preserved di_uid: preserved di_gid: preserved di_spare: remains clear Another debugging session showed that di_mtime and di_ctime are sometimes changed as a side effect of the final unlink. In ext2s under Linux, there is an explicit setting of the ctime on the final unlink (giving a deathtime) but under FreeBSD there is apparently only a change as a side effect of converting previous marks for update to updates. So only the least important information is preserved. You can only recover empty files including some of their metadata. > But to specifically answer your question, when an inode gets freed it > is also zeroed and any information in it is lost permanently. It's > not like MSDOS FAT where just a bit gets set in the directory entry > and the information remains valid until it is re-allocated and overwritten. It's actually quite similar, with small implementation details making a big difference. With msdosfs, at least under FreeBSD and even more fundamentally than for ffs[1-2] (since the FAT corresponds to the bitmap of free blocks so it must be cleared), truncation of the file results in all the block numbers for the file in the FAT being cleared. However, the first block (cluster) number in the directory entry somehow escapes clearing by the truncate, so the first cluster of a deleted file can be recovered until it or its directory entry is overwritten. ffs[1-2] can potentially retain info for _all_ blocks since it has more indirection -- the inode and indirect blocks provide places to retain all the old block numbers just like msdosfs's directory entry provides a place to retain 1. For directory entries, removal is almost equally harmless for ffs[1-2] as for msdosfs. Here is a difference of hex dumps of ffs1 directories for live and dead entries "foo" and "bar" % --- live Tue Apr 11 22:09:28 2006 % +++ dead Tue Apr 11 22:09:37 2006 % @@ -1,5 +1,5 @@ % 00000000 02 00 00 00 0C 00 04 01 2E 00 00 00 02 00 00 00 ................ ---d_ino--- recln ty nl -name -pad- ---d_ino--- % -00000010 0C 00 04 02 2E 2E 00 00 03 00 00 00 10 00 04 05 ................ % +00000010 0C 00 04 02 2E 2E 00 00 03 00 00 00 E8 01 04 05 ................ recln ty nl --name-- pd ---d_ino--- recln ty nl % 00000020 2E 73 6E 61 70 00 00 00 04 00 00 00 0C 00 08 03 .snap........... ----name--------- -pad- ... % 00000030 66 6F 6F 00 05 00 00 00 CC 01 04 03 62 61 72 00 foo.........bar. % 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Removal of "foo" and "bar" has just changed d_reclen for ".snap" so that the subsequent entries are inside the entry for ".snap" and thus not seen. The subsequent entries including their inode numbers are preserved. More complicated removals would result in a more complicated layout. I think compaction of directory entries rarely if ever occrurs. Recovery of deleted directory entries in msdosfs is certainly simpler since the record length is constant, but the constant record length also makes overwrites of directory entries more likely. Part of the cleared i_mode field in inodes can be recovered from the d_type field in directory entries, but it would be better to use an uncleared i_mode as a consistency check for recovered directory entries. > IOW, there is no easy way to undelete a file. This is currently true, except in the rare case where undelete(2) works. Bruce From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 13:41:51 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D5DD116A402 for ; Tue, 11 Apr 2006 13:41:51 +0000 (UTC) (envelope-from Nicolas.Kowalski@imag.fr) Received: from imag.imag.fr (imag.imag.fr [129.88.30.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4A6BE43D48 for ; Tue, 11 Apr 2006 13:41:50 +0000 (GMT) (envelope-from Nicolas.Kowalski@imag.fr) Received: from mail-veri.imag.fr (mail-veri.imag.fr [129.88.43.52]) by imag.imag.fr (8.13.6/8.13.6) with ESMTP id k3BDfk87029016 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 11 Apr 2006 15:41:46 +0200 (CEST) Received: from corbeau.imag.fr ([129.88.43.162]) by mail-veri.imag.fr with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1FTJ7p-0004oK-Ug for freebsd-fs@FreeBSD.org; Tue, 11 Apr 2006 15:41:45 +0200 Received: from kowalski by corbeau.imag.fr with local (Exim 4.50) id 1FTJ7p-0002oW-Pk for freebsd-fs@FreeBSD.org; Tue, 11 Apr 2006 15:41:45 +0200 To: freebsd-fs@FreeBSD.org References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> <443A8842.6060802@centtech.com> <443A97F9.8090601@centtech.com> <443B8EC1.8080004@centtech.com> From: Nicolas KOWALSKI Date: Tue, 11 Apr 2006 15:41:45 +0200 In-Reply-To: <443B8EC1.8080004@centtech.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: Nicolas Kowalski X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (imag.imag.fr [129.88.30.1]); Tue, 11 Apr 2006 15:41:46 +0200 (CEST) X-IMAG-MailScanner-Information: Please contact IMAG DMI for more information X-IMAG-MailScanner: Found to be clean X-MailScanner-From: kowalski@imag.fr Cc: Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 13:41:52 -0000 Eric Anderson writes: > Nicolas KOWALSKI wrote: >> Eric Anderson writes: >> >>> Nicolas KOWALSKI wrote: >>>> Eric Anderson writes: >>>> >>>>> Nicolas KOWALSKI wrote: >>>>>> Yes, this is exactly what is happening. To add some precision, some >>>>>> students here use calculation applications >>>>>> that allocate a lot of disk space, ususally more than their allowed >>>>>> home quotas; when by error they launch these apps in their home >>>>>> directories, instead of their workstation dedicated space, it makes >>>>>> the server go to its knees on the NFS client side. >>>>> When you say 'to it's knees' - what do you mean exactly? How many >>>>> clients do you have, how much memory is on the server, and how many >>>>> nfsd threads are you using? What kind of load average do you see >>>>> during this (on the server)? >>>> Sorry for the imprecision. >>>> The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320 >>>> 76GB disks and controller. It is accessed by NFS from ~100 Unix >>>> (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The >>>> network connection is GB ethernet. >>>> During slowdowns, it's only from a NFS client view that the server >>>> does not respond. For example, a simple 'ls' in my home directory is >>>> almost immediate, but when it slows down, it can take up to 2 minutes. >>>> On the server, the load average goes to 0.5, compared to a default >>>> maximum of 0.15-0.20. The nfsd processus shows them in the state >>>> "biowr" in top, but nothing is really written, because the quotas >>>> system block any further writes to the user exceeding her/his quotas. >>>> >>> In this case (which is what I suspected), try bumping up your nfsd >>> threads to 128. I set mine very high (I have around 1000 clients), >>> and I can say there aren't really ill-effects besides a bit of memory >>> usage (which you have plenty of). I suspect increasing the threads >>> will neutralize this problem for you. >> Using 128 nfsd threads, I stressed the server, by running on a NFS >> client a small C program, writting continuously in a file, so that the >> user "biguser" (account stored on /export/home2) exceeds his quota. >> It half-works: during the test, users working on another disk >> (/export/home) did not see any difference, but users working on the >> same disk that "biguser" (/export/home2) where almost halted. >> So, this is better, because before everybody was halted, but there is >> still a problem. >> Any other tips ? > > Watch gstat during the testing, and see if the disk that holds the > full partition is really busy. I'm betting it's thrashing the disk > continually checking for free space. I don't think there's any way to > avoid that. Mh, I did not find this "gstat" tool on the system or in the ports; perhaps is it in >= 5.x ? (the server is running 4.11-p15). It is sad I can not do anything about it: such a server pulled down by a single NFS-client. :-( However, Many Thanks for your help and tips. Best regards, -- Nicolas From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 15:12:36 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E16C416A400 for ; Tue, 11 Apr 2006 15:12:35 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7E69C43D45 for ; Tue, 11 Apr 2006 15:12:35 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k3BFCYMU084847; Tue, 11 Apr 2006 10:12:34 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <443BC755.1080905@centtech.com> Date: Tue, 11 Apr 2006 10:12:21 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5 (X11/20060402) MIME-Version: 1.0 To: Nicolas KOWALSKI References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> <443A8842.6060802@centtech.com> <443A97F9.8090601@centtech.com> <443B8EC1.8080004@centtech.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1391/Tue Apr 11 04:53:41 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 15:12:36 -0000 Nicolas KOWALSKI wrote: > Eric Anderson writes: > >> Nicolas KOWALSKI wrote: >>> Eric Anderson writes: >>> >>>> Nicolas KOWALSKI wrote: >>>>> Eric Anderson writes: >>>>> >>>>>> Nicolas KOWALSKI wrote: >>>>>>> Yes, this is exactly what is happening. To add some precision, some >>>>>>> students here use calculation applications >>>>>>> that allocate a lot of disk space, ususally more than their allowed >>>>>>> home quotas; when by error they launch these apps in their home >>>>>>> directories, instead of their workstation dedicated space, it makes >>>>>>> the server go to its knees on the NFS client side. >>>>>> When you say 'to it's knees' - what do you mean exactly? How many >>>>>> clients do you have, how much memory is on the server, and how many >>>>>> nfsd threads are you using? What kind of load average do you see >>>>>> during this (on the server)? >>>>> Sorry for the imprecision. >>>>> The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320 >>>>> 76GB disks and controller. It is accessed by NFS from ~100 Unix >>>>> (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The >>>>> network connection is GB ethernet. >>>>> During slowdowns, it's only from a NFS client view that the server >>>>> does not respond. For example, a simple 'ls' in my home directory is >>>>> almost immediate, but when it slows down, it can take up to 2 minutes. >>>>> On the server, the load average goes to 0.5, compared to a default >>>>> maximum of 0.15-0.20. The nfsd processus shows them in the state >>>>> "biowr" in top, but nothing is really written, because the quotas >>>>> system block any further writes to the user exceeding her/his quotas. >>>>> >>>> In this case (which is what I suspected), try bumping up your nfsd >>>> threads to 128. I set mine very high (I have around 1000 clients), >>>> and I can say there aren't really ill-effects besides a bit of memory >>>> usage (which you have plenty of). I suspect increasing the threads >>>> will neutralize this problem for you. >>> Using 128 nfsd threads, I stressed the server, by running on a NFS >>> client a small C program, writting continuously in a file, so that the >>> user "biguser" (account stored on /export/home2) exceeds his quota. >>> It half-works: during the test, users working on another disk >>> (/export/home) did not see any difference, but users working on the >>> same disk that "biguser" (/export/home2) where almost halted. >>> So, this is better, because before everybody was halted, but there is >>> still a problem. >>> Any other tips ? >> Watch gstat during the testing, and see if the disk that holds the >> full partition is really busy. I'm betting it's thrashing the disk >> continually checking for free space. I don't think there's any way to >> avoid that. > > Mh, I did not find this "gstat" tool on the system or in the ports; > perhaps is it in >= 5.x ? (the server is running 4.11-p15). > > It is sad I can not do anything about it: such a server pulled down by > a single NFS-client. :-( *sigh* - I should really pay more attention to the beginning of the thread. I thought you were on 5.x, so my mistake. You'll need to use iostat to see what's busy with your disks. I strongly recommend going to 6.x if you can.. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 17:43:17 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8BCC416A404 for ; Tue, 11 Apr 2006 17:43:17 +0000 (UTC) (envelope-from Nicolas.Kowalski@imag.fr) Received: from imag.imag.fr (imag.imag.fr [129.88.30.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id D26DC43D5F for ; Tue, 11 Apr 2006 17:43:15 +0000 (GMT) (envelope-from Nicolas.Kowalski@imag.fr) Received: from mail-veri.imag.fr (mail-veri.imag.fr [129.88.43.52]) by imag.imag.fr (8.13.6/8.13.6) with ESMTP id k3BHhBwu011122 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NO) for ; Tue, 11 Apr 2006 19:43:11 +0200 (CEST) Received: from corbeau.imag.fr ([129.88.43.162]) by mail-veri.imag.fr with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1FTMtT-0002Ry-5t for freebsd-fs@FreeBSD.org; Tue, 11 Apr 2006 19:43:11 +0200 Received: from kowalski by corbeau.imag.fr with local (Exim 4.50) id 1FTMtT-0003Ak-08 for freebsd-fs@FreeBSD.org; Tue, 11 Apr 2006 19:43:11 +0200 To: freebsd-fs@FreeBSD.org References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> <443A8842.6060802@centtech.com> <443A97F9.8090601@centtech.com> <443B8EC1.8080004@centtech.com> <443BC755.1080905@centtech.com> From: Nicolas KOWALSKI Date: Tue, 11 Apr 2006 19:43:10 +0200 In-Reply-To: <443BC755.1080905@centtech.com> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: Nicolas Kowalski X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-1.6 (imag.imag.fr [129.88.30.1]); Tue, 11 Apr 2006 19:43:11 +0200 (CEST) X-IMAG-MailScanner-Information: Please contact IMAG DMI for more information X-IMAG-MailScanner: Found to be clean X-MailScanner-From: kowalski@imag.fr Cc: Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 17:43:17 -0000 Eric Anderson writes: > Nicolas KOWALSKI wrote: >> Eric Anderson writes: >> >>> Nicolas KOWALSKI wrote: >>>> Eric Anderson writes: >>>> >>>>> Nicolas KOWALSKI wrote: >>>>>> Eric Anderson writes: >>>>>> >>>>>>> Nicolas KOWALSKI wrote: >>>>>>>> Yes, this is exactly what is happening. To add some precision, some >>>>>>>> students here use calculation applications >>>>>>>> that allocate a lot of disk space, ususally more than their allowed >>>>>>>> home quotas; when by error they launch these apps in their home >>>>>>>> directories, instead of their workstation dedicated space, it makes >>>>>>>> the server go to its knees on the NFS client side. >>>>>>> When you say 'to it's knees' - what do you mean exactly? How many >>>>>>> clients do you have, how much memory is on the server, and how many >>>>>>> nfsd threads are you using? What kind of load average do you see >>>>>>> during this (on the server)? >>>>>> Sorry for the imprecision. >>>>>> The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320 >>>>>> 76GB disks and controller. It is accessed by NFS from ~100 Unix >>>>>> (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The >>>>>> network connection is GB ethernet. >>>>>> During slowdowns, it's only from a NFS client view that the server >>>>>> does not respond. For example, a simple 'ls' in my home directory is >>>>>> almost immediate, but when it slows down, it can take up to 2 minutes. >>>>>> On the server, the load average goes to 0.5, compared to a default >>>>>> maximum of 0.15-0.20. The nfsd processus shows them in the state >>>>>> "biowr" in top, but nothing is really written, because the quotas >>>>>> system block any further writes to the user exceeding her/his quotas. >>>>>> >>>>> In this case (which is what I suspected), try bumping up your nfsd >>>>> threads to 128. I set mine very high (I have around 1000 clients), >>>>> and I can say there aren't really ill-effects besides a bit of memory >>>>> usage (which you have plenty of). I suspect increasing the threads >>>>> will neutralize this problem for you. >>>> Using 128 nfsd threads, I stressed the server, by running on a NFS >>>> client a small C program, writting continuously in a file, so that the >>>> user "biguser" (account stored on /export/home2) exceeds his quota. >>>> It half-works: during the test, users working on another disk >>>> (/export/home) did not see any difference, but users working on the >>>> same disk that "biguser" (/export/home2) where almost halted. >>>> So, this is better, because before everybody was halted, but there is >>>> still a problem. >>>> Any other tips ? >>> Watch gstat during the testing, and see if the disk that holds the >>> full partition is really busy. I'm betting it's thrashing the disk >>> continually checking for free space. I don't think there's any way to >>> avoid that. >> Mh, I did not find this "gstat" tool on the system or in the ports; >> perhaps is it in >= 5.x ? (the server is running 4.11-p15). >> It is sad I can not do anything about it: such a server pulled down >> by >> a single NFS-client. :-( > > > *sigh* - I should really pay more attention to the beginning of the > thread. I thought you were on 5.x, so my mistake. You'll need to use > iostat to see what's busy with your disks. Ok, I'll check with that. Thanks. > I strongly recommend going to 6.x if you can.. Yep, it's planned, but only in a few months, when all students/teachers will go to the beach. ;-) Just for my knowledge, what will improve the situation ? Better Locking ? If I read the start of the thread, it looks like the actual quotas system is still under the giant lock, so... -- Nicolas From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 17:51:19 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 243D016A40D for ; Tue, 11 Apr 2006 17:51:19 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 377C543D8B for ; Tue, 11 Apr 2006 17:51:04 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k3BHoxQr092420; Tue, 11 Apr 2006 12:50:59 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <443BEC75.5000808@centtech.com> Date: Tue, 11 Apr 2006 12:50:45 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5 (X11/20060402) MIME-Version: 1.0 To: Nicolas KOWALSKI References: <20060329152608.GB1375@deviant.kiev.zoral.com.ua> <20060410144904.GC1408@deviant.kiev.zoral.com.ua> <443A7C8E.4020203@centtech.com> <443A8842.6060802@centtech.com> <443A97F9.8090601@centtech.com> <443B8EC1.8080004@centtech.com> <443BC755.1080905@centtech.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1391/Tue Apr 11 04:53:41 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: [patch] giant-less quotas for UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 17:51:19 -0000 Nicolas KOWALSKI wrote: > Eric Anderson writes: > >> Nicolas KOWALSKI wrote: >>> Eric Anderson writes: >>> >>>> Nicolas KOWALSKI wrote: >>>>> Eric Anderson writes: >>>>> >>>>>> Nicolas KOWALSKI wrote: >>>>>>> Eric Anderson writes: >>>>>>> >>>>>>>> Nicolas KOWALSKI wrote: >>>>>>>>> Yes, this is exactly what is happening. To add some precision, some >>>>>>>>> students here use calculation applications >>>>>>>>> that allocate a lot of disk space, ususally more than their allowed >>>>>>>>> home quotas; when by error they launch these apps in their home >>>>>>>>> directories, instead of their workstation dedicated space, it makes >>>>>>>>> the server go to its knees on the NFS client side. >>>>>>>> When you say 'to it's knees' - what do you mean exactly? How many >>>>>>>> clients do you have, how much memory is on the server, and how many >>>>>>>> nfsd threads are you using? What kind of load average do you see >>>>>>>> during this (on the server)? >>>>>>> Sorry for the imprecision. >>>>>>> The server is a Dual-Xeon 2.8Ghz, 2GB of RAM, using SCSI3 Ultra320 >>>>>>> 76GB disks and controller. It is accessed by NFS from ~100 Unix >>>>>>> (Linux, Solaris) clients, and by Samba from ~15 Windows XP. The >>>>>>> network connection is GB ethernet. >>>>>>> During slowdowns, it's only from a NFS client view that the server >>>>>>> does not respond. For example, a simple 'ls' in my home directory is >>>>>>> almost immediate, but when it slows down, it can take up to 2 minutes. >>>>>>> On the server, the load average goes to 0.5, compared to a default >>>>>>> maximum of 0.15-0.20. The nfsd processus shows them in the state >>>>>>> "biowr" in top, but nothing is really written, because the quotas >>>>>>> system block any further writes to the user exceeding her/his quotas. >>>>>>> >>>>>> In this case (which is what I suspected), try bumping up your nfsd >>>>>> threads to 128. I set mine very high (I have around 1000 clients), >>>>>> and I can say there aren't really ill-effects besides a bit of memory >>>>>> usage (which you have plenty of). I suspect increasing the threads >>>>>> will neutralize this problem for you. >>>>> Using 128 nfsd threads, I stressed the server, by running on a NFS >>>>> client a small C program, writting continuously in a file, so that the >>>>> user "biguser" (account stored on /export/home2) exceeds his quota. >>>>> It half-works: during the test, users working on another disk >>>>> (/export/home) did not see any difference, but users working on the >>>>> same disk that "biguser" (/export/home2) where almost halted. >>>>> So, this is better, because before everybody was halted, but there is >>>>> still a problem. >>>>> Any other tips ? >>>> Watch gstat during the testing, and see if the disk that holds the >>>> full partition is really busy. I'm betting it's thrashing the disk >>>> continually checking for free space. I don't think there's any way to >>>> avoid that. >>> Mh, I did not find this "gstat" tool on the system or in the ports; >>> perhaps is it in >= 5.x ? (the server is running 4.11-p15). >>> It is sad I can not do anything about it: such a server pulled down >>> by >>> a single NFS-client. :-( >> >> *sigh* - I should really pay more attention to the beginning of the >> thread. I thought you were on 5.x, so my mistake. You'll need to use >> iostat to see what's busy with your disks. > > Ok, I'll check with that. Thanks. > >> I strongly recommend going to 6.x if you can.. > > Yep, it's planned, but only in a few months, when all > students/teachers will go to the beach. ;-) > > Just for my knowledge, what will improve the situation ? Better > Locking ? If I read the start of the thread, it looks like the actual > quotas system is still under the giant lock, so... > There are a lot of file system improvements, from locking to just plain bug fixing. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Apr 11 19:01:34 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B822416A402 for ; Tue, 11 Apr 2006 19:01:34 +0000 (UTC) (envelope-from meinhardto@lagerlouts.com) Received: from lagerlouts.com (BSN-77-132-30.dial-up.dsl.siol.net [193.77.132.30]) by mx1.FreeBSD.org (Postfix) with SMTP id D6E5243D49 for ; Tue, 11 Apr 2006 19:01:33 +0000 (GMT) (envelope-from meinhardto@lagerlouts.com) Message-ID: <000001c65d9a$460ea730$3242a8c0@pwg25> From: "Barakat Meinhardt" To: freebsd-fs@freebsd.org Date: Tue, 11 Apr 2006 15:01:00 -0400 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: eiuui news X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Barakat Meinhardt List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Apr 2006 19:01:34 -0000 De i ar Home Ow j ne w r ,=20 =20 Your c d re r di o t doesn't matter to us !=20 =20 If you O z WN real e n st u at h e and want=20 I b MME v DI s AT a E c f as g h to s d pe f nd ANY way you like,=20 or simply wish to L k OWE k R your monthly=20 p c ayme f nt e s by a third or more,=20 here are the d z eal k s we have T h ODA n Y :=20 =20 $ 48 g 8 , 000 at a 3 , s 67% f p ixe a d - r r at h e=20 $ 3 i 72 , 000 at a 3 , x 90% v q ari l ab c le - r f at h e=20 $ 49 z 2 , 000 at a 3 , r 21% in z te g res p t - only=20 $ 2 h 48 , 000 at a 3 , n 36% f c ix p ed - r c at u e=20 $ 1 c 98 , 000 at a 3 h , 55% v e ari v ab d le - r t at w e=20 =20 H q urr l y, when these d a eal c s are gone,=20 they are gone ! =20 Don't worry about ap f pro c va t l, your=20 c e re f di l t will not d n isquali x fy you !=20 =20 V a is x it our si l te =20 =20 Sincerely, Barakat Meinhardt=20 =20 A t pp w rov y al Manager From owner-freebsd-fs@FreeBSD.ORG Wed Apr 12 03:52:48 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0FB0916A402 for ; Wed, 12 Apr 2006 03:52:48 +0000 (UTC) (envelope-from pfgshield-freebsd@yahoo.com) Received: from web32703.mail.mud.yahoo.com (web32703.mail.mud.yahoo.com [68.142.207.247]) by mx1.FreeBSD.org (Postfix) with SMTP id 92BCB43D45 for ; Wed, 12 Apr 2006 03:52:47 +0000 (GMT) (envelope-from pfgshield-freebsd@yahoo.com) Received: (qmail 20428 invoked by uid 60001); 12 Apr 2006 03:52:46 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=lg3vLVs1QUAuAUqx16WwtXuXGu2mR1ZGbor/NmkPK4g5IMUhTLT28EqDLW42rkOUhQzUJom+W96otZe+A3i6EyfbiLcGGWfrGrqpVkTcbTzEvPsq6912ZGt0aw4KSEx+Uv4HUnquJfenYPBvk1Tsp7foVl4Wd7m6nMH+oz6/iiI= ; Message-ID: <20060412035246.20426.qmail@web32703.mail.mud.yahoo.com> Received: from [69.79.51.65] by web32703.mail.mud.yahoo.com via HTTP; Wed, 12 Apr 2006 05:52:46 CEST Date: Wed, 12 Apr 2006 05:52:46 +0200 (CEST) From: To: Eric Anderson , freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Cc: Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pfgshield-freebsd@yahoo.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Apr 2006 03:52:48 -0000 Hi; FWIW, if my memory serves well... files used to be overwritten three times to specifically avoid them from being recovered (security/privacy concerns).. according to the rm(1) manpage this is done by the -P option, perhaps it's not the default anymore. cheers, Pedro. ___________________________________ Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB http://mail.yahoo.it From owner-freebsd-fs@FreeBSD.ORG Wed Apr 12 10:43:50 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9A53A16A403 for ; Wed, 12 Apr 2006 10:43:50 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id A5A3543D48 for ; Wed, 12 Apr 2006 10:43:49 +0000 (GMT) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id 62B6D2096; Wed, 12 Apr 2006 12:43:44 +0200 (CEST) X-Spam-Tests: AWL,BAYES_00,FORGED_RCVD_HELO X-Spam-Learn: ham X-Spam-Score: -2.4/3.0 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on tim.des.no Received: from xps.des.no (des.no [80.203.243.180]) by tim.des.no (Postfix) with ESMTP id 4A0252081; Wed, 12 Apr 2006 12:43:44 +0200 (CEST) Received: by xps.des.no (Postfix, from userid 1001) id 1CBA033C36; Wed, 12 Apr 2006 12:43:44 +0200 (CEST) From: des@des.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) To: pfgshield-freebsd@yahoo.com References: <20060412035246.20426.qmail@web32703.mail.mud.yahoo.com> Date: Wed, 12 Apr 2006 12:43:43 +0200 In-Reply-To: <20060412035246.20426.qmail@web32703.mail.mud.yahoo.com> (pfgshield-freebsd@yahoo.com's message of "Wed, 12 Apr 2006 05:52:46 +0200 (CEST)") Message-ID: <86sloj9iqo.fsf@xps.des.no> User-Agent: Gnus/5.110003 (No Gnus v0.3) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Apr 2006 10:43:50 -0000 writes: > FWIW, if my memory serves well... files used to be overwritten three time= s to > specifically avoid them from being recovered (security/privacy concerns). That was never the case for UFS. DES --=20 Dag-Erling Sm=F8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Wed Apr 12 13:32:26 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 436F716A403 for ; Wed, 12 Apr 2006 13:32:26 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 335F543D62 for ; Wed, 12 Apr 2006 13:32:18 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (enmbcn@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k3CDWC4b086257 for ; Wed, 12 Apr 2006 15:32:17 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k3CDWCtp086256; Wed, 12 Apr 2006 15:32:12 +0200 (CEST) (envelope-from olli) Date: Wed, 12 Apr 2006 15:32:12 +0200 (CEST) Message-Id: <200604121332.k3CDWCtp086256@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG In-Reply-To: <200603241646.58415.mi+mx@aldan.algebra.com> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.0-20051224 ("Ronay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Wed, 12 Apr 2006 15:32:17 +0200 (CEST) Cc: Subject: Re: How 'honest' is fstat(1)? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-fs@FreeBSD.ORG List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Apr 2006 13:32:26 -0000 Mikhail Teterin wrote: > As a policy I try to keep my /, /usr, and other filesystems (except /var > and /home) mounted readonly. (Not so much for security as for safety.) That's not a bad idea, basically. > When I need to make a modification, I remount them: > > mount -orw -u / > > make the change, and remount back: > > mount -oro -u /usr/local Note that you will lose any additional options that way, such as "nosuid". To retain the current options, use a command like this: mount -u -o current,ro /usr/local Or to use the options from your /etc/fstab: mount -u -o fstab,ro /usr/local > This works for "small" changes, but sometimes, however, after a bigger on > (such as rebuilding of some ports), the last step fails with "busy". Maybe there are still things in the softupdates backlog. Those don't appear in fstat(1) output because they're not associated with a particular process or descriptor. Is the FS still busy after waiting a few minutes? However, I _think_ the unmount() should just block in that case until all softupdates changes are flushed to disk, not report EBUSY. But I'm not 100% sure. > At this time nothing should have a file open, and nothing does according to > fstat. The command: > > fstat | awk '$5 == "/usr/local" && $NF != "r"' > > does not list anything. Does it list anything when you omit the check for $NF!="r"? Did you try to use lsof(8) instead (from ports collection)? > My only guess is, the earlier versions of the just reinstalled executables are > still running and that trigger's the rarely noticed bug. It would be a bug, because running executables should not be open for writing. In the past, the mount update code (-u flag) had bugs which could even lead to FS corruption when you switched back and forth between ro and rw multiple times. But I believe those bugs were fixed. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "If you aim the gun at your foot and pull the trigger, it's UNIX's job to ensure reliable delivery of the bullet to where you aimed the gun (in this case, Mr. Foot)." -- Terry Lambert, FreeBSD-hackers mailing list. From owner-freebsd-fs@FreeBSD.ORG Wed Apr 12 13:40:56 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4A77116A404 for ; Wed, 12 Apr 2006 13:40:56 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3527F43D7B for ; Wed, 12 Apr 2006 13:40:49 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (zmjktu@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k3CDehJ3087110 for ; Wed, 12 Apr 2006 15:40:49 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k3CDehYV087109; Wed, 12 Apr 2006 15:40:43 +0200 (CEST) (envelope-from olli) Date: Wed, 12 Apr 2006 15:40:43 +0200 (CEST) Message-Id: <200604121340.k3CDehYV087109@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG In-Reply-To: <443AFB03.6060301@samsco.org> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.0-20051224 ("Ronay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Wed, 12 Apr 2006 15:40:49 +0200 (CEST) Cc: Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-fs@FreeBSD.ORG List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Apr 2006 13:40:56 -0000 Scott Long wrote: > [...] IOW, there is no easy way to undelete a file. This isn't directly related to the question, but snapshots are very handy to "undelete" files. Just set up a cronjob that creates (and rotates) hourly and daily snapshots. When a user accidentally deletes a file, he can just copy it back from the latest snapshot where the file still exists. The best thing is that no admin intervention is required, no backup media have to be shuffled etc. [*] You could even make an "undelete" alias (or script) which copies files back from the snapshot. (I've seen setups where the admins replaced rm(1) with a script that moved the files to a "wastebasket" directory, which appeared to me as a gross hack. Using snapshots as described above is a much better solution and more efficient.) Best regards Oliver [*] Of course, it doesn't replace a real backup (if you lose the disk, you lose all data including all snapshots). -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "FreeBSD is Yoda, Linux is Luke Skywalker" -- Daniel C. Sobral From owner-freebsd-fs@FreeBSD.ORG Thu Apr 13 08:16:26 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DF20416A407 for ; Thu, 13 Apr 2006 08:16:26 +0000 (UTC) (envelope-from trhodes@FreeBSD.org) Received: from pittgoth.com (ns1.pittgoth.com [216.38.206.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3822643D48 for ; Thu, 13 Apr 2006 08:16:25 +0000 (GMT) (envelope-from trhodes@FreeBSD.org) Received: from localhost (net-ix.gw.ai.net [205.134.160.6] (may be forged)) (authenticated bits=0) by pittgoth.com (8.13.4/8.13.4) with ESMTP id k3D9I2ED002567 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 13 Apr 2006 05:18:03 -0400 (EDT) (envelope-from trhodes@FreeBSD.org) Date: Thu, 13 Apr 2006 04:16:18 -0400 From: Tom Rhodes To: Bruce Evans Message-Id: <20060413041618.6aa7d8c7.trhodes@FreeBSD.org> In-Reply-To: <20060411210858.G46778@delplex.bde.org> References: <1144687418.11014.9.camel@diegows> <443AFB03.6060301@samsco.org> <20060411210858.G46778@delplex.bde.org> X-Mailer: Sylpheed version 1.0.5 (GTK+ 1.2.10; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Apr 2006 08:16:27 -0000 On Tue, 11 Apr 2006 22:56:11 +1000 (EST) Bruce Evans wrote: > On Mon, 10 Apr 2006, Scott Long wrote: > > > IOW, there is no easy way to undelete a file. > > This is currently true, except in the rare case where undelete(2) works. Oh, so it has worked for someone. I always wonder why we have this functionality when I have never been able to make it work. Using rm(1) that is. -- Tom Rhodes From owner-freebsd-fs@FreeBSD.ORG Fri Apr 14 06:47:29 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 80C8D16A400; Fri, 14 Apr 2006 06:47:29 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F01843D53; Fri, 14 Apr 2006 06:47:29 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 22FB01A4D95; Thu, 13 Apr 2006 23:47:29 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 3F2F551406; Fri, 14 Apr 2006 02:47:28 -0400 (EDT) Date: Fri, 14 Apr 2006 02:47:27 -0400 From: Kris Kennaway To: Tom Rhodes Message-ID: <20060414064727.GA77608@xor.obsecurity.org> References: <1144687418.11014.9.camel@diegows> <443AFB03.6060301@samsco.org> <20060411210858.G46778@delplex.bde.org> <20060413041618.6aa7d8c7.trhodes@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="cWoXeonUoKmBZSoM" Content-Disposition: inline In-Reply-To: <20060413041618.6aa7d8c7.trhodes@FreeBSD.org> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@FreeBSD.org Subject: Re: How a file is deleted in ufs2? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Apr 2006 06:47:29 -0000 --cWoXeonUoKmBZSoM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Apr 13, 2006 at 04:16:18AM -0400, Tom Rhodes wrote: > On Tue, 11 Apr 2006 22:56:11 +1000 (EST) > Bruce Evans wrote: >=20 > > On Mon, 10 Apr 2006, Scott Long wrote: > >=20 > > > IOW, there is no easy way to undelete a file. > >=20 > > This is currently true, except in the rare case where undelete(2) works. >=20 > Oh, so it has worked for someone. I always wonder why we have this > functionality when I have never been able to make it work. Using > rm(1) that is. DESCRIPTION The undelete() system call attempts to recover the deleted file named = by path. Currently, this works only when the named object is a whiteout = in a union file system.=20 Kris --cWoXeonUoKmBZSoM Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (FreeBSD) iD8DBQFEP0V/Wry0BWjoQKURAsvfAKCfXk/gHrBvSJ5vmIOLl9ecVc6LkgCcDRQ+ 4PbG40Gee7O+MEYmzmMog2g= =IuRf -----END PGP SIGNATURE----- --cWoXeonUoKmBZSoM--