From owner-freebsd-fs@FreeBSD.ORG Sun Aug 6 03:43:01 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AD38816A4DE for ; Sun, 6 Aug 2006 03:43:01 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3596943D45 for ; Sun, 6 Aug 2006 03:43:00 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.24] (andersonbox4.centtech.com [192.168.42.24]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k763gx2u053781; Sat, 5 Aug 2006 22:42:59 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44D56555.5020107@centtech.com> Date: Sat, 05 Aug 2006 22:43:17 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.5 (X11/20060802) MIME-Version: 1.0 To: Eric Anderson References: <20060803200158.61098.qmail@web30315.mail.mud.yahoo.com> <44D3A7D1.1070406@centtech.com> In-Reply-To: <44D3A7D1.1070406@centtech.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1637/Sat Aug 5 17:37:00 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: locking questions (regarding file systems) (SOLVED) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Aug 2006 03:43:01 -0000 On 08/04/06 15:02, Eric Anderson wrote: > On 08/03/06 15:01, R. B. Riddick wrote: >> --- Eric Anderson wrote: >>> I just did, and looks like the g_vfs_close directly after the g_vfs_open >>> block (I copy-pasted) works ok. When unmounting, it then panics on >>> g_vfs_close, probably because it's already closed from before. >>> >> So the sequence: >> ... >> g_vfs_open(...); >> g_vfs_close(...); >> ... >> works fine. >> >> What r u doing between the mount and the unmount calls? >> Maybe something changes, so that ...close() deadlocks? >> >> Did u try to put debug printf's in the g_vfs_close() function? :-) > > No, I've been trying to avoid that :) > >> Or is there a program somewhere, that uses g_vfs_open/close? >> Maybe u could learn/copy&paste from hpfs? >> http://fxr.watson.org/fxr/ident?i=g_vfs_close > > I've looked at other fs's, and it's helpful, but it's a bug in my code > that is the problem. :) > > I think I know the block of code the problem is in, now I just have to > hunt it down. If I move the g_vfs_close around, I can get a panic like: > > panic: lockmgr: locking against myself Looks like I had to do this prior to the g_vfs_close: vinvalbuf(devvp, V_SAVE, td, 0, 0); That seemed to do the trick.. Thanks for all the help! Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 01:12:11 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C93C916A4DA; Tue, 8 Aug 2006 01:12:11 +0000 (UTC) (envelope-from adamsmithx@hotmail.com) Received: from bay0-omc1-s3.bay0.hotmail.com (bay0-omc1-s3.bay0.hotmail.com [65.54.246.75]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6DCF443D45; Tue, 8 Aug 2006 01:12:11 +0000 (GMT) (envelope-from adamsmithx@hotmail.com) Received: from hotmail.com ([207.46.11.117]) by bay0-omc1-s3.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 7 Aug 2006 18:10:43 -0700 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Mon, 7 Aug 2006 18:10:42 -0700 Message-ID: Received: from 207.46.11.123 by by123fd.bay123.hotmail.msn.com with HTTP; Tue, 08 Aug 2006 01:10:41 GMT X-Originating-IP: [66.93.45.209] X-Originating-Email: [adamsmithx@hotmail.com] X-Sender: adamsmithx@hotmail.com From: "Joseph Maxwell" To: freebsd-hackers@freebsd.org, freebsd-questions-request@freebsd.org, freebsd-fs@freebsd.org Date: Mon, 07 Aug 2006 18:10:41 -0700 Mime-Version: 1.0 Content-Type: text/plain; format=flowed X-OriginalArrivalTime: 08 Aug 2006 01:10:42.0874 (UTC) FILETIME=[7853B1A0:01C6BA87] Cc: Subject: NFS Problems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 01:12:11 -0000 Hope I have the right site here Hello, Indigo2 machine in network w/ FreeBSD, both configured for nfs server/client functions, and "nfsd" daemon running on both. IRIX idaho 6.5 04101930 IP22 FreeBSD presto 4.2-RELEASE FreeBSD 4.2-RELEASE #0: Mon Nov 20 13:02:55 GMT 2000 jkh@bento.FreeBSD.org:/usr/src/sys/compile/GENERIC i386 (Yes an old system inst. on an infrequently used critical and stable point in our system. Not eager to perturb.) FreeBSD machine mounts IRIX disks OK, but Indigo2 does not mount FreeBSD disks. Changed to manual approach Setup fstab: /dev/root / xfs rw,raw=/dev/rroot 0 0 /dev/dsk/dks0d2s7 /disk_02/ xfs rw 0 0 /swap/swap1 swap swap pri=2 0 0 /vswap/vswap1 swap swap pri=5,vlength=409600 0 0 /vswap/vswap2 swap swap pri=5,vlength=1024000 0 0 presto:/usr /diskA-1 nfs soft,rw 0 0 presto:/disk2 /diskA-2 nfs soft,rw 0 0 Then attempted to mount presto disks mount -h presto -v mount: presto:/disk2 server not responding: Port mapper failure - Timed out mount: NFS version 3 mount failed, trying NFS version 2 mount: retrying /diskA-2 mount: NFS version 3 mount failed, trying NFS version 2 mount: NFS version 3 mount failed, trying NFS version 2 mount: NFS version 3 mount failed, trying NFS version 2 mount: NFS version 3 mount failed, trying NFS version 2 Could someone identify problem & suggest corrective action? Thanks ---------------------------------------------------------------------------- Hello, FreeBSD machine in network w/ Indigo2 SGI machine, both configured for nfs server/client functions, and "nfsd" daemon running on both. FreeBSD presto 4.2-RELEASE FreeBSD 4.2-RELEASE #0: Mon Nov 20 13:02:55 GMT 2000 jkh@bento.FreeBSD.org:/usr/src/sys/compile/GENERIC i386 IRIX idaho 6.5 04101930 IP22 FreeBSD machine mounts IRIX disks OK, but Indigo2 does not mount FreeBSD disks. Found couple of problems w/ FreeBSD, portmap, mountd etc. don't know if I have them all corrected but SGI machine cannot mount FreeBSD disks ------------------ >From FreeBSD machine: In rc.conf ==> nfs_client_enable="YES" nfs_server_enable="YES" ------------------ >From FreeBSD machine: cat /etc/exports /usr -alldirs idaho /disk2 -alldirs idaho ------------------ >From SGI machine rpcinfo -p presto program vers proto port 100000 2 tcp 111 portmapper 100000 2 udp 111 portmapper 100005 3 udp 997 mountd 100005 3 tcp 1021 mountd 100005 1 udp 997 mountd 100005 1 tcp 1021 mountd >From FreeBSD machine rpcinfo -p idaho program vers proto port 100000 2 tcp 111 portmapper 100000 2 udp 111 portmapper 100003 2 udp 2049 nfs 100003 3 udp 2049 nfs 100003 2 tcp 2049 nfs 100003 3 tcp 2049 nfs 100024 1 udp 798 status 100024 1 tcp 800 status 100021 1 udp 2049 nlockmgr 100021 3 udp 2049 nlockmgr 100021 4 udp 2049 nlockmgr 100021 1 tcp 2049 nlockmgr 100021 3 tcp 2049 nlockmgr 100021 4 tcp 2049 nlockmgr 100005 1 tcp 1024 mountd 100005 3 tcp 1024 mountd 100005 1 udp 1026 mountd 100005 3 udp 1026 mountd 391004 1 tcp 1025 391004 1 udp 1027 100001 1 udp 1028 rstatd 100001 2 udp 1028 rstatd 100001 3 udp 1028 rstatd 100008 1 udp 1029 walld 100002 1 udp 1030 rusersd 100011 1 udp 1031 rquotad 100012 1 udp 1032 sprayd 391011 1 tcp 1026 391002 1 tcp 1027 391002 2 tcp 1027 391006 1 udp 1033 391029 1 tcp 1028 100083 1 tcp 1029 ttdbserver 391017 1 tcp 889 -------------- mutual enerties of each are in each others /etc/hosts and /etc/host.allow files >From fstab on SGI machine: ......... ......... presto:/usr /diskA-1 nfs soft,rw 0 0 presto:/disk2 /diskA-2 nfs soft,rw 0 0 ------------------ Attempts at mounting presto disks from SGI machine idaho ==> mount -h idaho -v mount: NFS version 3 mount failed, trying NFS version 2. mount: idaho:/disk2 on /diskA-2: Connection timed out mount: retrying /diskA-2 mount: unknown filesystem type: nfs mount: giving up on: /diskA-2 ------------------ Could someone suggest something else to look at, identify problem & suggest corrective action? Thanks _________________________________________________________________ On the road to retirement? Check out MSN Life Events for advice on how to get there! http://lifeevents.msn.com/category.aspx?cid=Retirement From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 01:17:55 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A0B6B16A4DF; Tue, 8 Aug 2006 01:17:55 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [210.51.165.229]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3994F43D6B; Tue, 8 Aug 2006 01:17:45 +0000 (GMT) (envelope-from delphij@delphij.net) Received: from localhost (tarsier.geekcn.org [210.51.165.229]) by tarsier.geekcn.org (Postfix) with ESMTP id A6689EB3C3B; Tue, 8 Aug 2006 09:17:43 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([210.51.165.229]) by localhost (mail.geekcn.org [210.51.165.229]) (amavisd-new, port 10024) with ESMTP id juJ8zAv+kTTp; Tue, 8 Aug 2006 09:17:41 +0800 (CST) Received: from [10.217.12.96] (sina152-194.staff.sina.com.cn [61.135.152.194]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTP id 2A31BEB267E; Tue, 8 Aug 2006 09:17:40 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=subject:from:to:cc:in-reply-to:references:content-type: organization:date:message-id:mime-version:x-mailer; b=cY0ViI/jWKe6k2sgK5r78BXlkFshYxdvan/w+FhZoOA+WkX9H9oiTtZ/ZHFNq46GP k7wVcHA7pxGav1qiaOxSQ== From: Xin LI To: Joseph Maxwell In-Reply-To: References: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-9cH1HVtFJYAyiVFlovfs" Organization: The FreeBSD Project Date: Tue, 08 Aug 2006 09:17:25 +0800 Message-Id: <1154999845.927.2.camel@spirit> Mime-Version: 1.0 X-Mailer: Evolution 2.6.3 FreeBSD GNOME Team Port Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, freebsd-questions-request@freebsd.org Subject: Re: NFS Problems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 01:17:55 -0000 --=-9cH1HVtFJYAyiVFlovfs Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, =E5=9C=A8 2006-08-07=E4=B8=80=E7=9A=84 18:10 -0700=EF=BC=8CJoseph Maxwell= =E5=86=99=E9=81=93=EF=BC=9A > >From FreeBSD machine: > In rc.conf =3D=3D> > nfs_client_enable=3D"YES" > nfs_server_enable=3D"YES" > ------------------ [...] > >From fstab on SGI machine: > ......... > ......... > presto:/usr /diskA-1 nfs soft,rw 0 0 > presto:/disk2 /diskA-2 nfs soft,rw 0 0 > ------------------ > Attempts at mounting presto disks from SGI machine idaho =3D=3D> > mount -h idaho -v > mount: NFS version 3 mount failed, trying NFS version 2. > mount: idaho:/disk2 on /diskA-2: Connection timed out > mount: retrying > /diskA-2 > mount: unknown filesystem type: nfs > mount: giving up on: > /diskA-2 > ------------------ >=20 > Could someone suggest something else to look at, identify problem & sugge= st=20 > corrective action? Have you tried some other combinations, for instance, using TCP mounts? Sometimes that would be a good workaround for NFS mounting issues... Cheers, --=20 Xin LI http://www.delphij.net/ --=-9cH1HVtFJYAyiVFlovfs Content-Type: application/pgp-signature; name=signature.asc Content-Description: =?UTF-8?Q?=E8=BF=99=E6=98=AF=E4=BF=A1=E4=BB=B6=E7=9A=84=E6=95=B0?= =?UTF-8?Q?=E5=AD=97=E7=AD=BE=E5=90=8D=E9=83=A8=E5=88=86?= -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQBE1+YlhcUczkLqiksRAnt1AKCfOuIA1knHIQLSfp3vMeahBeGSvgCgvgPQ pgOXjXRO1sOF8F4sSoybDKI= =O94k -----END PGP SIGNATURE----- --=-9cH1HVtFJYAyiVFlovfs-- From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 05:14:19 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 43BC516A4DD for ; Tue, 8 Aug 2006 05:14:19 +0000 (UTC) (envelope-from qdolan@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.179]) by mx1.FreeBSD.org (Postfix) with ESMTP id A6F0B43D46 for ; Tue, 8 Aug 2006 05:14:18 +0000 (GMT) (envelope-from qdolan@gmail.com) Received: by py-out-1112.google.com with SMTP id c59so19680pyc for ; Mon, 07 Aug 2006 22:14:18 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:mime-version:in-reply-to:references:content-type:message-id:from:subject:date:to:x-mailer; b=X0nv8hxjus08eQYHwR5d74mUAlDHGGGvJwsUDKmgxcnqqXPmtkklAWy46qalnqcWNABbv7pqO05zUFFnzn13M+uRij8IU/LoS0gnHv7bMWFpqi7EhtbiuyQEniJ1WYdfBTpZIRdu3w7jPXEMc2OdsPFDablBwGgnCaehF5N8VA4= Received: by 10.35.51.13 with SMTP id d13mr13872116pyk; Mon, 07 Aug 2006 22:14:17 -0700 (PDT) Received: from ?172.22.1.30? ( [203.13.70.60]) by mx.gmail.com with ESMTP id 10sm151342nzo.2006.08.07.22.14.14; Mon, 07 Aug 2006 22:14:16 -0700 (PDT) Mime-Version: 1.0 (Apple Message framework v752.2) In-Reply-To: References: Message-Id: <06530464-7491-4BC1-97C7-87285B03F2EB@gmail.com> From: Q Date: Tue, 8 Aug 2006 15:14:10 +1000 To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Problem with ufs not releasing vm_pages on busy volume. (soft updates related) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 05:14:19 -0000 On 02/08/2006, at 8:10 PM, Q wrote: > I have a problem that seems to be related to something ufs related > not releasing some vm_pages on busy filesystems. I have two servers > running PostgreSQL, one running 6.0-RELEASE, the other 6.1-RELEASE. > Both are under the same (fairly heavy) load, performing the same > operations in bursts every five minutes. The filesystems in > question are 450-500Gig, each server using a different brand of > RAID card, they both have soft-updates enabled. > > The problem is that both servers are seeing an accumulation of > about 100Mb of active pages per day (looking at > vm.stats.vm.v_active_count) that never get released. The only way > to release these pages is to unmount the filesystem and remount it. > Failing to do this results in the server eventually locking up. > > If someone could provide me with some direction on how to go about > tracking down what might be causing this to happen it would be much > appreciated. I have narrowed the cause of this issue down further to something to do with soft updates. If I turn off soft updates for the filesystem hosting the database the system no longer accumulates active vm_pages constantly. Instead for accumulating 100Mb a day of active vm pages until all memory is consumed, it will hover around 50-60Mb with soft updates disabled. If someone familiar with the softupdates code is willing to help me pinpoint the cause of this problem it would be much appreciated. -- Seeya...Q -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- _____ / Quinton Dolan - qdolan@gmail.com __ __/ / / __/ / / / __ / _/ / / Gold Coast, QLD, Australia __/ __/ __/ ____/ / - / Ph: +61 419 729 806 _______ / _\ From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 11:24:35 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0455F16A4E5; Tue, 8 Aug 2006 11:24:35 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5037143D55; Tue, 8 Aug 2006 11:24:33 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k78BOWPE094225; Tue, 8 Aug 2006 06:24:33 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44D87483.8090308@centtech.com> Date: Tue, 08 Aug 2006 06:24:51 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.5 (X11/20060802) MIME-Version: 1.0 To: Joseph Maxwell References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1640/Mon Aug 7 20:11:04 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, freebsd-questions-request@freebsd.org Subject: Re: NFS Problems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 11:24:35 -0000 On 08/07/06 20:10, Joseph Maxwell wrote: > Hope I have the right site here > > Hello, > > Indigo2 machine in network w/ FreeBSD, both configured for nfs server/client > functions, and "nfsd" daemon running on both. > > IRIX idaho 6.5 04101930 IP22 > > FreeBSD presto 4.2-RELEASE FreeBSD 4.2-RELEASE #0: Mon Nov 20 13:02:55 GMT > 2000 jkh@bento.FreeBSD.org:/usr/src/sys/compile/GENERIC i386 > (Yes an old system inst. on an infrequently used critical and stable point > in our system. Not eager to perturb.) > > FreeBSD machine mounts IRIX disks OK, but Indigo2 does not mount FreeBSD > disks. > > Changed to manual approach > > Setup fstab: > /dev/root / xfs rw,raw=/dev/rroot 0 0 > /dev/dsk/dks0d2s7 /disk_02/ xfs rw 0 0 > /swap/swap1 swap swap pri=2 0 0 > /vswap/vswap1 swap swap pri=5,vlength=409600 0 0 > /vswap/vswap2 swap swap pri=5,vlength=1024000 0 0 > presto:/usr /diskA-1 nfs soft,rw 0 0 > presto:/disk2 /diskA-2 nfs soft,rw 0 0 > > Then attempted to mount presto disks > mount -h presto -v > mount: presto:/disk2 server not responding: Port mapper failure - Timed out > mount: NFS version 3 mount failed, trying NFS version 2 > mount: retrying > /diskA-2 > mount: NFS version 3 mount failed, trying NFS version 2 > mount: NFS version 3 mount failed, trying NFS version 2 > mount: NFS version 3 mount failed, trying NFS version 2 > mount: NFS version 3 mount failed, trying NFS version 2 > > Could someone identify problem & suggest corrective action? [..snip..] >>From SGI machine > rpcinfo -p presto > program vers proto port > 100000 2 tcp 111 portmapper > 100000 2 udp 111 portmapper > 100005 3 udp 997 mountd > 100005 3 tcp 1021 mountd > 100005 1 udp 997 mountd > 100005 1 tcp 1021 mountd I think the problem is that you are missing the nfs registrations (and possibly rstatd and nlockmgr too if you need them). I think you can do: nfsd -r to re-register the entries in rpc. You should check the process list on presto and see if nfsd is running, and also check for rstatd, and lockd. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 14:32:50 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DC9B116A4E0 for ; Tue, 8 Aug 2006 14:32:50 +0000 (UTC) (envelope-from rick@snowhite.cis.uoguelph.ca) Received: from mailhub.cs.uoguelph.ca (mailhub.cs.uoguelph.ca [131.104.94.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3695F43D5F for ; Tue, 8 Aug 2006 14:32:49 +0000 (GMT) (envelope-from rick@snowhite.cis.uoguelph.ca) Received: from snowhite.cis.uoguelph.ca (snowhite.cis.uoguelph.ca [131.104.48.1]) by mailhub.cs.uoguelph.ca (8.13.1/8.13.1) with ESMTP id k78EWmJX009640; Tue, 8 Aug 2006 10:32:48 -0400 Received: (from rick@localhost) by snowhite.cis.uoguelph.ca (8.9.3/8.9.3) id KAA01588; Tue, 8 Aug 2006 10:33:44 -0400 (EDT) Date: Tue, 8 Aug 2006 10:33:44 -0400 (EDT) From: rick@snowhite.cis.uoguelph.ca Message-Id: <200608081433.KAA01588@snowhite.cis.uoguelph.ca> To: adamsmithx@hotmail.com X-Scanned-By: MIMEDefang 2.52 on 131.104.94.205 Cc: freebsd-fs@freebsd.org Subject: NFS Problems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 14:32:50 -0000 > FreeBSD machine mounts IRIX disks OK, but Indigo2 does not mount FreeBSD > disks. If re-registering doesn't make it work, try a mount forced to nfsv2. (mount -t nfs -o -2 blah blah) Irix's NFSv3 was notoriously buggy in the old days and I have no idea if 6.5 is in that category. rick From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 16:07:33 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0716916A4E0 for ; Tue, 8 Aug 2006 16:07:33 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8057443D46 for ; Tue, 8 Aug 2006 16:07:32 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k78G7VxX039472; Tue, 8 Aug 2006 11:07:31 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44D8B6D6.6070209@centtech.com> Date: Tue, 08 Aug 2006 11:07:50 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.5 (X11/20060802) MIME-Version: 1.0 To: Q References: <06530464-7491-4BC1-97C7-87285B03F2EB@gmail.com> In-Reply-To: <06530464-7491-4BC1-97C7-87285B03F2EB@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1640/Mon Aug 7 20:11:04 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: Problem with ufs not releasing vm_pages on busy volume. (soft updates related) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 16:07:33 -0000 On 08/08/06 00:14, Q wrote: > On 02/08/2006, at 8:10 PM, Q wrote: > >> I have a problem that seems to be related to something ufs related >> not releasing some vm_pages on busy filesystems. I have two servers >> running PostgreSQL, one running 6.0-RELEASE, the other 6.1-RELEASE. >> Both are under the same (fairly heavy) load, performing the same >> operations in bursts every five minutes. The filesystems in >> question are 450-500Gig, each server using a different brand of >> RAID card, they both have soft-updates enabled. >> >> The problem is that both servers are seeing an accumulation of >> about 100Mb of active pages per day (looking at >> vm.stats.vm.v_active_count) that never get released. The only way >> to release these pages is to unmount the filesystem and remount it. >> Failing to do this results in the server eventually locking up. >> >> If someone could provide me with some direction on how to go about >> tracking down what might be causing this to happen it would be much >> appreciated. > > I have narrowed the cause of this issue down further to something to > do with soft updates. If I turn off soft updates for the filesystem > hosting the database the system no longer accumulates active vm_pages > constantly. Instead for accumulating 100Mb a day of active vm pages > until all memory is consumed, it will hover around 50-60Mb with soft > updates disabled. > > If someone familiar with the softupdates code is willing to help me > pinpoint the cause of this problem it would be much appreciated. > Is it possible for you to upgrade to the latest 6-STABLE branch, just to make sure that the issue hasn't been fixed already? Is there any way to reproduce this on another box for testing? (I assume not, due to the nature of these things) Also - I wonder if doing a snapshot on the filesystem would flush out the pages - is that something you can try? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 17:33:57 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0433716A4DA for ; Tue, 8 Aug 2006 17:33:57 +0000 (UTC) (envelope-from adamsch1@yahoo.com) Received: from web31803.mail.mud.yahoo.com (web31803.mail.mud.yahoo.com [68.142.207.66]) by mx1.FreeBSD.org (Postfix) with SMTP id 9B9AD43D46 for ; Tue, 8 Aug 2006 17:33:56 +0000 (GMT) (envelope-from adamsch1@yahoo.com) Received: (qmail 93511 invoked by uid 60001); 8 Aug 2006 17:33:56 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=dfMH/rMh6edxFT6uwEBTyzirV23bDjuySrqBXGtgzKuC1sU8zxvy8XRjAnzTh5dBBOvEBEr7cHupRzbzdxT9a37HF4BVWGtR5KOllk4UEOt9TCBCTnXxQf1ipvSZ+X0zHrRhhlTsP9VbaaZ3Zdop8Qm2TmUK90kvGs4y6XphPLo= ; Message-ID: <20060808173356.93509.qmail@web31803.mail.mud.yahoo.com> Received: from [207.126.230.225] by web31803.mail.mud.yahoo.com via HTTP; Tue, 08 Aug 2006 10:33:56 PDT Date: Tue, 8 Aug 2006 10:33:56 -0700 (PDT) From: Shane Adams To: Eric Anderson , Q In-Reply-To: <44D8B6D6.6070209@centtech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-fs@freebsd.org Subject: Re: Problem with ufs not releasing vm_pages on busy volume. (soft updates related) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Shane Adams List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 17:33:57 -0000 Perhaps its just me but I found taking a snapshot on a 80 gig drive that is relativly idle, frooze my system for several seconds (to the point I was about to give it the finger). I figured something is broken on my end cause I couldn't imagine anyone using the feature if this is how it works normally. Just curious if anyone would share their experiences with snapshotting in a production envioronment? Shane ----- Original Message ---- From: Eric Anderson To: Q Cc: freebsd-fs@freebsd.org Sent: Tuesday, August 8, 2006 9:07:50 AM Subject: Re: Problem with ufs not releasing vm_pages on busy volume. (soft updates related) On 08/08/06 00:14, Q wrote: > On 02/08/2006, at 8:10 PM, Q wrote: > >> I have a problem that seems to be related to something ufs related >> not releasing some vm_pages on busy filesystems. I have two servers >> running PostgreSQL, one running 6.0-RELEASE, the other 6.1-RELEASE. >> Both are under the same (fairly heavy) load, performing the same >> operations in bursts every five minutes. The filesystems in >> question are 450-500Gig, each server using a different brand of >> RAID card, they both have soft-updates enabled. >> >> The problem is that both servers are seeing an accumulation of >> about 100Mb of active pages per day (looking at >> vm.stats.vm.v_active_count) that never get released. The only way >> to release these pages is to unmount the filesystem and remount it. >> Failing to do this results in the server eventually locking up. >> >> If someone could provide me with some direction on how to go about >> tracking down what might be causing this to happen it would be much >> appreciated. > > I have narrowed the cause of this issue down further to something to > do with soft updates. If I turn off soft updates for the filesystem > hosting the database the system no longer accumulates active vm_pages > constantly. Instead for accumulating 100Mb a day of active vm pages > until all memory is consumed, it will hover around 50-60Mb with soft > updates disabled. > > If someone familiar with the softupdates code is willing to help me > pinpoint the cause of this problem it would be much appreciated. > Is it possible for you to upgrade to the latest 6-STABLE branch, just to make sure that the issue hasn't been fixed already? Is there any way to reproduce this on another box for testing? (I assume not, due to the nature of these things) Also - I wonder if doing a snapshot on the filesystem would flush out the pages - is that something you can try? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ _______________________________________________ freebsd-fs@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 17:52:36 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6F88A16A4DF for ; Tue, 8 Aug 2006 17:52:36 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0467443D55 for ; Tue, 8 Aug 2006 17:52:35 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k78HqYR1056367; Tue, 8 Aug 2006 12:52:34 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44D8CF75.60003@centtech.com> Date: Tue, 08 Aug 2006 12:52:53 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.5 (X11/20060802) MIME-Version: 1.0 To: Shane Adams References: <20060808173356.93509.qmail@web31803.mail.mud.yahoo.com> In-Reply-To: <20060808173356.93509.qmail@web31803.mail.mud.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1640/Mon Aug 7 20:11:04 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: Problem with ufs not releasing vm_pages on busy volume. (soft updates related) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 17:52:36 -0000 On 08/08/06 12:33, Shane Adams wrote: > Perhaps its just me but I found taking a snapshot on a 80 gig drive that is relativly idle, frooze my system for several seconds (to the point I was about to give it the finger). I figured something is broken on my end cause I couldn't imagine anyone using the feature if this is how it works normally. > > Just curious if anyone would share their experiences with snapshotting in a production envioronment? Unfortunately, that is how the snapshots work. Larger filesystems take longer. The more inodes that are used, the longer it takes. If you end up stat'ing the directory that contains the snapshot, then it will block, and so will its parent, probably the /, which will stop everything. You could try burying the snapshot down a few more levels, and see if you hang then too - I suppose you will. Eric > ----- Original Message ---- > From: Eric Anderson > To: Q > Cc: freebsd-fs@freebsd.org > Sent: Tuesday, August 8, 2006 9:07:50 AM > Subject: Re: Problem with ufs not releasing vm_pages on busy volume. (soft updates related) > > On 08/08/06 00:14, Q wrote: >> On 02/08/2006, at 8:10 PM, Q wrote: >> >>> I have a problem that seems to be related to something ufs related >>> not releasing some vm_pages on busy filesystems. I have two servers >>> running PostgreSQL, one running 6.0-RELEASE, the other 6.1-RELEASE. >>> Both are under the same (fairly heavy) load, performing the same >>> operations in bursts every five minutes. The filesystems in >>> question are 450-500Gig, each server using a different brand of >>> RAID card, they both have soft-updates enabled. >>> >>> The problem is that both servers are seeing an accumulation of >>> about 100Mb of active pages per day (looking at >>> vm.stats.vm.v_active_count) that never get released. The only way >>> to release these pages is to unmount the filesystem and remount it. >>> Failing to do this results in the server eventually locking up. >>> >>> If someone could provide me with some direction on how to go about >>> tracking down what might be causing this to happen it would be much >>> appreciated. >> I have narrowed the cause of this issue down further to something to >> do with soft updates. If I turn off soft updates for the filesystem >> hosting the database the system no longer accumulates active vm_pages >> constantly. Instead for accumulating 100Mb a day of active vm pages >> until all memory is consumed, it will hover around 50-60Mb with soft >> updates disabled. >> >> If someone familiar with the softupdates code is willing to help me >> pinpoint the cause of this problem it would be much appreciated. >> > > > Is it possible for you to upgrade to the latest 6-STABLE branch, just to > make sure that the issue hasn't been fixed already? > > Is there any way to reproduce this on another box for testing? (I > assume not, due to the nature of these things) > > Also - I wonder if doing a snapshot on the filesystem would flush out > the pages - is that something you can try? > > Eric > > -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 19:52:56 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B78EE16A4DA; Tue, 8 Aug 2006 19:52:56 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id D456843D45; Tue, 8 Aug 2006 19:52:55 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 312EB51393; Tue, 8 Aug 2006 21:52:53 +0200 (CEST) Received: from localhost (dlx245.neoplus.adsl.tpnet.pl [83.24.53.245]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id E083D50E96; Tue, 8 Aug 2006 21:52:47 +0200 (CEST) Date: Tue, 8 Aug 2006 21:52:02 +0200 From: Pawel Jakub Dawidek To: freebsd-fs@FreeBSD.org Message-ID: <20060808195202.GA1564@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2oS5YaxWCcQjTEyO" Content-Disposition: inline X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 19:52:56 -0000 --2oS5YaxWCcQjTEyO Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. I'm pleased to announce hopefully the final gjournal patches. The biggest change is removal of magic .deleted/ directory. Now orphaned objects (files/directories which were deleted, but are still open) are handled in a different way - in every CG there is a counter which tells how many such orphaned objects are there in the given CG. There is also a global unreferenced inodes counter in the superblock. On power failure or system crash, fast fsck version will be run on journaled file system which only checks CG for unreferenced inodes. Multi terabyte file system is checked in few minutes, but it depends on how many such objects we have. The global counter in the superblock allows to stop scanning CGs when it goes to 0. Another change is that now from a configuration point of view gjournal is threated simlar to soft-updates - you need to specify -J flag at newfs(8) time or use 'tunefs -J enable '. 'gjournal' is no longer a mount option. To create journaled UFS file system on ad0s1d partition: # gjournal load # gjournal label ad0s1d # newfs -J /dev/ad0s1d.journal # mount -o noatime,async /dev/ad0s1d.journal /mnt On reboot 'fsck_ffs -p /dev/ad0s1d.journal' will perform fast check. One can still run regular check by not giving the -p option. BTW. This should allow to have root file system on gjournal partition, but haven't verified it yet - just a theory. Patches for 7-CURRENT: http://people.freebsd.org/~pjd/patches/gjournal_20060808.patch Patches for 6-STABLE: http://people.freebsd.org/~pjd/patches/gjournal6_20060808.patch To apply the patches: # cd /usr/src # mkdir sys/geom/journal sbin/geom/class/journal sys/modules/geom/geom_jou= rnal # patch < gjournal_20060808.patch Now you need to recompile the world and the kernel. Remember to add 'options UFS_GJOURNAL' to your kernel configuration file. If there will be no bug reports nor strong objections, I'd like commit gjournal in a week or two first to 7-CURRENT and then to 6-STABLE (maybe before 6.2-RELEASE, but it depends on testing and feedback from users). Enjoy! --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --2oS5YaxWCcQjTEyO Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE2OtiForvXbEpPzQRAjQYAKD2fc0LlYXB+tU9byaHGUZeX9ylGACghiYv KgXkFP1OeBhEaeAiwAXGt9o= =UosT -----END PGP SIGNATURE----- --2oS5YaxWCcQjTEyO-- From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 21:02:42 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 98C5A16A4DA for ; Tue, 8 Aug 2006 21:02:42 +0000 (UTC) (envelope-from markus@trippelsdorf.de) Received: from blue-ld-038.synserver.de (smtp-out-42.synserver.de [217.119.50.42]) by mx1.FreeBSD.org (Postfix) with SMTP id C924143D8F for ; Tue, 8 Aug 2006 21:02:20 +0000 (GMT) (envelope-from markus@trippelsdorf.de) Received: (qmail 25454 invoked by uid 0); 8 Aug 2006 21:02:15 -0000 X-SynServer-RemoteDnsName: port-212-202-34-169.dynamic.qsc.de X-SynServer-AuthUser: markus@trippelsdorf.de Received: from port-212-202-34-169.dynamic.qsc.de (HELO bsd.trippelsdorf.de) (212.202.34.169) by mx-06.synserver.de with SMTP; 8 Aug 2006 21:02:15 -0000 Date: Tue, 8 Aug 2006 23:02:11 +0200 From: Markus Trippelsdorf To: Pawel Jakub Dawidek Message-ID: <20060808210211.GA730@bsd.trippelsdorf.de> References: <20060808195202.GA1564@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060808195202.GA1564@garage.freebsd.pl> Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 21:02:42 -0000 On Tue, Aug 08, 2006 at 09:52:02PM +0200, Pawel Jakub Dawidek wrote: > > I'm pleased to announce hopefully the final gjournal patches. ... > > Patches for 6-STABLE: > > http://people.freebsd.org/~pjd/patches/gjournal6_20060808.patch > > To apply the patches: > > # cd /usr/src > # mkdir sys/geom/journal sbin/geom/class/journal sys/modules/geom/geom_journal > # patch < gjournal_20060808.patch > > Now you need to recompile the world and the kernel. Remember to add > 'options UFS_GJOURNAL' to your kernel configuration file. It does not compile with -Werror on my AMD64 system running STABLE: cc -O2 -pipe -march=k8 -DRESCUE -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wcast-align -Wunused-parameter -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -c /usr/src/bin/test/test.c /usr/src/sbin/fsck_ffs/gjournal.c: In function `gjournal_check': /usr/src/sbin/fsck_ffs/gjournal.c:717: warning: dereferencing type-punned pointer will break strict-aliasing rules *** Error code 1 -- Markus From owner-freebsd-fs@FreeBSD.ORG Tue Aug 8 21:41:25 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C20E116A4DA for ; Tue, 8 Aug 2006 21:41:25 +0000 (UTC) (envelope-from markus@trippelsdorf.de) Received: from blue-ld-038.synserver.de (smtp-out-42.synserver.de [217.119.50.42]) by mx1.FreeBSD.org (Postfix) with SMTP id AA2C743D77 for ; Tue, 8 Aug 2006 21:41:19 +0000 (GMT) (envelope-from markus@trippelsdorf.de) Received: (qmail 6259 invoked by uid 0); 8 Aug 2006 21:41:17 -0000 X-SynServer-RemoteDnsName: port-212-202-34-169.dynamic.qsc.de X-SynServer-AuthUser: markus@trippelsdorf.de Received: from port-212-202-34-169.dynamic.qsc.de (HELO bsd.trippelsdorf.de) (212.202.34.169) by mx-06.synserver.de with SMTP; 8 Aug 2006 21:41:17 -0000 Date: Tue, 8 Aug 2006 23:41:16 +0200 From: Markus Trippelsdorf To: Pawel Jakub Dawidek Message-ID: <20060808214116.GB730@bsd.trippelsdorf.de> References: <20060808195202.GA1564@garage.freebsd.pl> <20060808210211.GA730@bsd.trippelsdorf.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060808210211.GA730@bsd.trippelsdorf.de> Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 21:41:25 -0000 On Tue, Aug 08, 2006 at 11:02:11PM +0200, Markus Trippelsdorf wrote: > On Tue, Aug 08, 2006 at 09:52:02PM +0200, Pawel Jakub Dawidek wrote: > > > > I'm pleased to announce hopefully the final gjournal patches. > ... > > > > Patches for 6-STABLE: > > > > http://people.freebsd.org/~pjd/patches/gjournal6_20060808.patch > > > > To apply the patches: > > > > # cd /usr/src > > # mkdir sys/geom/journal sbin/geom/class/journal sys/modules/geom/geom_journal > > # patch < gjournal_20060808.patch > > > > Now you need to recompile the world and the kernel. Remember to add > > 'options UFS_GJOURNAL' to your kernel configuration file. > > It does not compile with -Werror on my AMD64 system running STABLE: > > cc -O2 -pipe -march=k8 -DRESCUE -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wcast-align -Wunused-parameter -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -c /usr/src/bin/test/test.c > /usr/src/sbin/fsck_ffs/gjournal.c: In function `gjournal_check': > /usr/src/sbin/fsck_ffs/gjournal.c:717: warning: dereferencing type-punned pointer will break strict-aliasing rules > *** Error code 1 Adding -fno-strict-aliasing to the CFLAGS of sbin/fsck_ffs/Makefile solved the problem for now. -- Markus From owner-freebsd-fs@FreeBSD.ORG Wed Aug 9 00:43:28 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1821016A4E7; Wed, 9 Aug 2006 00:43:28 +0000 (UTC) (envelope-from doconnor@gsoft.com.au) Received: from cain.gsoft.com.au (cain.gsoft.com.au [203.31.81.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3682443D53; Wed, 9 Aug 2006 00:43:26 +0000 (GMT) (envelope-from doconnor@gsoft.com.au) Received: from inchoate.gsoft.com.au (ppp226-175.lns2.adl4.internode.on.net [203.122.226.175]) (authenticated bits=0) by cain.gsoft.com.au (8.13.5/8.13.4) with ESMTP id k790hPWn019650 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 9 Aug 2006 10:13:25 +0930 (CST) (envelope-from doconnor@gsoft.com.au) From: "Daniel O'Connor" To: freebsd-current@freebsd.org Date: Wed, 9 Aug 2006 10:13:24 +0930 User-Agent: KMail/1.9.3 References: <20060808195202.GA1564@garage.freebsd.pl> In-Reply-To: <20060808195202.GA1564@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart15824597.KjTrxe88hv"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200608091013.25981.doconnor@gsoft.com.au> X-Spam-Score: 0 () X-Scanned-By: MIMEDefang 2.56 on 203.31.81.10 Cc: freebsd-fs@freebsd.org, Pawel Jakub Dawidek , freebsd-geom@freebsd.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2006 00:43:28 -0000 --nextPart15824597.KjTrxe88hv Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Wednesday 09 August 2006 05:22, Pawel Jakub Dawidek wrote: > I'm pleased to announce hopefully the final gjournal patches. This has diffs which remove stge presumably because you did your diff at=20 exactly the wrong time :) (The mods to CAM/ATA/etc are part of the patch for flushing cache I am=20 guessing) It also appears to remove 'MNT_JAILDEVFS" because it uses the same bitmask,= I=20 am guessing that's a mistake. =2D-=20 Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C --nextPart15824597.KjTrxe88hv Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (FreeBSD) iD8DBQBE2S+t5ZPcIHs/zowRAkX5AJ9pIi4YmcKa79I3XOs4ZzVX+UGCAACeNiyt ZXtTLLcegk6O7FxfJ+Hk0Ag= =Pb0b -----END PGP SIGNATURE----- --nextPart15824597.KjTrxe88hv-- From owner-freebsd-fs@FreeBSD.ORG Wed Aug 9 00:44:58 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E313116A4E0 for ; Wed, 9 Aug 2006 00:44:57 +0000 (UTC) (envelope-from qdolan@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.180]) by mx1.FreeBSD.org (Postfix) with ESMTP id C1B3843D62 for ; Wed, 9 Aug 2006 00:44:37 +0000 (GMT) (envelope-from qdolan@gmail.com) Received: by py-out-1112.google.com with SMTP id w49so93370pyg for ; Tue, 08 Aug 2006 17:44:36 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:in-reply-to:references:mime-version:content-type:message-id:cc:content-transfer-encoding:from:subject:date:to:x-mailer; b=PQT6XyAZIGFB+Iis08KZ7O5pwu9qk2gLogxMB35mQi30O31WfRbMfNotLY0tXOQTzG4gUVH7lmAQeSdfsWNdqkICCfxevfQ7arm9WRY0SGkeswiavbksNx9YyjtgsoYb2a2SYzqyAM+y08evx/PfTRMNrSe7k3MGaw9otrACKrQ= Received: by 10.35.126.7 with SMTP id d7mr454452pyn; Tue, 08 Aug 2006 17:44:36 -0700 (PDT) Received: from ?172.22.1.30? ( [203.13.70.60]) by mx.gmail.com with ESMTP id 12sm529840nzn.2006.08.08.17.44.35; Tue, 08 Aug 2006 17:44:36 -0700 (PDT) In-Reply-To: <44D8B6D6.6070209@centtech.com> References: <06530464-7491-4BC1-97C7-87285B03F2EB@gmail.com> <44D8B6D6.6070209@centtech.com> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Q Date: Wed, 9 Aug 2006 10:44:31 +1000 To: Eric Anderson X-Mailer: Apple Mail (2.752.2) Cc: freebsd-fs@freebsd.org Subject: Re: Problem with ufs not releasing vm_pages on busy volume. (soft updates related) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2006 00:44:58 -0000 On 09/08/2006, at 2:07 AM, Eric Anderson wrote: > On 08/08/06 00:14, Q wrote: >> On 02/08/2006, at 8:10 PM, Q wrote: >>> I have a problem that seems to be related to something ufs >>> related not releasing some vm_pages on busy filesystems. I have >>> two servers running PostgreSQL, one running 6.0-RELEASE, the >>> other 6.1-RELEASE. Both are under the same (fairly heavy) load, >>> performing the same operations in bursts every five minutes. The >>> filesystems in question are 450-500Gig, each server using a >>> different brand of RAID card, they both have soft-updates enabled. >>> >>> The problem is that both servers are seeing an accumulation of >>> about 100Mb of active pages per day (looking at >>> vm.stats.vm.v_active_count) that never get released. The only >>> way to release these pages is to unmount the filesystem and >>> remount it. Failing to do this results in the server eventually >>> locking up. >>> >>> If someone could provide me with some direction on how to go >>> about tracking down what might be causing this to happen it >>> would be much appreciated. >> I have narrowed the cause of this issue down further to something >> to do with soft updates. If I turn off soft updates for the >> filesystem hosting the database the system no longer accumulates >> active vm_pages constantly. Instead for accumulating 100Mb a day >> of active vm pages until all memory is consumed, it will hover >> around 50-60Mb with soft updates disabled. >> If someone familiar with the softupdates code is willing to help >> me pinpoint the cause of this problem it would be much appreciated. > > > Is it possible for you to upgrade to the latest 6-STABLE branch, > just to make sure that the issue hasn't been fixed already? I did a buildworld on one the machine running 6.0-RELEASE last night. I just have to schedule some downtime to do the upgrade sometime today. Having two identical servers has it's advantages. > Is there any way to reproduce this on another box for testing? (I > assume not, due to the nature of these things) I think this bug is very circumstance specific, and my database design just happens to exercise the bug. The fact that I have two identical servers experiencing exactly the same problem does helps narrow the field of possibilities, but the servers are my no means "expendable". > Also - I wonder if doing a snapshot on the filesystem would flush > out the pages - is that something you can try? Possibly. I will see what I can do. -- Seeya...Q -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- _____ / Quinton Dolan - qdolan@gmail.com __ __/ / / __/ / / / __ / _/ / / Gold Coast, QLD, Australia __/ __/ __/ ____/ / - / Ph: +61 419 729 806 _______ / _\ From owner-freebsd-fs@FreeBSD.ORG Wed Aug 9 06:25:06 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5F41F16A4DA; Wed, 9 Aug 2006 06:25:06 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id A696D43D45; Wed, 9 Aug 2006 06:25:05 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id F00825138F; Wed, 9 Aug 2006 08:25:03 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id C6AFD51339; Wed, 9 Aug 2006 08:24:57 +0200 (CEST) Date: Wed, 9 Aug 2006 08:24:12 +0200 From: Pawel Jakub Dawidek To: Daniel O'Connor Message-ID: <20060809062412.GA1182@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <200608091013.25981.doconnor@gsoft.com.au> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="uAKRQypu60I7Lcqm" Content-Disposition: inline In-Reply-To: <200608091013.25981.doconnor@gsoft.com.au> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2006 06:25:06 -0000 --uAKRQypu60I7Lcqm Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 09, 2006 at 10:13:24AM +0930, Daniel O'Connor wrote: > On Wednesday 09 August 2006 05:22, Pawel Jakub Dawidek wrote: > > I'm pleased to announce hopefully the final gjournal patches. >=20 > This has diffs which remove stge presumably because you did your diff at= =20 > exactly the wrong time :) Oops, I updated the patch. Thanks. > (The mods to CAM/ATA/etc are part of the patch for flushing cache I am=20 > guessing) That's right. > It also appears to remove 'MNT_JAILDEVFS" because it uses the same bitmas= k, I=20 > am guessing that's a mistake. No, it's not. MNT_JAILDEVFS is unused (it is already removed from HEAD). --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --uAKRQypu60I7Lcqm Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE2X+MForvXbEpPzQRAs8gAJ9/Oga7Om5IQdCzvNaduB4rHOZ7fwCferAM EJh3Wpwi3BLlCFv50wSnV/4= =hrlA -----END PGP SIGNATURE----- --uAKRQypu60I7Lcqm-- From owner-freebsd-fs@FreeBSD.ORG Wed Aug 9 10:57:02 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9C8A716A4E2 for ; Wed, 9 Aug 2006 10:57:02 +0000 (UTC) (envelope-from igorr@speechpro.com) Received: from speechpro.ru (speech-tech-2.ip.PeterStar.net [81.3.190.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id E9F0443D88 for ; Wed, 9 Aug 2006 10:56:56 +0000 (GMT) (envelope-from igorr@speechpro.com) Received: from [192.168.2.26] (helo=sysadm.stc) by s1.stc with esmtp (Exim 4.53 (FreeBSD)) id 1GAlk7-000Otj-2c for freebsd-fs@freebsd.org; Wed, 09 Aug 2006 14:56:55 +0400 Received: from localhost.stc ([127.0.0.1] helo=sysadm.stc) by sysadm.stc with esmtp (Exim 4.62 (FreeBSD)) (envelope-from ) id 1GAlkv-00031X-Gv for freebsd-fs@freebsd.org; Wed, 09 Aug 2006 14:57:45 +0400 Received: (from igorr@localhost) by sysadm.stc (8.13.6/8.13.6/Submit) id k79Avj7x011626 for freebsd-fs@freebsd.org; Wed, 9 Aug 2006 14:57:45 +0400 (MSD) (envelope-from igorr) Date: Wed, 9 Aug 2006 14:57:45 +0400 From: Igor Robul To: freebsd-fs@freebsd.org Message-ID: <20060809105745.GD19441@sysadm.stc> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.12-2006-07-14 X-Archived: Yes Subject: gjournal compilation failure X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2006 10:57:02 -0000 Hello, I have just tried to compile world with latest gjournal patch, and I got cc -pipe -O2 -I/usr/src/sbin/fsck_ffs -DRESCUE -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -c /usr/src/sbin/fsck_ffs/gjournal.c /usr/src/sbin/fsck_ffs/gjournal.c: In function `gjournal_check': /usr/src/sbin/fsck_ffs/gjournal.c:717: warning: dereferencing type-punned pointer will break strict-aliasing rules *** Error code 1 From owner-freebsd-fs@FreeBSD.ORG Wed Aug 9 12:59:19 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 431B216A4DA; Wed, 9 Aug 2006 12:59:19 +0000 (UTC) (envelope-from b.candler@pobox.com) Received: from rune.pobox.com (rune.pobox.com [208.210.124.79]) by mx1.FreeBSD.org (Postfix) with ESMTP id E0E7343D55; Wed, 9 Aug 2006 12:59:18 +0000 (GMT) (envelope-from b.candler@pobox.com) Received: from rune (localhost [127.0.0.1]) by rune.pobox.com (Postfix) with ESMTP id A4D8E7AE7A; Wed, 9 Aug 2006 08:59:37 -0400 (EDT) Received: from mappit.local.linnet.org (212-74-113-67.static.dsl.as9105.com [212.74.113.67]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by rune.sasl.smtp.pobox.com (Postfix) with ESMTP id C760F7A116; Wed, 9 Aug 2006 08:59:33 -0400 (EDT) Received: from lists by mappit.local.linnet.org with local (Exim 4.61 (FreeBSD)) (envelope-from ) id 1GAneP-000Hah-EQ; Wed, 09 Aug 2006 13:59:09 +0100 Date: Wed, 9 Aug 2006 13:59:09 +0100 From: Brian Candler To: Pawel Jakub Dawidek Message-ID: <20060809125909.GB67566@uk.tiscali.com> References: <20060808195202.GA1564@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060808195202.GA1564@garage.freebsd.pl> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2006 12:59:19 -0000 On Tue, Aug 08, 2006 at 09:52:02PM +0200, Pawel Jakub Dawidek wrote: > Another change is that now from a configuration point of view gjournal > is threated simlar to soft-updates - you need to specify -J flag at > newfs(8) time or use 'tunefs -J enable '. 'gjournal' is no longer > a mount option. Does this restrict gjournal to UFS only? I've not been following this closely, but I saw the original announcement: "GJournal was designed to journal GEOM providers, so it actually works below file system layer, but it has hooks which allow to work with file systems. In other words, gjournal is not file system-depended, it can work probably with any file system with minimum knowledge about it. I implemented only UFS support." I just wondered if gjournal support is likely to be limited to UFS now. Regards, Brian. From owner-freebsd-fs@FreeBSD.ORG Wed Aug 9 13:04:15 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F7B116A4E1; Wed, 9 Aug 2006 13:04:15 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8F76543D6A; Wed, 9 Aug 2006 13:04:14 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 149AC5138F; Wed, 9 Aug 2006 15:04:13 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 2DC095133B; Wed, 9 Aug 2006 15:04:08 +0200 (CEST) Date: Wed, 9 Aug 2006 15:03:18 +0200 From: Pawel Jakub Dawidek To: Brian Candler Message-ID: <20060809130318.GG1182@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <20060809125909.GB67566@uk.tiscali.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6lCXDTVICvIQMz0h" Content-Disposition: inline In-Reply-To: <20060809125909.GB67566@uk.tiscali.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2006 13:04:15 -0000 --6lCXDTVICvIQMz0h Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 09, 2006 at 01:59:09PM +0100, Brian Candler wrote: > On Tue, Aug 08, 2006 at 09:52:02PM +0200, Pawel Jakub Dawidek wrote: > > Another change is that now from a configuration point of view gjournal > > is threated simlar to soft-updates - you need to specify -J flag at > > newfs(8) time or use 'tunefs -J enable '. 'gjournal' is no longer > > a mount option. >=20 > Does this restrict gjournal to UFS only? >=20 > I've not been following this closely, but I saw the original announcement: >=20 > "GJournal was designed to journal GEOM providers, so it actually works > below file system layer, but it has hooks which allow to work with > file systems. In other words, gjournal is not file system-depended, > it can work probably with any file system with minimum knowledge > about it. I implemented only UFS support." >=20 > I just wondered if gjournal support is likely to be limited to UFS now. You can use gjournal without file system, but most users are interested in journaling the UFS file system. This is the only file system supported currently by gjournal. If you want to use it with another file system, support for it has to be added to gjournal. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --6lCXDTVICvIQMz0h Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE2d0WForvXbEpPzQRAhAzAJ0WRRNtlgJhz0Be6zKlDPRLi0J3mACgzJGL 1aBH06fo8TamKSplTZLtznE= =P8qf -----END PGP SIGNATURE----- --6lCXDTVICvIQMz0h-- From owner-freebsd-fs@FreeBSD.ORG Wed Aug 9 23:18:06 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 367A516A4DA for ; Wed, 9 Aug 2006 23:18:06 +0000 (UTC) (envelope-from geetakhetan@hotmail.com) Received: from bay0-omc1-s4.bay0.hotmail.com (bay0-omc1-s4.bay0.hotmail.com [65.54.246.76]) by mx1.FreeBSD.org (Postfix) with ESMTP id F1D7B43D49 for ; Wed, 9 Aug 2006 23:18:05 +0000 (GMT) (envelope-from geetakhetan@hotmail.com) Received: from hotmail.com ([64.4.51.35]) by bay0-omc1-s4.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.1830); Wed, 9 Aug 2006 16:18:05 -0700 Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Wed, 9 Aug 2006 16:18:05 -0700 Message-ID: Received: from 64.4.51.220 by by107fd.bay107.hotmail.msn.com with HTTP; Wed, 09 Aug 2006 23:18:01 GMT X-Originating-IP: [65.219.128.225] X-Originating-Email: [geetakhetan@hotmail.com] X-Sender: geetakhetan@hotmail.com From: "Geeta Khetan" To: freebsd-fs@freebsd.org Date: Wed, 09 Aug 2006 16:18:01 -0700 X-OriginalArrivalTime: 09 Aug 2006 23:18:05.0284 (UTC) FILETIME=[1150CA40:01C6BC0A] X-Mailman-Approved-At: Thu, 10 Aug 2006 00:07:51 +0000 MIME-Version: 1.0 Content-Type: text/plain; format="flowed" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: bitmap of filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2006 23:18:06 -0000 Is there a global bitmap that keeps track of busy/free blocks over the entire file system ? Is there a API or command i can use for that map? Thanks From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 00:21:52 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D7DC416A4DF for ; Thu, 10 Aug 2006 00:21:52 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3BE2543D49 for ; Thu, 10 Aug 2006 00:21:51 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [192.168.254.11] (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k7A0LiHA012184; Wed, 9 Aug 2006 18:21:50 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <44DA7C18.9050802@samsco.org> Date: Wed, 09 Aug 2006 18:21:44 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.13) Gecko/20060414 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Geeta Khetan References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.4 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: freebsd-fs@freebsd.org Subject: Re: bitmap of filesystem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 00:21:52 -0000 Geeta Khetan wrote: > > Is there a global bitmap that keeps track of busy/free blocks over the > entire file system ? Is there a API or command i can use for that map? > Thanks No, the bitmaps are stored on a per-cylinder group basis. You can use cgread() in libufs to read each group and then examine the bitmaps with the cg* macros in /sys/ufs/ffs/fs.h. Scott From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 12:04:25 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 333E516A4DD for ; Thu, 10 Aug 2006 12:04:25 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2396243D60 for ; Thu, 10 Aug 2006 12:04:07 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id DDDF251396; Thu, 10 Aug 2006 14:04:05 +0200 (CEST) Received: from localhost (ana50.internetdsl.tpnet.pl [83.17.82.50]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id BE32F51392; Thu, 10 Aug 2006 14:04:01 +0200 (CEST) Date: Thu, 10 Aug 2006 14:03:12 +0200 From: Pawel Jakub Dawidek To: Igor Robul Message-ID: <20060810120312.GC22917@garage.freebsd.pl> References: <20060809105745.GD19441@sysadm.stc> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OBd5C1Lgu00Gd/Tn" Content-Disposition: inline In-Reply-To: <20060809105745.GD19441@sysadm.stc> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: gjournal compilation failure X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 12:04:25 -0000 --OBd5C1Lgu00Gd/Tn Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 09, 2006 at 02:57:45PM +0400, Igor Robul wrote: > Hello,=20 >=20 > I have just tried to compile world with latest gjournal patch, and I got >=20 > cc -pipe -O2 -I/usr/src/sbin/fsck_ffs -DRESCUE -Wsystem-headers -Werror > -Wall -Wno-format-y2k -Wno-uninitialized -c > /usr/src/sbin/fsck_ffs/gjournal.c > /usr/src/sbin/fsck_ffs/gjournal.c: In function `gjournal_check': > /usr/src/sbin/fsck_ffs/gjournal.c:717: warning: dereferencing > type-punned pointer will break strict-aliasing rules > *** Error code 1 Is this amd64? Markus Trippelsdorf reported this problem already. As a workaround he added -fno-strict-aliasing to the CFLAGS of sbin/fsck_ffs/Makefile. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --OBd5C1Lgu00Gd/Tn Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE2yCAForvXbEpPzQRApA0AKCGmzZKNzMAbne2bplOXWF5sAX2tQCgjVFb 0+vk43Le2SAvA+W0fXFygyQ= =DWxd -----END PGP SIGNATURE----- --OBd5C1Lgu00Gd/Tn-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 13:39:05 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 74CFD16A4DD; Thu, 10 Aug 2006 13:39:05 +0000 (UTC) (envelope-from des@des.no) Received: from tim.des.no (tim.des.no [194.63.250.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id 00C1B43D45; Thu, 10 Aug 2006 13:39:04 +0000 (GMT) (envelope-from des@des.no) Received: from tim.des.no (localhost [127.0.0.1]) by spam.des.no (Postfix) with ESMTP id 7FE8020A9; Thu, 10 Aug 2006 15:39:00 +0200 (CEST) X-Spam-Tests: AWL X-Spam-Learn: disabled X-Spam-Score: 0.0/3.0 X-Spam-Checker-Version: SpamAssassin 3.1.3 (2006-06-01) on tim.des.no Received: from xps.des.no (des.no [80.203.243.180]) by tim.des.no (Postfix) with ESMTP id 6CCCF2085; Thu, 10 Aug 2006 15:39:00 +0200 (CEST) Received: by xps.des.no (Postfix, from userid 1001) id 4398C33C28; Thu, 10 Aug 2006 15:39:00 +0200 (CEST) From: des@des.no (Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=) To: Markus Trippelsdorf References: <20060808195202.GA1564@garage.freebsd.pl> <20060808210211.GA730@bsd.trippelsdorf.de> <20060808214116.GB730@bsd.trippelsdorf.de> Date: Thu, 10 Aug 2006 15:39:00 +0200 In-Reply-To: <20060808214116.GB730@bsd.trippelsdorf.de> (Markus Trippelsdorf's message of "Tue, 8 Aug 2006 23:41:16 +0200") Message-ID: <868xlw3e2z.fsf@xps.des.no> User-Agent: Gnus/5.110003 (No Gnus v0.3) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, Pawel Jakub Dawidek , freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 13:39:05 -0000 Markus Trippelsdorf writes: > Adding -fno-strict-aliasing to the CFLAGS of sbin/fsck_ffs/Makefile > solved the problem for now. -fno-strict-aliasing does not solve anything, it just prevents the compiler from telling you about bugs in your code. DES --=20 Dag-Erling Sm=F8rgrav - des@des.no From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 13:54:47 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A43A716A4DA for ; Thu, 10 Aug 2006 13:54:47 +0000 (UTC) (envelope-from markus@trippelsdorf.de) Received: from blue-ld-033.synserver.de (smtp-out-44.synserver.de [217.119.50.44]) by mx1.FreeBSD.org (Postfix) with SMTP id 20A7443D4C for ; Thu, 10 Aug 2006 13:54:45 +0000 (GMT) (envelope-from markus@trippelsdorf.de) Received: (qmail 28888 invoked by uid 0); 10 Aug 2006 13:54:28 -0000 X-SynServer-RemoteDnsName: port-212-202-34-169.dynamic.qsc.de X-SynServer-AuthUser: markus@trippelsdorf.de Received: from port-212-202-34-169.dynamic.qsc.de (HELO bsd.trippelsdorf.de) (212.202.34.169) by mx-05.synserver.de with SMTP; 10 Aug 2006 13:54:27 -0000 Date: Thu, 10 Aug 2006 15:54:26 +0200 From: Markus Trippelsdorf To: Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?= Message-ID: <20060810135426.GA778@bsd.trippelsdorf.de> References: <20060808195202.GA1564@garage.freebsd.pl> <20060808210211.GA730@bsd.trippelsdorf.de> <20060808214116.GB730@bsd.trippelsdorf.de> <868xlw3e2z.fsf@xps.des.no> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <868xlw3e2z.fsf@xps.des.no> Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, Pawel Jakub Dawidek , freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 13:54:47 -0000 On Thu, Aug 10, 2006 at 03:39:00PM +0200, Dag-Erling Smørgrav wrote: > Markus Trippelsdorf writes: > > Adding -fno-strict-aliasing to the CFLAGS of sbin/fsck_ffs/Makefile > > solved the problem for now. > > -fno-strict-aliasing does not solve anything, it just prevents the > compiler from telling you about bugs in your code. Which must be the reason that it is used when building the kernel on AMD64. ;-) -- Markus From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 14:05:27 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 64F7016A4DF; Thu, 10 Aug 2006 14:05:27 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE88143D70; Thu, 10 Aug 2006 14:05:26 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id 3EBC25CB6; Thu, 10 Aug 2006 10:05:26 -0400 (EDT) X-Virus-Scanned: amavisd-new at codefab.com Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PPna4MOjhCCy; Thu, 10 Aug 2006 10:05:25 -0400 (EDT) Received: from [192.168.1.251] (pool-68-161-117-245.ny325.east.verizon.net [68.161.117.245]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id CD2CF5C6E; Thu, 10 Aug 2006 10:05:24 -0400 (EDT) Message-ID: <44DB3D1A.1030605@mac.com> Date: Thu, 10 Aug 2006 10:05:14 -0400 From: Chuck Swiger User-Agent: Thunderbird 1.5.0.5 (Windows/20060719) MIME-Version: 1.0 To: =?ISO-8859-1?Q?Dag-Erling_Sm=F8rgrav?= References: <20060808195202.GA1564@garage.freebsd.pl> <20060808210211.GA730@bsd.trippelsdorf.de> <20060808214116.GB730@bsd.trippelsdorf.de> <868xlw3e2z.fsf@xps.des.no> In-Reply-To: <868xlw3e2z.fsf@xps.des.no> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 14:05:27 -0000 Dag-Erling Smørgrav wrote: > Markus Trippelsdorf writes: >> Adding -fno-strict-aliasing to the CFLAGS of sbin/fsck_ffs/Makefile >> solved the problem for now. > > -fno-strict-aliasing does not solve anything, it just prevents the > compiler from telling you about bugs in your code. It also prevents the compiler's optimizer from assuming that a pointer dereference may impact datatypes of other sizes. Since there exists C code that changes, say, int16_t's or int32_t's one byte at a time, using that flag prevents the compiler from generating buggy executables from C sources which do not adhere to the assumptions made by "strict aliasing". -- -Chuck From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 15:33:44 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E167616A4DA; Thu, 10 Aug 2006 15:33:44 +0000 (UTC) (envelope-from kmacy@fsmware.com) Received: from demos.bsdclusters.com (demos.bsdclusters.com [69.55.225.36]) by mx1.FreeBSD.org (Postfix) with ESMTP id BE72F43D77; Thu, 10 Aug 2006 15:33:39 +0000 (GMT) (envelope-from kmacy@fsmware.com) Received: from demos.bsdclusters.com (demos [69.55.225.36]) by demos.bsdclusters.com (8.12.8p1/8.12.8) with ESMTP id k7AFXblZ045353; Thu, 10 Aug 2006 08:33:37 -0700 (PDT) (envelope-from kmacy@fsmware.com) Received: from localhost (kmacy@localhost) by demos.bsdclusters.com (8.12.8p1/8.12.8/Submit) with ESMTP id k7AFXaS5045332; Thu, 10 Aug 2006 08:33:37 -0700 (PDT) X-Authentication-Warning: demos.bsdclusters.com: kmacy owned process doing -bs Date: Thu, 10 Aug 2006 08:33:36 -0700 (PDT) From: Kip Macy X-X-Sender: kmacy@demos.bsdclusters.com To: Markus Trippelsdorf In-Reply-To: <20060810135426.GA778@bsd.trippelsdorf.de> Message-ID: <20060810083206.D27159@demos.bsdclusters.com> References: <20060808195202.GA1564@garage.freebsd.pl> <20060808210211.GA730@bsd.trippelsdorf.de> <20060808214116.GB730@bsd.trippelsdorf.de> <868xlw3e2z.fsf@xps.des.no> <20060810135426.GA778@bsd.trippelsdorf.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-fs@FreeBSD.org, Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?= , freebsd-current@FreeBSD.org, Pawel Jakub Dawidek , freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 15:33:45 -0000 > > -fno-strict-aliasing does not solve anything, it just prevents the > > compiler from telling you about bugs in your code. > > Which must be the reason that it is used when building the kernel on AMD64. > ;-) Actually, whenever the kernel is compiled with -O2. "Type punning" is sufficiently uncommon in the kernel proper that it is correctable there. However, it is pervasive throughout the device drivers. -Kip From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 18:47:36 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 387F416A4E1; Thu, 10 Aug 2006 18:47:36 +0000 (UTC) (envelope-from craig@xfoil.gank.org) Received: from ion.gank.org (ion.gank.org [69.55.238.164]) by mx1.FreeBSD.org (Postfix) with ESMTP id 99D5B43D70; Thu, 10 Aug 2006 18:47:28 +0000 (GMT) (envelope-from craig@xfoil.gank.org) Received: by ion.gank.org (Postfix, from userid 1001) id E6865117D0; Thu, 10 Aug 2006 13:47:27 -0500 (CDT) Date: Thu, 10 Aug 2006 13:47:23 -0500 From: Craig Boston To: Pawel Jakub Dawidek Message-ID: <20060810184702.GA8567@nowhere> Mail-Followup-To: Craig Boston , Pawel Jakub Dawidek , freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org References: <20060808195202.GA1564@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="24zk1gE8NUlDmwG9" Content-Disposition: inline In-Reply-To: <20060808195202.GA1564@garage.freebsd.pl> User-Agent: Mutt/1.4.2.2i Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 18:47:36 -0000 --24zk1gE8NUlDmwG9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, It's great to see this project so close to completion! I'm trying it out on a couple machines to see how it goes. A few comments and questions: * It took me a little by surprise that it carves 1G out of the device for the journal. Depending on the size of the device that can be a pretty hefty price to pay (and I didn't see any mention of it in the setup notes). For a couple of my smaller filesystems I reduced it to 512MB. Perhaps some algorithm for auto-sizing the journal based on the size / expected workload of the device would be in order? * Attached is a quick patch for geom_eli to allow it to pass BIO_FLUSH down to its backing device. It seems like the right thing to do and fixes the "BIO_FLUSH not supported" warning on my laptop that uses a geli encrypted disk. * On a different system, however, it complains about it even on a raw ATA slice: atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0 ata0: on atapci1 ad0: 114473MB at ata0-master UDMA100 GEOM_JOURNAL: BIO_FLUSH not supported by ad0s1e. It seems like a reasonably modern controller and disk, at least it should be capable of issuing a cache flush command. Not sure why it doesn't like it :/ * How "close" does the filesystem need to be to the gjournal device in order for the UFS hooks to work? Directly on it? The geom stack on my laptop currently looks something like this: [geom_disk] ad0 <- [geom_eli] ad0.eli <- [geom_gpt] ad0.elip6 <- [geom_label] gjtest <- [geom_journal] gjtest.journal <- UFS I was wondering if an arrangement like this would work: [geom_journal] ad0p6.journal <- [geom_eli] ad0p6.journaleli <- UFS and if it would be any more efficient (journal the encrypted data rather than encrypt the journal). Or even gjournal the whole disk at once? Haven't been brave enough to try gjournal on root yet, but my /usr and /compile (src, obj, ports) partitions are already on it so I'm sure I'll try it soon ;) Craig --24zk1gE8NUlDmwG9 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="geli-bioflush.patch" === sys/geom/eli/g_eli.c ================================================================== --- sys/geom/eli/g_eli.c (revision 23) +++ sys/geom/eli/g_eli.c (local) @@ -340,6 +340,7 @@ case BIO_READ: case BIO_WRITE: case BIO_GETATTR: + case BIO_FLUSH: break; case BIO_DELETE: /* @@ -375,6 +376,7 @@ wakeup(sc); break; case BIO_GETATTR: + case BIO_FLUSH: cbp->bio_done = g_std_done; cp = LIST_FIRST(&sc->sc_geom->consumer); cbp->bio_to = cp->provider; --24zk1gE8NUlDmwG9-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 19:07:03 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 11AD016A4DF; Thu, 10 Aug 2006 19:07:03 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id A4BD643D49; Thu, 10 Aug 2006 19:07:02 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id k7AJ714T017972; Thu, 10 Aug 2006 14:07:01 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44DB83E9.2090402@centtech.com> Date: Thu, 10 Aug 2006 14:07:21 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.5 (X11/20060802) MIME-Version: 1.0 To: Craig Boston , Pawel Jakub Dawidek , freebsd-fs@FreeBSD.org, freebsd-geom@FreeBSD.org References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> In-Reply-To: <20060810184702.GA8567@nowhere> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1644/Wed Aug 9 22:55:42 2006 on mh2.centtech.com X-Virus-Status: Clean Cc: Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 19:07:03 -0000 On 08/10/06 13:47, Craig Boston wrote: > Hi, > > It's great to see this project so close to completion! I'm trying it > out on a couple machines to see how it goes. > > A few comments and questions: > > * It took me a little by surprise that it carves 1G out of the device > for the journal. Depending on the size of the device that can be a > pretty hefty price to pay (and I didn't see any mention of it in the > setup notes). For a couple of my smaller filesystems I reduced it to > 512MB. Perhaps some algorithm for auto-sizing the journal based on > the size / expected workload of the device would be in order? The size of the journal is related to the write speed of the journeled device and the journal swap time (default 10 seconds). Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 19:19:11 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 20E6116A4DD; Thu, 10 Aug 2006 19:19:11 +0000 (UTC) (envelope-from craig@xfoil.gank.org) Received: from ion.gank.org (ion.gank.org [69.55.238.164]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3590343D4C; Thu, 10 Aug 2006 19:19:10 +0000 (GMT) (envelope-from craig@xfoil.gank.org) Received: by ion.gank.org (Postfix, from userid 1001) id 99214117D0; Thu, 10 Aug 2006 14:19:09 -0500 (CDT) Date: Thu, 10 Aug 2006 14:19:09 -0500 From: Craig Boston To: Eric Anderson Message-ID: <20060810191908.GA32326@nowhere> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> <44DB83E9.2090402@centtech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <44DB83E9.2090402@centtech.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@FreeBSD.org, Pawel Jakub Dawidek , freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 19:19:11 -0000 On Thu, Aug 10, 2006 at 02:07:21PM -0500, Eric Anderson wrote: > The size of the journal is related to the write speed of the journeled > device and the journal swap time (default 10 seconds). I was going to ask if that was theoretical max speed for the bus or physical write speed, but it appears that this is not (yet?) the case: sbin/geom/class/journal/geom_journal.c: if (jsize == -1) { /* * No journal size specified. 1GB should be safe * default. */ jsize = 1073741824ULL; } msize = g_get_mediasize(data); ssize = g_get_sectorsize(data); if (jsize + ssize >= msize) { gctl_error(req, "Provider too small for journalling. " "You can try smaller jsize (default is %jd).", jsize); return; } From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 19:21:59 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3308316A4E0; Thu, 10 Aug 2006 19:21:59 +0000 (UTC) (envelope-from w@expro.pl) Received: from mailin3.expro.pl (mailin3.expro.pl [193.25.166.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4955D43D6E; Thu, 10 Aug 2006 19:21:55 +0000 (GMT) (envelope-from w@expro.pl) Received: from europa-1.dmz.exprozone ([10.0.16.45]:57461 helo=europa.expro.pl) (envelope-sender ) by mailin3.expro.pl with esmtp (Exim 4.50) id 1GBG6L-0006JC-5q; Thu, 10 Aug 2006 21:21:53 +0200 Received: by europa.expro.pl (Postfix, from userid 1001) id D27A3536116; Thu, 10 Aug 2006 21:21:52 +0200 (CEST) Date: Thu, 10 Aug 2006 21:21:52 +0200 From: Jan Srzednicki To: Pawel Jakub Dawidek Message-ID: <20060810192152.GE57857@europa.expro.pl> References: <20060808195202.GA1564@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060808195202.GA1564@garage.freebsd.pl> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 19:21:59 -0000 On Tue, Aug 08, 2006 at 09:52:02PM +0200, Pawel Jakub Dawidek wrote: > Hi. > > I'm pleased to announce hopefully the final gjournal patches. > [..] > Now you need to recompile the world and the kernel. Remember to add > 'options UFS_GJOURNAL' to your kernel configuration file. > > If there will be no bug reports nor strong objections, I'd like commit > gjournal in a week or two first to 7-CURRENT and then to 6-STABLE (maybe > before 6.2-RELEASE, but it depends on testing and feedback from users). Hello, I'd happily perform some testing, only if the aac(4) driver would also be patched. I know I can run gjournal w/o the BIO_FLUSH feature, but that would make any performance results rather worthles, wouldn't it? -- Jan Srzednicki w@expro.pl From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 19:29:45 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4CE3916A4DF; Thu, 10 Aug 2006 19:29:45 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4AC1943D66; Thu, 10 Aug 2006 19:29:41 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 8BB0F5133B; Thu, 10 Aug 2006 21:29:40 +0200 (CEST) Received: from localhost (dkv208.neoplus.adsl.tpnet.pl [83.24.25.208]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 50F8550EA7; Thu, 10 Aug 2006 21:29:34 +0200 (CEST) Date: Thu, 10 Aug 2006 21:28:41 +0200 From: Pawel Jakub Dawidek To: Craig Boston , freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Message-ID: <20060810192841.GA1345@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="M9NhX3UHpAaciwkO" Content-Disposition: inline In-Reply-To: <20060810184702.GA8567@nowhere> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 19:29:45 -0000 --M9NhX3UHpAaciwkO Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Aug 10, 2006 at 01:47:23PM -0500, Craig Boston wrote: > Hi, >=20 > It's great to see this project so close to completion! I'm trying it > out on a couple machines to see how it goes. >=20 > A few comments and questions: >=20 > * It took me a little by surprise that it carves 1G out of the device > for the journal. Depending on the size of the device that can be a > pretty hefty price to pay (and I didn't see any mention of it in the > setup notes). For a couple of my smaller filesystems I reduced it to > 512MB. Perhaps some algorithm for auto-sizing the journal based on > the size / expected workload of the device would be in order? It will be pointed out in documentation when I finally prepare it. I don't have plans about autosizing currently. > * Attached is a quick patch for geom_eli to allow it to pass BIO_FLUSH > down to its backing device. It seems like the right thing to do and > fixes the "BIO_FLUSH not supported" warning on my laptop that uses a > geli encrypted disk. I've this already in my perforce tree. I also implemented BIO_FLUSH passing in gmirror and graid3. I also added a flag for gmirror and graid3 which says "don't resynchronize components after a power failure - trust they are consistent". And they are always consistent when placed below gjournal. > * On a different system, however, it complains about it even on a raw > ATA slice: >=20 > atapci1: port 0x1f0-0x1f7,0x3f6,0x170= -0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0 > ata0: on atapci1 > ad0: 114473MB at ata0-master UDMA100 > GEOM_JOURNAL: BIO_FLUSH not supported by ad0s1e. >=20 > It seems like a reasonably modern controller and disk, at least it > should be capable of issuing a cache flush command. Not sure why it > doesn't like it :/ We would need to add some printfs to diagnoze this probably - you can try adding some lines to ad_init() to get this: if (atadev->param.support.command1 & ATA_SUPPORT_WRITECACHE) { if (ata_wc) ata_controlcmd(dev, ATA_SETFEATURES, ATA_SF_ENAB_WCACHE, 0, 0); else ata_controlcmd(dev, ATA_SETFEATURES, ATA_SF_DIS_WCACHE, 0, 0); } else { printf("ad_init: WRITE CACHE not supported by ad%d.\n", device_get_unit(dev)); } > * How "close" does the filesystem need to be to the gjournal device in > order for the UFS hooks to work? Directly on it? >=20 > The geom stack on my laptop currently looks something like this: >=20 > [geom_disk] ad0 <- [geom_eli] ad0.eli <- [geom_gpt] ad0.elip6 <- > [geom_label] gjtest <- [geom_journal] gjtest.journal <- UFS >=20 > I was wondering if an arrangement like this would work: >=20 > [geom_journal] ad0p6.journal <- [geom_eli] ad0p6.journaleli <- UFS >=20 > and if it would be any more efficient (journal the encrypted data > rather than encrypt the journal). Or even gjournal the whole disk at > once? When you mount file system it sends BIO_GETATTR "GJOURNAL::provider" requests. So as long as classes between the file system and gjournal provider pass BIO_GETATTR down, it will work. On my home machine I've the following configuration: raid3/DATA1.elid.journal So it's UFS over gjournal over bsdlabel over geli over raid3 over ata. I prefer to put gjournal on the top, because it gives consistency to layers below it. For example I can use geli with bigger sector size (sector size greater than disk sector size in encryption-only-mode can be unreliable on power failures, which is not the case when gjournal is above geli), I can turn off synchronization of gmirror/graid3 after a power failure, etc. On the other hand configuring geli on top of gjournal can be more effective for large files - geli will not encrypt the data twice. Fortunatelly with GEOM you can freely mix your puzzles. > Haven't been brave enough to try gjournal on root yet, but my /usr and > /compile (src, obj, ports) partitions are already on it so I'm sure I'll > try it soon ;) Markus Trippelsdorf reported that it doesn't work out of the box, but he manage to make it to work with some small changes to fsck_ffs(8). --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --M9NhX3UHpAaciwkO Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE24jpForvXbEpPzQRAmenAKC/J05IojZltHSXJFETFfsgAqMYZwCdHXeQ /EXU/FMCMvMFGhyqVW6JlNE= =LvdY -----END PGP SIGNATURE----- --M9NhX3UHpAaciwkO-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 19:44:02 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 927A416A4DE; Thu, 10 Aug 2006 19:44:02 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id EA6E143D49; Thu, 10 Aug 2006 19:43:59 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 7B93B5133B; Thu, 10 Aug 2006 21:43:58 +0200 (CEST) Received: from localhost (dkv208.neoplus.adsl.tpnet.pl [83.24.25.208]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 92E4C5131F; Thu, 10 Aug 2006 21:43:53 +0200 (CEST) Date: Thu, 10 Aug 2006 21:43:04 +0200 From: Pawel Jakub Dawidek To: Jan Srzednicki Message-ID: <20060810194304.GB1345@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810192152.GE57857@europa.expro.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="CdrF4e02JqNVZeln" Content-Disposition: inline In-Reply-To: <20060810192152.GE57857@europa.expro.pl> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 19:44:02 -0000 --CdrF4e02JqNVZeln Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Aug 10, 2006 at 09:21:52PM +0200, Jan Srzednicki wrote: > On Tue, Aug 08, 2006 at 09:52:02PM +0200, Pawel Jakub Dawidek wrote: > > Hi. > >=20 > > I'm pleased to announce hopefully the final gjournal patches. > >=20 > [..] > > Now you need to recompile the world and the kernel. Remember to add > > 'options UFS_GJOURNAL' to your kernel configuration file. > >=20 > > If there will be no bug reports nor strong objections, I'd like commit > > gjournal in a week or two first to 7-CURRENT and then to 6-STABLE (maybe > > before 6.2-RELEASE, but it depends on testing and feedback from users). >=20 > Hello, >=20 > I'd happily perform some testing, only if the aac(4) driver would also > be patched. I know I can run gjournal w/o the BIO_FLUSH feature, but > that would make any performance results rather worthles, wouldn't it? =46rom my tests BIO_FLUSH doesn't have huge impact on performance (if at all), but I suspect it greatly depends on specific HW. Unfortunately I'm not able to implement BIO_FLUSH to all out storage drivers. Currently BIO_FLUSH is supported by ata(4) (/dev/a[dr]X), da(4) and amr(4). --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --CdrF4e02JqNVZeln Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE24xIForvXbEpPzQRAic2AJ4xyBiy+trSzSSosMOoagEfo+tXGQCgzMW0 Qq+z9DJ52cGZwaJh1rbJbhM= =Xq3r -----END PGP SIGNATURE----- --CdrF4e02JqNVZeln-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 20:00:15 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8000416A4DE for ; Thu, 10 Aug 2006 20:00:15 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30306.mail.mud.yahoo.com (web30306.mail.mud.yahoo.com [209.191.69.68]) by mx1.FreeBSD.org (Postfix) with SMTP id 4EBC043D55 for ; Thu, 10 Aug 2006 20:00:10 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 98204 invoked by uid 60001); 10 Aug 2006 20:00:09 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=SwQQzOP9LFfE7G4hbjAVzsnTa+1e7hjziy9QWYDQb+O2bPe5jMzGjaXx6vy/G1tjfhTaTunAwHwJ2cnV9tg0wl25KSiLg9UPckc82WJmou06oFYcYvjsHra8e88cOf43MSCISPrRRt3OwfIn/s+ssGGy6xgvAPz+7wcA61sxgaU= ; Message-ID: <20060810200009.98202.qmail@web30306.mail.mud.yahoo.com> Received: from [213.54.70.229] by web30306.mail.mud.yahoo.com via HTTP; Thu, 10 Aug 2006 13:00:09 PDT Date: Thu, 10 Aug 2006 13:00:09 -0700 (PDT) From: "R. B. Riddick" To: Craig Boston , Eric Anderson In-Reply-To: <20060810191908.GA32326@nowhere> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-fs@FreeBSD.org, Pawel Jakub Dawidek , freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 20:00:15 -0000 --- Craig Boston wrote: > sbin/geom/class/journal/geom_journal.c: > > if (jsize == -1) { > /* > * No journal size specified. 1GB should be safe > * default. > */ > jsize = 1073741824ULL; > } > msize = g_get_mediasize(data); > ssize = g_get_sectorsize(data); > if (jsize + ssize >= msize) { > gctl_error(req, "Provider too small for journalling. > " > "You can try smaller jsize (default is %jd).", > jsize); > return; > The gctl_error message shall be rewritten: It should be: "You can try smaller journal size (%jd too large).", since "default" implies for me 1GB, while it might well be, that somebody chose 10GB. -Arne --- 2*2=5 __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 20:01:38 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2DD8A16A524; Thu, 10 Aug 2006 20:01:38 +0000 (UTC) (envelope-from w@expro.pl) Received: from mailin1.expro.pl (mailin1.expro.pl [193.25.166.20]) by mx1.FreeBSD.org (Postfix) with ESMTP id 833EE43D46; Thu, 10 Aug 2006 20:01:37 +0000 (GMT) (envelope-from w@expro.pl) Received: from europa-1.dmz.exprozone ([10.0.16.45] helo=europa.expro.pl) (envelope-sender ) by mailin1.expro.pl with esmtp (Exim 4.50) id 1GBGil-0004Pu-Ow; Thu, 10 Aug 2006 22:01:35 +0200 Received: by europa.expro.pl (Postfix, from userid 1001) id 13FAF536116; Thu, 10 Aug 2006 22:01:35 +0200 (CEST) Date: Thu, 10 Aug 2006 22:01:35 +0200 From: Jan Srzednicki To: Pawel Jakub Dawidek Message-ID: <20060810200135.GF57857@europa.expro.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810192152.GE57857@europa.expro.pl> <20060810194304.GB1345@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060810194304.GB1345@garage.freebsd.pl> User-Agent: Mutt/1.5.12-2006-07-14 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 20:01:38 -0000 On Thu, Aug 10, 2006 at 09:43:04PM +0200, Pawel Jakub Dawidek wrote: > On Thu, Aug 10, 2006 at 09:21:52PM +0200, Jan Srzednicki wrote: > > > > Hello, > > > > I'd happily perform some testing, only if the aac(4) driver would also > > be patched. I know I can run gjournal w/o the BIO_FLUSH feature, but > > that would make any performance results rather worthles, wouldn't it? > > From my tests BIO_FLUSH doesn't have huge impact on performance (if at > all), but I suspect it greatly depends on specific HW. > > Unfortunately I'm not able to implement BIO_FLUSH to all out storage > drivers. Currently BIO_FLUSH is supported by ata(4) (/dev/a[dr]X), > da(4) and amr(4). I see. Does that mean that all CAM-interfaced devices (like asr(4) or twa(4)) have this feature fully implemented? -- Jan Srzednicki w@expro.pl From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 21:05:38 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9668116A4DE; Thu, 10 Aug 2006 21:05:38 +0000 (UTC) (envelope-from craig@xfoil.gank.org) Received: from ion.gank.org (ion.gank.org [69.55.238.164]) by mx1.FreeBSD.org (Postfix) with ESMTP id 20A0443D46; Thu, 10 Aug 2006 21:05:37 +0000 (GMT) (envelope-from craig@xfoil.gank.org) Received: by ion.gank.org (Postfix, from userid 1001) id 7BC1C117C9; Thu, 10 Aug 2006 16:05:37 -0500 (CDT) Date: Thu, 10 Aug 2006 16:05:35 -0500 From: Craig Boston To: Pawel Jakub Dawidek Message-ID: <20060810210535.GA9065@nowhere> Mail-Followup-To: Craig Boston , Pawel Jakub Dawidek , freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> <20060810192841.GA1345@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060810192841.GA1345@garage.freebsd.pl> User-Agent: Mutt/1.4.2.2i Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 21:05:38 -0000 On Thu, Aug 10, 2006 at 09:28:41PM +0200, Pawel Jakub Dawidek wrote: > When you mount file system it sends BIO_GETATTR "GJOURNAL::provider" > requests. So as long as classes between the file system and gjournal > provider pass BIO_GETATTR down, it will work. > > I prefer to put gjournal on the top, because it gives consistency to > layers below it. [...] > > [snip] > > Fortunatelly with GEOM you can freely mix your puzzles. Would there be any ill effects to putting multiple filesystems inside a single gjournal, say with a bsdlabel or gpt on it? If it worked and didn't confuse the UFS integration, it might save some space by not having multiple (default 1G) journals. I'm unsure if it would improve or degrage performance for concurrent access. Craig From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 21:18:50 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6941216A4DA for ; Thu, 10 Aug 2006 21:18:50 +0000 (UTC) (envelope-from bryants@gmail.com) Received: from nz-out-0102.google.com (nz-out-0102.google.com [64.233.162.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0268743D4C for ; Thu, 10 Aug 2006 21:18:48 +0000 (GMT) (envelope-from bryants@gmail.com) Received: by nz-out-0102.google.com with SMTP id 13so211541nzn for ; Thu, 10 Aug 2006 14:18:48 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=jyqLjnm6xDrgE6SPljexeZivTSL1+pw1erEhc16Qb4Q9PlyRKoBJ+MOebxyNJ81wJxnjkBXJ79pKiSjGVTnk6xdHlrwIQ0iLO7O3jPKo3eYMur0O8dHpoduccGQibgI3aBAYneGtDskMmXyXyBefeKXeaap8wKlR+PH21Lh0rD4= Received: by 10.65.160.7 with SMTP id m7mr3120293qbo; Thu, 10 Aug 2006 14:18:48 -0700 (PDT) Received: by 10.65.20.2 with HTTP; Thu, 10 Aug 2006 14:18:47 -0700 (PDT) Message-ID: Date: Thu, 10 Aug 2006 17:18:47 -0400 From: "Sean Bryant" To: "Pawel Jakub Dawidek" In-Reply-To: <20060808195202.GA1564@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20060808195202.GA1564@garage.freebsd.pl> Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 21:18:50 -0000 On 8/8/06, Pawel Jakub Dawidek wrote: > Hi. > > I'm pleased to announce hopefully the final gjournal patches. > > The biggest change is removal of magic .deleted/ directory. Now orphaned > objects (files/directories which were deleted, but are still open) are > handled in a different way - in every CG there is a counter which tells > how many such orphaned objects are there in the given CG. There is also > a global unreferenced inodes counter in the superblock. > > On power failure or system crash, fast fsck version will be run on > journaled file system which only checks CG for unreferenced inodes. > Multi terabyte file system is checked in few minutes, but it depends on > how many such objects we have. The global counter in the superblock > allows to stop scanning CGs when it goes to 0. > > Another change is that now from a configuration point of view gjournal > is threated simlar to soft-updates - you need to specify -J flag at > newfs(8) time or use 'tunefs -J enable '. 'gjournal' is no longer > a mount option. > > To create journaled UFS file system on ad0s1d partition: > > # gjournal load > # gjournal label ad0s1d > # newfs -J /dev/ad0s1d.journal > # mount -o noatime,async /dev/ad0s1d.journal /mnt > > On reboot 'fsck_ffs -p /dev/ad0s1d.journal' will perform fast check. One > can still run regular check by not giving the -p option. > > BTW. This should allow to have root file system on gjournal partition, > but haven't verified it yet - just a theory. > > Patches for 7-CURRENT: > > http://people.freebsd.org/~pjd/patches/gjournal_20060808.patch > > Patches for 6-STABLE: > > http://people.freebsd.org/~pjd/patches/gjournal6_20060808.patch > > To apply the patches: > > # cd /usr/src > # mkdir sys/geom/journal sbin/geom/class/journal sys/modules/geom/geom_journal > # patch < gjournal_20060808.patch > > Now you need to recompile the world and the kernel. Remember to add > 'options UFS_GJOURNAL' to your kernel configuration file. > > If there will be no bug reports nor strong objections, I'd like commit > gjournal in a week or two first to 7-CURRENT and then to 6-STABLE (maybe > before 6.2-RELEASE, but it depends on testing and feedback from users). > > Enjoy! > > -- > Pawel Jakub Dawidek http://www.wheel.pl > pjd@FreeBSD.org http://www.FreeBSD.org > FreeBSD committer Am I Evil? Yes, I Am! > > > Okay I have applied the latest patches. And everything works fine on a new filesystem (newfsing). Trying to add it to an existing file system I'm getting problems. This might be my fault. So I'll give it another go before reporting back my problems. -- Sean Bryant From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 22:30:45 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5746D16A4DD; Thu, 10 Aug 2006 22:30:45 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 20F8E43D49; Thu, 10 Aug 2006 22:30:43 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id A05625138A; Fri, 11 Aug 2006 00:30:41 +0200 (CEST) Received: from localhost (dkd162.neoplus.adsl.tpnet.pl [83.24.7.162]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 571F25131F; Fri, 11 Aug 2006 00:30:33 +0200 (CEST) Date: Fri, 11 Aug 2006 00:29:42 +0200 From: Pawel Jakub Dawidek To: Craig Boston , freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Message-ID: <20060810222942.GA1389@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> <20060810192841.GA1345@garage.freebsd.pl> <20060810210535.GA9065@nowhere> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="opJtzjQTFsWo+cga" Content-Disposition: inline In-Reply-To: <20060810210535.GA9065@nowhere> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 22:30:45 -0000 --opJtzjQTFsWo+cga Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Aug 10, 2006 at 04:05:35PM -0500, Craig Boston wrote: > On Thu, Aug 10, 2006 at 09:28:41PM +0200, Pawel Jakub Dawidek wrote: > > When you mount file system it sends BIO_GETATTR "GJOURNAL::provider" > > requests. So as long as classes between the file system and gjournal > > provider pass BIO_GETATTR down, it will work. > >=20 > > I prefer to put gjournal on the top, because it gives consistency to > > layers below it. [...] > > > > [snip] > >=20 > > Fortunatelly with GEOM you can freely mix your puzzles. >=20 > Would there be any ill effects to putting multiple filesystems inside a > single gjournal, say with a bsdlabel or gpt on it? I don't see any problems with such configuration, but there are no optimization like "synchronize all file systems and then switch journals", so it will go like "synchronize one file system switch journals, synchronize another file system, switch journals, etc.". --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --opJtzjQTFsWo+cga Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE27NWForvXbEpPzQRAgIBAKDdvEiLIsHM/Pgfh4WfDe1ruU4p0ACggHvi 0XASFxTKlzxDY2tYBtHeiTM= =9MBA -----END PGP SIGNATURE----- --opJtzjQTFsWo+cga-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 22:37:33 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 386D316A4DD; Thu, 10 Aug 2006 22:37:33 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 84DB643D46; Thu, 10 Aug 2006 22:37:32 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 633145138A; Fri, 11 Aug 2006 00:37:31 +0200 (CEST) Received: from localhost (dkd162.neoplus.adsl.tpnet.pl [83.24.7.162]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id E03635131F; Fri, 11 Aug 2006 00:37:26 +0200 (CEST) Date: Fri, 11 Aug 2006 00:36:37 +0200 From: Pawel Jakub Dawidek To: Craig Boston , freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Message-ID: <20060810223637.GB1389@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> <20060810192841.GA1345@garage.freebsd.pl> <20060810210535.GA9065@nowhere> <20060810222942.GA1389@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="eJnRUKwClWJh1Khz" Content-Disposition: inline In-Reply-To: <20060810222942.GA1389@garage.freebsd.pl> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 22:37:33 -0000 --eJnRUKwClWJh1Khz Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 11, 2006 at 12:29:42AM +0200, Pawel Jakub Dawidek wrote: > On Thu, Aug 10, 2006 at 04:05:35PM -0500, Craig Boston wrote: > > On Thu, Aug 10, 2006 at 09:28:41PM +0200, Pawel Jakub Dawidek wrote: > > > When you mount file system it sends BIO_GETATTR "GJOURNAL::provider" > > > requests. So as long as classes between the file system and gjournal > > > provider pass BIO_GETATTR down, it will work. > > >=20 > > > I prefer to put gjournal on the top, because it gives consistency to > > > layers below it. [...] > > > > > > [snip] > > >=20 > > > Fortunatelly with GEOM you can freely mix your puzzles. > >=20 > > Would there be any ill effects to putting multiple filesystems inside a > > single gjournal, say with a bsdlabel or gpt on it? >=20 > I don't see any problems with such configuration, but there are no > optimization like "synchronize all file systems and then switch > journals", so it will go like "synchronize one file system switch > journals, synchronize another file system, switch journals, etc.". Ok, on second thought it won't work. If you have many file systems on one gjournal, you synchronize one file system and switch journals leaving the rest of file systems in an inconsistent state. To make it work reliable I'd need to implement the optimization I mentioned, ie. sync all file systems and then switch journals. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --eJnRUKwClWJh1Khz Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE27T1ForvXbEpPzQRAnwYAJ412dgP+UIfoEgtDD7jrx+ot3UnVwCg81XB H1HlYHamfl2nw/yCB7wK7PI= =qDh1 -----END PGP SIGNATURE----- --eJnRUKwClWJh1Khz-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 22:39:57 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3428B16A4E5; Thu, 10 Aug 2006 22:39:57 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8E38743D46; Thu, 10 Aug 2006 22:39:55 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 468B451392; Fri, 11 Aug 2006 00:39:54 +0200 (CEST) Received: from localhost (dkd162.neoplus.adsl.tpnet.pl [83.24.7.162]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 685045138F; Fri, 11 Aug 2006 00:39:50 +0200 (CEST) Date: Fri, 11 Aug 2006 00:39:01 +0200 From: Pawel Jakub Dawidek To: Jan Srzednicki Message-ID: <20060810223901.GC1389@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810192152.GE57857@europa.expro.pl> <20060810194304.GB1345@garage.freebsd.pl> <20060810200135.GF57857@europa.expro.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2JFBq9zoW8cOFH7v" Content-Disposition: inline In-Reply-To: <20060810200135.GF57857@europa.expro.pl> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 22:39:57 -0000 --2JFBq9zoW8cOFH7v Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Aug 10, 2006 at 10:01:35PM +0200, Jan Srzednicki wrote: > On Thu, Aug 10, 2006 at 09:43:04PM +0200, Pawel Jakub Dawidek wrote: > > On Thu, Aug 10, 2006 at 09:21:52PM +0200, Jan Srzednicki wrote: > > >=20 > > > Hello, > > >=20 > > > I'd happily perform some testing, only if the aac(4) driver would also > > > be patched. I know I can run gjournal w/o the BIO_FLUSH feature, but > > > that would make any performance results rather worthles, wouldn't it? > >=20 > > From my tests BIO_FLUSH doesn't have huge impact on performance (if at > > all), but I suspect it greatly depends on specific HW. > >=20 > > Unfortunately I'm not able to implement BIO_FLUSH to all out storage > > drivers. Currently BIO_FLUSH is supported by ata(4) (/dev/a[dr]X), > > da(4) and amr(4). >=20 > I see. Does that mean that all CAM-interfaced devices (like asr(4) or > twa(4)) have this feature fully implemented? That's the theory. I'm using standard SCSI commands for this. From the tests we made it seems that it works ok with twa(4)/twe(4). I'd still be glad if someone with SCSI/CAM foo could look at the code. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --2JFBq9zoW8cOFH7v Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE27WFForvXbEpPzQRAsFkAJ46DsXZV5L5AUJiHiS7eiB0rdAFowCeOdTb r2YyP364McN1aci/YjNokQs= =XR71 -----END PGP SIGNATURE----- --2JFBq9zoW8cOFH7v-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 22:54:07 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CA77816A4DF for ; Thu, 10 Aug 2006 22:54:07 +0000 (UTC) (envelope-from bryants@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.179]) by mx1.FreeBSD.org (Postfix) with ESMTP id 709EC43D6D for ; Thu, 10 Aug 2006 22:54:05 +0000 (GMT) (envelope-from bryants@gmail.com) Received: by py-out-1112.google.com with SMTP id c59so1100708pyc for ; Thu, 10 Aug 2006 15:54:04 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=XVLRl63MpCDcXhrAo2IXweB0uQ23Eq8vTxJoeYXJi8y0rijMTg3WPty1Yy6pkrkfjBL+HEvp/AJl1sK5su04zvFblsAGicyluLRXfRCGxsnRsHZJCvnpRsIaiGOPeJat0vbbRt13dibUrK/AjwdXdP3O7rHQeb2FOGwY7sugoa8= Received: by 10.65.211.16 with SMTP id n16mr3224631qbq; Thu, 10 Aug 2006 15:54:04 -0700 (PDT) Received: by 10.65.20.2 with HTTP; Thu, 10 Aug 2006 15:54:04 -0700 (PDT) Message-ID: Date: Thu, 10 Aug 2006 18:54:04 -0400 From: "Sean Bryant" To: "Pawel Jakub Dawidek" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20060808195202.GA1564@garage.freebsd.pl> Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 22:54:07 -0000 On 8/10/06, Sean Bryant wrote: > On 8/8/06, Pawel Jakub Dawidek wrote: > > Hi. > > > > I'm pleased to announce hopefully the final gjournal patches. > > > > The biggest change is removal of magic .deleted/ directory. Now orphaned > > objects (files/directories which were deleted, but are still open) are > > handled in a different way - in every CG there is a counter which tells > > how many such orphaned objects are there in the given CG. There is also > > a global unreferenced inodes counter in the superblock. > > > > On power failure or system crash, fast fsck version will be run on > > journaled file system which only checks CG for unreferenced inodes. > > Multi terabyte file system is checked in few minutes, but it depends on > > how many such objects we have. The global counter in the superblock > > allows to stop scanning CGs when it goes to 0. > > > > Another change is that now from a configuration point of view gjournal > > is threated simlar to soft-updates - you need to specify -J flag at > > newfs(8) time or use 'tunefs -J enable '. 'gjournal' is no longer > > a mount option. > > > > To create journaled UFS file system on ad0s1d partition: > > > > # gjournal load > > # gjournal label ad0s1d > > # newfs -J /dev/ad0s1d.journal > > # mount -o noatime,async /dev/ad0s1d.journal /mnt > > > > On reboot 'fsck_ffs -p /dev/ad0s1d.journal' will perform fast check. One > > can still run regular check by not giving the -p option. > > > > BTW. This should allow to have root file system on gjournal partition, > > but haven't verified it yet - just a theory. > > > > Patches for 7-CURRENT: > > > > http://people.freebsd.org/~pjd/patches/gjournal_20060808.patch > > > > Patches for 6-STABLE: > > > > http://people.freebsd.org/~pjd/patches/gjournal6_20060808.patch > > > > To apply the patches: > > > > # cd /usr/src > > # mkdir sys/geom/journal sbin/geom/class/journal sys/modules/geom/geom_journal > > # patch < gjournal_20060808.patch > > > > Now you need to recompile the world and the kernel. Remember to add > > 'options UFS_GJOURNAL' to your kernel configuration file. > > > > If there will be no bug reports nor strong objections, I'd like commit > > gjournal in a week or two first to 7-CURRENT and then to 6-STABLE (maybe > > before 6.2-RELEASE, but it depends on testing and feedback from users). > > > > Enjoy! > > > > -- > > Pawel Jakub Dawidek http://www.wheel.pl > > pjd@FreeBSD.org http://www.FreeBSD.org > > FreeBSD committer Am I Evil? Yes, I Am! > > > > > > > > Okay I have applied the latest patches. And everything works fine on a > new filesystem (newfsing). > > Trying to add it to an existing file system I'm getting problems. This > might be my fault. So I'll give it another go before reporting back my > problems. > > -- > Sean Bryant > Alright. After testing I've found the problem. You can enable journaling on an existing filesystem, provided the journaling provider is not the data provider. I could not do the following: tunefs -J enable ad4s1e gjournal label ad4s1e fsck_ffs -p ad4s1e I would get cannot read BLK XXXX. And various other problems. I don't see where you've actually stated this. I did have much success with the following: tunefs -j enable ad4s1e gjournal label ad4s1e ad6 (which is an used volume) fsck_ffs -p ad4s1e.journal I should note I tried this in a varity of ways. With only the previous resulting in a success existing filesystem using gjournal. During each of the trials I would set the configuration back to the original before attemping any more experimenation. Results: [failed] tunefs -J enable ad4s1e gjournal label ad4s1e fsck_ffs -p ad4s1e // failed [failed] gjournal label ad4s1e tunefs -J enable ad4s1e fsck_ffs -p ad4s1e // failed [failed] gjournal label ad4s1e tunefs -J enable ad4s1e.journal fsck_ffs -p ad4s1e // falied [passes and verified] tunefs -j enable ad4s1e gjournal label ad4s1e ad6 -- Sean Bryant From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 23:23:51 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7A77D16A4DA; Thu, 10 Aug 2006 23:23:51 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9D16D43D4C; Thu, 10 Aug 2006 23:23:50 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4D5B85138F; Fri, 11 Aug 2006 01:23:49 +0200 (CEST) Received: from localhost (dkd162.neoplus.adsl.tpnet.pl [83.24.7.162]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 3679C5133B; Fri, 11 Aug 2006 01:23:44 +0200 (CEST) Date: Fri, 11 Aug 2006 01:22:55 +0200 From: Pawel Jakub Dawidek To: Sean Bryant Message-ID: <20060810232255.GE1389@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EXKGNeO8l0xGFBjy" Content-Disposition: inline In-Reply-To: X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 23:23:51 -0000 --EXKGNeO8l0xGFBjy Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Aug 10, 2006 at 06:54:04PM -0400, Sean Bryant wrote: > On 8/10/06, Sean Bryant wrote: > Alright. After testing I've found the problem. You can enable > journaling on an existing filesystem, provided the journaling provider > is not the data provider. >=20 > I could not do the following: > tunefs -J enable ad4s1e > gjournal label ad4s1e > fsck_ffs -p ad4s1e Yes, maybe I wasn't clear on this. I think I described it only slightly in my first gjournal announce. GJournal is not file system journaling, it needs separate journal space. If you 'gjournal label' only one partition, it will put journal at the end on this partition and leave space for data at the begining of the partition. To convert existing file system to gjournal you have to: 1. Have separate partition for journal. 2. Confirm that partition with your file system is _not_ 4 sectors aligned. If it is not 4 sectors aligned, it is safe for gjournal to use its last sector for metadata. You can verify this by running: test `diskinfo /dev/ | awk '{print $4}' | xargs -J X echo = X % 4 | bc` -eq 0 && echo ok || echo not ok Then you need to run those commands: # gjournal label /dev/ /dev/ # tunefs -n disable -J enable /dev/.journal --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --EXKGNeO8l0xGFBjy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE27/OForvXbEpPzQRAjK9AKDgZYIDACmY9bnt+4V18PW8FbfU1wCeKPNE b18rY78wkqV6tk7KNywzZZw= =Ns5V -----END PGP SIGNATURE----- --EXKGNeO8l0xGFBjy-- From owner-freebsd-fs@FreeBSD.ORG Thu Aug 10 23:27:41 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0290616A4DD for ; Thu, 10 Aug 2006 23:27:41 +0000 (UTC) (envelope-from bryants@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.183]) by mx1.FreeBSD.org (Postfix) with ESMTP id A747A43D53 for ; Thu, 10 Aug 2006 23:27:39 +0000 (GMT) (envelope-from bryants@gmail.com) Received: by py-out-1112.google.com with SMTP id c59so1113428pyc for ; Thu, 10 Aug 2006 16:27:38 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=fJU1zmTxkU9H0gspXbIN6SaR5NxIyrp9oQsi0IDQtd7X4ApZAjgn3sjfYSYLXxCOxlUjI40CYdT5CuZHLpy8jGGZKAH3FngZqQzKX6rFdQjZ08GOGr/i4uQPkmn67xL6uA4gS1Vr3Bo/u6CESESiP6vKdzKLdsL3lGNoZOVI2FA= Received: by 10.65.95.14 with SMTP id x14mr3198801qbl; Thu, 10 Aug 2006 16:27:38 -0700 (PDT) Received: by 10.65.20.2 with HTTP; Thu, 10 Aug 2006 16:27:38 -0700 (PDT) Message-ID: Date: Thu, 10 Aug 2006 19:27:38 -0400 From: "Sean Bryant" To: "Pawel Jakub Dawidek" In-Reply-To: <20060810232255.GE1389@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20060808195202.GA1564@garage.freebsd.pl> <20060810232255.GE1389@garage.freebsd.pl> Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 23:27:41 -0000 On 8/10/06, Pawel Jakub Dawidek wrote: > On Thu, Aug 10, 2006 at 06:54:04PM -0400, Sean Bryant wrote: > > On 8/10/06, Sean Bryant wrote: > > Alright. After testing I've found the problem. You can enable > > journaling on an existing filesystem, provided the journaling provider > > is not the data provider. > > > > I could not do the following: > > tunefs -J enable ad4s1e > > gjournal label ad4s1e > > fsck_ffs -p ad4s1e > > Yes, maybe I wasn't clear on this. I think I described it only slightly > in my first gjournal announce. > > GJournal is not file system journaling, it needs separate journal space. > If you 'gjournal label' only one partition, it will put journal at the > end on this partition and leave space for data at the begining of the > partition. > > To convert existing file system to gjournal you have to: > 1. Have separate partition for journal. > 2. Confirm that partition with your file system is _not_ 4 sectors > aligned. If it is not 4 sectors aligned, it is safe for gjournal to > use its last sector for metadata. You can verify this by running: > > test `diskinfo /dev/ | awk '{print $4}' | xargs -J X echo X % 4 | bc` -eq 0 && echo ok || echo not ok > > Then you need to run those commands: > > # gjournal label /dev/ /dev/ > # tunefs -n disable -J enable /dev/.journal > > -- > Pawel Jakub Dawidek http://www.wheel.pl > pjd@FreeBSD.org http://www.FreeBSD.org > FreeBSD committer Am I Evil? Yes, I Am! > > > Okay that clears it up a bit. I thought it would adjust the size available or something. By putting the journal at the end. Okay thanks :) But it works. and well. -- Sean Bryant From owner-freebsd-fs@FreeBSD.ORG Fri Aug 11 02:49:22 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B44B316A4DF; Fri, 11 Aug 2006 02:49:22 +0000 (UTC) (envelope-from jd@ugcs.caltech.edu) Received: from mark.ugcs.caltech.edu (mark.ugcs.caltech.edu [131.215.176.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5B0D443D45; Fri, 11 Aug 2006 02:49:22 +0000 (GMT) (envelope-from jd@ugcs.caltech.edu) Received: by mark.ugcs.caltech.edu (Postfix, from userid 3640) id DAA7F3F050; Thu, 10 Aug 2006 19:49:21 -0700 (PDT) Date: Thu, 10 Aug 2006 19:49:21 -0700 From: Paul Allen To: Pawel Jakub Dawidek Message-ID: <20060811024921.GF308@mark.ugcs.caltech.edu> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> <20060810192841.GA1345@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060810192841.GA1345@garage.freebsd.pl> Sender: jd@ugcs.caltech.edu Cc: freebsd-fs@freebsd.org, Craig Boston , freebsd-geom@freebsd.org, freebsd-arch@freebsd.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2006 02:49:22 -0000 It's a bit disturbing that a geom-class quite far away from the storage drivers presumes that the proper action here is a cache flush. The underlying hardware may support tagged command queuing (i.e., SCSIs ability to receive not only transaction completion notications but also to permit partial-orderings to be dictated to the controller) or native-command queuing (command completion). It's true that this functionality may not always work as advertised but that's a problem to be solved with dev. sysctls, not by taking a LCD approach in a high-level geom class. This really needs broader architecture consideration, not just what it takes it make it work. Paul >From Pawel Jakub Dawidek , Thu, Aug 10, 2006 at 09:28:41PM +0200: > On Thu, Aug 10, 2006 at 01:47:23PM -0500, Craig Boston wrote: > > Hi, > > > > It's great to see this project so close to completion! I'm trying it > > out on a couple machines to see how it goes. > > > > A few comments and questions: > > > > * It took me a little by surprise that it carves 1G out of the device > > for the journal. Depending on the size of the device that can be a > > pretty hefty price to pay (and I didn't see any mention of it in the > > setup notes). For a couple of my smaller filesystems I reduced it to > > 512MB. Perhaps some algorithm for auto-sizing the journal based on > > the size / expected workload of the device would be in order? > > It will be pointed out in documentation when I finally prepare it. > I don't have plans about autosizing currently. > > > * Attached is a quick patch for geom_eli to allow it to pass BIO_FLUSH > > down to its backing device. It seems like the right thing to do and > > fixes the "BIO_FLUSH not supported" warning on my laptop that uses a > > geli encrypted disk. > > I've this already in my perforce tree. I also implemented BIO_FLUSH > passing in gmirror and graid3. > > I also added a flag for gmirror and graid3 which says "don't > resynchronize components after a power failure - trust they are > consistent". And they are always consistent when placed below gjournal. > > > * On a different system, however, it complains about it even on a raw > > ATA slice: > > > > atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0 > > ata0: on atapci1 > > ad0: 114473MB at ata0-master UDMA100 > > GEOM_JOURNAL: BIO_FLUSH not supported by ad0s1e. > > > > It seems like a reasonably modern controller and disk, at least it > > should be capable of issuing a cache flush command. Not sure why it > > doesn't like it :/ > > We would need to add some printfs to diagnoze this probably - you can > try adding some lines to ad_init() to get this: > > if (atadev->param.support.command1 & ATA_SUPPORT_WRITECACHE) { > if (ata_wc) > ata_controlcmd(dev, ATA_SETFEATURES, ATA_SF_ENAB_WCACHE, 0, 0); > else > ata_controlcmd(dev, ATA_SETFEATURES, ATA_SF_DIS_WCACHE, 0, 0); > } else { > printf("ad_init: WRITE CACHE not supported by ad%d.\n", > device_get_unit(dev)); > } > > > * How "close" does the filesystem need to be to the gjournal device in > > order for the UFS hooks to work? Directly on it? > > > > The geom stack on my laptop currently looks something like this: > > > > [geom_disk] ad0 <- [geom_eli] ad0.eli <- [geom_gpt] ad0.elip6 <- > > [geom_label] gjtest <- [geom_journal] gjtest.journal <- UFS > > > > I was wondering if an arrangement like this would work: > > > > [geom_journal] ad0p6.journal <- [geom_eli] ad0p6.journaleli <- UFS > > > > and if it would be any more efficient (journal the encrypted data > > rather than encrypt the journal). Or even gjournal the whole disk at > > once? > > When you mount file system it sends BIO_GETATTR "GJOURNAL::provider" > requests. So as long as classes between the file system and gjournal > provider pass BIO_GETATTR down, it will work. > > On my home machine I've the following configuration: > > raid3/DATA1.elid.journal > > So it's UFS over gjournal over bsdlabel over geli over raid3 over ata. > > I prefer to put gjournal on the top, because it gives consistency to > layers below it. For example I can use geli with bigger sector size > (sector size greater than disk sector size in encryption-only-mode can > be unreliable on power failures, which is not the case when gjournal is > above geli), I can turn off synchronization of gmirror/graid3 after a > power failure, etc. > > On the other hand configuring geli on top of gjournal can be more > effective for large files - geli will not encrypt the data twice. > > Fortunatelly with GEOM you can freely mix your puzzles. > > > Haven't been brave enough to try gjournal on root yet, but my /usr and > > /compile (src, obj, ports) partitions are already on it so I'm sure I'll > > try it soon ;) > > Markus Trippelsdorf reported that it doesn't work out of the box, but he > manage to make it to work with some small changes to fsck_ffs(8). > > -- > Pawel Jakub Dawidek http://www.wheel.pl > pjd@FreeBSD.org http://www.FreeBSD.org > FreeBSD committer Am I Evil? Yes, I Am! From owner-freebsd-fs@FreeBSD.ORG Fri Aug 11 04:02:10 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 684EC16A4DD for ; Fri, 11 Aug 2006 04:02:10 +0000 (UTC) (envelope-from dennisolvany@gmail.com) Received: from wx-out-0506.google.com (wx-out-0506.google.com [66.249.82.225]) by mx1.FreeBSD.org (Postfix) with ESMTP id E3BE543D46 for ; Fri, 11 Aug 2006 04:02:09 +0000 (GMT) (envelope-from dennisolvany@gmail.com) Received: by wx-out-0506.google.com with SMTP id i27so576090wxd for ; Thu, 10 Aug 2006 21:02:09 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:subject:from:to:content-type:date:message-id:mime-version:x-mailer:content-transfer-encoding; b=QAxDPvZ8s2SaqwoBeL2UmTjXW5624CwjOF82sRaxTepVhx1ig7aDWZnP6uJqhz9vMTZAXr7SnxnG3/DyBzX8QkKuYEgLOnzM6rUQ2M1dFVkFGhiVssZF67q5gLesBAjqIeA6o28/ewMbkx1Kol9qOY6Pj40vjsVxnnU+ccWFMUY= Received: by 10.70.44.5 with SMTP id r5mr4282407wxr; Thu, 10 Aug 2006 21:00:28 -0700 (PDT) Received: from dm.deadghost.com ( [195.16.87.34]) by mx.gmail.com with ESMTP id 10sm1560249wrl.2006.08.10.21.00.26; Thu, 10 Aug 2006 21:00:27 -0700 (PDT) From: Dennis Olvany To: freebsd-fs@freebsd.org Content-Type: text/plain Date: Thu, 10 Aug 2006 23:00:23 -0500 Message-Id: <1155268823.41063.4.camel@dm.deadghost.com> Mime-Version: 1.0 X-Mailer: Evolution 2.6.2 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Subject: smbfs.ko: netbios-dependent X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2006 04:02:10 -0000 Far as I can determine, the smbfs module does not support hostnames or ip addresses. This presents a severe limitation for using smbfs over an internetwork. At the very least, I'm wondering if it supports static mappings from a local file. Are there any plans to expand functionality in this area? From owner-freebsd-fs@FreeBSD.ORG Fri Aug 11 06:43:44 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4244916A4DD for ; Fri, 11 Aug 2006 06:43:44 +0000 (UTC) (envelope-from redchin@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.177]) by mx1.FreeBSD.org (Postfix) with ESMTP id BB4C443D45 for ; Fri, 11 Aug 2006 06:43:43 +0000 (GMT) (envelope-from redchin@gmail.com) Received: by py-out-1112.google.com with SMTP id c59so25510pyc for ; Thu, 10 Aug 2006 23:43:43 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=dnxSXG4KJyVko1+mb10N6kJItwXjwBcN2ng0IMjzRePi9lJe96OdnOUYJAGgdwEsQhkYVsF4LYRMC2FeGP8J5DAHCIl7g9xmc4HKGqlylH9pKHkTZL3DJAMNcFdF44C/DF7dYczuDL/+Ry3PyDcLQgz4+X7coa1nFhuqIEtZucM= Received: by 10.65.112.5 with SMTP id p5mr3496721qbm; Thu, 10 Aug 2006 23:43:42 -0700 (PDT) Received: by 10.64.114.3 with HTTP; Thu, 10 Aug 2006 23:43:42 -0700 (PDT) Message-ID: <1d3ed48c0608102343w2025e46bx7361287c415d6819@mail.gmail.com> Date: Thu, 10 Aug 2006 23:43:42 -0700 From: "Kevin Downey" To: "Dennis Olvany" In-Reply-To: <1155268823.41063.4.camel@dm.deadghost.com> MIME-Version: 1.0 References: <1155268823.41063.4.camel@dm.deadghost.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: smbfs.ko: netbios-dependent X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2006 06:43:44 -0000 On 8/10/06, Dennis Olvany wrote: > > Far as I can determine, the smbfs module does not support hostnames or > ip addresses. This presents a severe limitation for using smbfs over an > internetwork. At the very least, I'm wondering if it supports static > mappings from a local file. Are there any plans to expand functionality > in this area? > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > I would recommend reading the mount_smbfs manpage -- luctor et emergo From owner-freebsd-fs@FreeBSD.ORG Fri Aug 11 13:33:45 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9E85A16A4E2; Fri, 11 Aug 2006 13:33:45 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4F94743D45; Fri, 11 Aug 2006 13:33:45 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 8F93046D1A; Fri, 11 Aug 2006 09:33:44 -0400 (EDT) Date: Fri, 11 Aug 2006 14:33:44 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Pawel Jakub Dawidek In-Reply-To: <20060810232255.GE1389@garage.freebsd.pl> Message-ID: <20060811143257.E2260@fledge.watson.org> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810232255.GE1389@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, Sean Bryant , freebsd-geom@freebsd.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2006 13:33:45 -0000 On Fri, 11 Aug 2006, Pawel Jakub Dawidek wrote: > 2. Confirm that partition with your file system is _not_ 4 sectors > aligned. If it is not 4 sectors aligned, it is safe for gjournal to > use its last sector for metadata. You can verify this by running: > > test `diskinfo /dev/ | awk '{print $4}' | xargs -J X echo X % 4 | bc` -eq 0 && echo ok || echo not ok > > Then you need to run those commands: > > # gjournal label /dev/ /dev/ > # tunefs -n disable -J enable /dev/.journal Sounds like a "gjournal testalignment /dev/partition" would be a very useful thing. Or even just doing it as part of gjournal label? Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-fs@FreeBSD.ORG Fri Aug 11 13:51:44 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 30BB216A4DA for ; Fri, 11 Aug 2006 13:51:44 +0000 (UTC) (envelope-from dennisolvany@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F8FD43D55 for ; Fri, 11 Aug 2006 13:51:40 +0000 (GMT) (envelope-from dennisolvany@gmail.com) Received: by ug-out-1314.google.com with SMTP id m2so458784ugc for ; Fri, 11 Aug 2006 06:51:39 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=R6PyIjXhOkpIMS6yJ+LCGSmVXmYNQ/LX+TBtnMBPcfZQSjAXzpbKLwxdIbi20+WFCH2jC02BO+30vo7fUKZ2DS7OJ9cdic3zjU7Kt+VApkNKE8IembeRg+V8/Obrcpt30VkGi7hGkScrTmCyIoRn0/cupSkbUg5avd6nCUzari0= Received: by 10.67.89.5 with SMTP id r5mr4090185ugl; Fri, 11 Aug 2006 06:51:39 -0700 (PDT) Received: by 10.66.236.10 with HTTP; Fri, 11 Aug 2006 06:51:39 -0700 (PDT) Message-ID: Date: Fri, 11 Aug 2006 08:51:39 -0500 From: "Dennis Olvany" To: "Kevin Downey" In-Reply-To: <1d3ed48c0608102343w2025e46bx7361287c415d6819@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <1155268823.41063.4.camel@dm.deadghost.com> <1d3ed48c0608102343w2025e46bx7361287c415d6819@mail.gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: smbfs.ko: netbios-dependent X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2006 13:51:44 -0000 > On 8/10/06, Dennis Olvany wrote: > Far as I can determine, the smbfs module does not support hostnames or > ip addresses. > On 8/11/06, Kevin Downey wrote: > I would recommend reading the mount_smbfs manpage Thanks for pointing that out. There does indeed seem to be limited support for hostnames/ip addresses. Is this usage at all supported in fstab? From owner-freebsd-fs@FreeBSD.ORG Fri Aug 11 13:54:30 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A51C316A4DA; Fri, 11 Aug 2006 13:54:30 +0000 (UTC) (envelope-from thomas@FreeBSD.ORG) Received: from melamine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [82.225.155.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1494543D45; Fri, 11 Aug 2006 13:54:30 +0000 (GMT) (envelope-from thomas@FreeBSD.ORG) Received: by melamine.cuivre.fr.eu.org (Postfix, from userid 1000) id ABCD85C642; Fri, 11 Aug 2006 15:54:26 +0200 (CEST) Date: Fri, 11 Aug 2006 15:54:26 +0200 From: Thomas Quinot To: Pawel Jakub Dawidek Message-ID: <20060811135426.GA56740@melamine.cuivre.fr.eu.org> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> <20060810192841.GA1345@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060810192841.GA1345@garage.freebsd.pl> X-message-flag: WARNING! Using Outlook can damage your computer. User-Agent: Mutt/1.5.11 Cc: freebsd-fs@FreeBSD.org, Craig Boston , freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2006 13:54:30 -0000 * Pawel Jakub Dawidek, 2006-08-10 : > I also added a flag for gmirror and graid3 which says "don't > resynchronize components after a power failure - trust they are > consistent". And they are always consistent when placed below gjournal. That sounds a bit suprising; how does gjournal guarantee that a given write has been committed to the two providers of a mirror? Thomas. From owner-freebsd-fs@FreeBSD.ORG Fri Aug 11 14:00:23 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8214C16A4DE; Fri, 11 Aug 2006 14:00:23 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2075F43D5A; Fri, 11 Aug 2006 14:00:15 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 798BD5138F; Fri, 11 Aug 2006 16:00:14 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 868A35133B; Fri, 11 Aug 2006 16:00:10 +0200 (CEST) Date: Fri, 11 Aug 2006 15:59:19 +0200 From: Pawel Jakub Dawidek To: Thomas Quinot Message-ID: <20060811135919.GB48792@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> <20060810192841.GA1345@garage.freebsd.pl> <20060811135426.GA56740@melamine.cuivre.fr.eu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oC1+HKm2/end4ao3" Content-Disposition: inline In-Reply-To: <20060811135426.GA56740@melamine.cuivre.fr.eu.org> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, Craig Boston , freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2006 14:00:24 -0000 --oC1+HKm2/end4ao3 Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 11, 2006 at 03:54:26PM +0200, Thomas Quinot wrote: > * Pawel Jakub Dawidek, 2006-08-10 : >=20 > > I also added a flag for gmirror and graid3 which says "don't > > resynchronize components after a power failure - trust they are > > consistent". And they are always consistent when placed below gjournal. >=20 > That sounds a bit suprising; how does gjournal guarantee that a given > write has been committed to the two providers of a mirror? If the given write wasn't committed to all mirror components this bascially means, that we had a power failure and gjournal will not be clean, so it will be replayed or forgotten, leaving the data on mirror in a consistent state. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --oC1+HKm2/end4ao3 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE3I03ForvXbEpPzQRAgfBAJ9rPoeM+bdZE+kSNGZzw25CZDgATgCcC5wX fQ9rrXN1ycltmfqLY1LmTSk= =ePCK -----END PGP SIGNATURE----- --oC1+HKm2/end4ao3-- From owner-freebsd-fs@FreeBSD.ORG Fri Aug 11 14:03:49 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9773D16A580; Fri, 11 Aug 2006 14:03:49 +0000 (UTC) (envelope-from thomas@FreeBSD.ORG) Received: from melamine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [82.225.155.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id EF96943D4C; Fri, 11 Aug 2006 14:03:48 +0000 (GMT) (envelope-from thomas@FreeBSD.ORG) Received: by melamine.cuivre.fr.eu.org (Postfix, from userid 1000) id 44C795C642; Fri, 11 Aug 2006 16:03:48 +0200 (CEST) Date: Fri, 11 Aug 2006 16:03:48 +0200 From: Thomas Quinot To: Pawel Jakub Dawidek Message-ID: <20060811140348.GB56740@melamine.cuivre.fr.eu.org> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> <20060810192841.GA1345@garage.freebsd.pl> <20060811135426.GA56740@melamine.cuivre.fr.eu.org> <20060811135919.GB48792@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060811135919.GB48792@garage.freebsd.pl> X-message-flag: WARNING! Using Outlook can damage your computer. User-Agent: Mutt/1.5.11 Cc: freebsd-fs@FreeBSD.org, Craig Boston , freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2006 14:03:49 -0000 * Pawel Jakub Dawidek, 2006-08-11 : > If the given write wasn't committed to all mirror components this > bascially means, that we had a power failure and gjournal will not be > clean, so it will be replayed or forgotten, leaving the data on mirror > in a consistent state. Thanks for the clarification. So this means that the mirror might be transiently inconsistent, but will be consistent again as soon as the journal has been replayed? Thomas. From owner-freebsd-fs@FreeBSD.ORG Fri Aug 11 14:06:50 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9BE4F16A4DA; Fri, 11 Aug 2006 14:06:50 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id A11D843D4C; Fri, 11 Aug 2006 14:06:49 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id B3CAD5138F; Fri, 11 Aug 2006 16:06:48 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 54DA25133B; Fri, 11 Aug 2006 16:06:44 +0200 (CEST) Date: Fri, 11 Aug 2006 16:05:53 +0200 From: Pawel Jakub Dawidek To: Thomas Quinot Message-ID: <20060811140553.GC48792@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810184702.GA8567@nowhere> <20060810192841.GA1345@garage.freebsd.pl> <20060811135426.GA56740@melamine.cuivre.fr.eu.org> <20060811135919.GB48792@garage.freebsd.pl> <20060811140348.GB56740@melamine.cuivre.fr.eu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ghzN8eJ9Qlbqn3iT" Content-Disposition: inline In-Reply-To: <20060811140348.GB56740@melamine.cuivre.fr.eu.org> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, Craig Boston , freebsd-current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2006 14:06:50 -0000 --ghzN8eJ9Qlbqn3iT Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 11, 2006 at 04:03:48PM +0200, Thomas Quinot wrote: > * Pawel Jakub Dawidek, 2006-08-11 : >=20 > > If the given write wasn't committed to all mirror components this > > bascially means, that we had a power failure and gjournal will not be > > clean, so it will be replayed or forgotten, leaving the data on mirror > > in a consistent state. >=20 > Thanks for the clarification. So this means that the mirror might be > transiently inconsistent, but will be consistent again as soon as the > journal has been replayed? That's right. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --ghzN8eJ9Qlbqn3iT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE3I7BForvXbEpPzQRAh2EAJwLNrYcuaox1Wd1k4VTsC/LCSPVuwCguoBK sfZIhamYXALZcfPvOjfDPbs= =cm9C -----END PGP SIGNATURE----- --ghzN8eJ9Qlbqn3iT-- From owner-freebsd-fs@FreeBSD.ORG Sat Aug 12 10:04:30 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B401316A4DF; Sat, 12 Aug 2006 10:04:30 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 00F3043D45; Sat, 12 Aug 2006 10:04:29 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 2FB9551392; Sat, 12 Aug 2006 12:04:28 +0200 (CEST) Received: from localhost (dkx242.neoplus.adsl.tpnet.pl [83.24.27.242]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 5FC2151339; Sat, 12 Aug 2006 12:04:23 +0200 (CEST) Date: Sat, 12 Aug 2006 12:03:28 +0200 From: Pawel Jakub Dawidek To: Robert Watson Message-ID: <20060812100328.GB73241@garage.freebsd.pl> References: <20060808195202.GA1564@garage.freebsd.pl> <20060810232255.GE1389@garage.freebsd.pl> <20060811143257.E2260@fledge.watson.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="aM3YZ0Iwxop3KEKx" Content-Disposition: inline In-Reply-To: <20060811143257.E2260@fledge.watson.org> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, Sean Bryant , freebsd-geom@freebsd.org Subject: Re: GJournal (hopefully) final patches. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Aug 2006 10:04:30 -0000 --aM3YZ0Iwxop3KEKx Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 11, 2006 at 02:33:44PM +0100, Robert Watson wrote: >=20 > On Fri, 11 Aug 2006, Pawel Jakub Dawidek wrote: >=20 > >2. Confirm that partition with your file system is _not_ 4 sectors > > aligned. If it is not 4 sectors aligned, it is safe for gjournal to > > use its last sector for metadata. You can verify this by running: > > > > test `diskinfo /dev/ | awk '{print $4}' | xargs -J X ec= ho X % 4 | bc` -eq 0 && echo ok || echo not ok > > > >Then you need to run those commands: > > > > # gjournal label /dev/ /dev/ > > # tunefs -n disable -J enable /dev/.journal >=20 > Sounds like a "gjournal testalignment /dev/partition" would be a very use= ful thing. Or even just doing it as part of gjournal label? Sounds fair. I integrated those checks into 'label' subcommand: When configuring journaling on one provider only, check if UFS file system exists there and if it exists demand -f flag before destroying it. When configuring journaling on two providers, check if UFS exists on the data provider and if the last sector is used by the file system. If it is used, demand -f flag before overwritting it. Next, I want to think a bit how to make those checks more general, so they can be used also in other classes. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --aM3YZ0Iwxop3KEKx Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFE3adwForvXbEpPzQRArQjAKCG2gX0ac4N+mdVe13Bo/IJbAILgwCgka8D Ndh04x6cz/C/D7L8kfURqeo= =jYpF -----END PGP SIGNATURE----- --aM3YZ0Iwxop3KEKx--