From owner-freebsd-geom@FreeBSD.ORG Sun Sep 28 16:09:02 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16D7F1065686 for ; Sun, 28 Sep 2008 16:09:02 +0000 (UTC) (envelope-from volker@vwsoft.com) Received: from frontmail.ipactive.de (frontmail.maindns.de [85.214.95.103]) by mx1.freebsd.org (Postfix) with ESMTP id C57558FC1A for ; Sun, 28 Sep 2008 16:09:01 +0000 (UTC) (envelope-from volker@vwsoft.com) Received: from mail.vtec.ipme.de (Q7d63.q.ppp-pool.de [89.53.125.99]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by frontmail.ipactive.de (Postfix) with ESMTP id A240F12883F; Sun, 28 Sep 2008 17:44:27 +0200 (CEST) Received: from cesar.sz.vwsoft.com (unknown [192.168.18.3]) by mail.vtec.ipme.de (Postfix) with ESMTP id A3DB12E90F; Sun, 28 Sep 2008 17:42:51 +0200 (CEST) Message-ID: <48DFA64E.90905@vwsoft.com> Date: Sun, 28 Sep 2008 17:44:14 +0200 From: Volker User-Agent: Thunderbird 2.0.0.16 (X11/20080727) MIME-Version: 1.0 To: Giulio Ferro References: <48DE439C.4050505@zirakzigil.org> In-Reply-To: <48DE439C.4050505@zirakzigil.org> X-Enigmail-Version: 0.95.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit MailScanner-NULL-Check: 1223221380.98866@2bcD8COnkbxN0dse5bkgMQ X-MailScanner-ID: A3DB12E90F.68546 X-VWSoft-MailScanner: Found to be clean X-MailScanner-From: volker@vwsoft.com X-ipactive-MailScanner-Information: Please contact the ISP for more information X-ipactive-MailScanner: Found to be clean X-ipactive-MailScanner-From: volker@vwsoft.com Cc: pjd@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Problems with journal? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Sep 2008 16:09:02 -0000 On 12/23/-58 20:59, Giulio Ferro wrote: >
I'm experiencing very serious delay issues > in 2 production servers. > > The server are dual intel quad core > o.s. : freebsd 7 stable (yesterday) amd64, mfi driver for 2 physical > controllers: > 1) two sas disks in mirror (mfi0) > 2) one dell md1000 disk enclosure with 15 sata disks in raid 60 (total > ~7 TB) (mfi1) > > I created a single partition for the second disk, I put the geom_journal > on it, > I formatted it with ufs and mounted it on /data directory. > > from /etc/fstab > /dev/mfid1.journal /data ufs rw,async 2 0 > Giulio, I'm wondering about the device name "mfid1.journal". Please show us your partitioning scheme and output of ``gjournal list''. Volker From owner-freebsd-geom@FreeBSD.ORG Sun Sep 28 16:22:04 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32CA8106569A for ; Sun, 28 Sep 2008 16:22:04 +0000 (UTC) (envelope-from auryn@zirakzigil.org) Received: from mail.giulioferro.it (mail.giulioferro.it [85.18.102.52]) by mx1.freebsd.org (Postfix) with ESMTP id E3EF18FC19 for ; Sun, 28 Sep 2008 16:22:03 +0000 (UTC) (envelope-from auryn@zirakzigil.org) Received: from localhost (localhost [127.0.0.1]) by mail.giulioferro.it (Postfix) with ESMTP id 190A633C39; Sun, 28 Sep 2008 18:16:53 +0200 (CEST) X-Virus-Scanned: amavisd-new at giulioferro.it Received: from mail.giulioferro.it ([127.0.0.1]) by localhost (aurynwork1sv1.giulioferro.it [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hObjU49rz-kk; Sun, 28 Sep 2008 18:16:50 +0200 (CEST) Received: from [192.168.229.16] (localhost [127.0.0.1]) (Authenticated sender: gferro@giulioferro.it) by mail.giulioferro.it (Postfix) with ESMTP id 5984533C2E; Sun, 28 Sep 2008 18:16:50 +0200 (CEST) Message-ID: <48DFADA2.6030405@zirakzigil.org> Date: Sun, 28 Sep 2008 18:15:30 +0200 From: Giulio Ferro User-Agent: Thunderbird 2.0.0.16 (X11/20080724) MIME-Version: 1.0 To: Volker References: <48DE439C.4050505@zirakzigil.org> <48DFA64E.90905@vwsoft.com> In-Reply-To: <48DFA64E.90905@vwsoft.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: pjd@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Problems with journal? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Sep 2008 16:22:04 -0000 Volker wrote: > Giulio, > > I'm wondering about the device name "mfid1.journal". Please show us your > partitioning scheme and output of ``gjournal list''. > > Volker > Just to be more specific... The journal is only set on /data Under /data I have several jails running (only one running just now), and this obviously means that there are a lot of nullfs mounts: (from /etc/fstab) /data/jails/j/mroot 6.6T 259G 5.8T 4% /data/jails/j/olddb /data/jails/j/mroot 6.6T 259G 5.8T 4% /data/jails/j/cluster-mngt /data/jails/j/mroot 6.6T 259G 5.8T 4% /data/jails/j/cluster-data /data/jails/js/olddb 6.6T 259G 5.8T 4% /data/jails/j/olddb/s /data/jails/js/cluster-mngt 6.6T 259G 5.8T 4% /data/jails/j/cluster-mngt/s /data/jails/js/cluster-data 6.6T 259G 5.8T 4% /data/jails/j/cluster-data/s devfs 1.0K 1.0K 0B 100% /data/jails/j/olddb/dev From owner-freebsd-geom@FreeBSD.ORG Sun Sep 28 16:27:04 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34A0D1065686 for ; Sun, 28 Sep 2008 16:27:04 +0000 (UTC) (envelope-from auryn@zirakzigil.org) Received: from mail.giulioferro.it (mail.giulioferro.it [85.18.102.52]) by mx1.freebsd.org (Postfix) with ESMTP id E57948FC23 for ; Sun, 28 Sep 2008 16:27:03 +0000 (UTC) (envelope-from auryn@zirakzigil.org) Received: from localhost (localhost [127.0.0.1]) by mail.giulioferro.it (Postfix) with ESMTP id 1C6AA33C17; Sun, 28 Sep 2008 18:09:50 +0200 (CEST) X-Virus-Scanned: amavisd-new at giulioferro.it Received: from mail.giulioferro.it ([127.0.0.1]) by localhost (aurynwork1sv1.giulioferro.it [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l14w6mB80cnL; Sun, 28 Sep 2008 18:09:47 +0200 (CEST) Received: from [192.168.229.16] (localhost [127.0.0.1]) (Authenticated sender: gferro@giulioferro.it) by mail.giulioferro.it (Postfix) with ESMTP id 4C69C33C15; Sun, 28 Sep 2008 18:09:47 +0200 (CEST) Message-ID: <48DFABFB.7000002@zirakzigil.org> Date: Sun, 28 Sep 2008 18:08:27 +0200 From: Giulio Ferro User-Agent: Thunderbird 2.0.0.16 (X11/20080724) MIME-Version: 1.0 To: Volker References: <48DE439C.4050505@zirakzigil.org> <48DFA64E.90905@vwsoft.com> In-Reply-To: <48DFA64E.90905@vwsoft.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: pjd@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Problems with journal? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Sep 2008 16:27:04 -0000 Volker wrote: > On 12/23/-58 20:59, Giulio Ferro wrote: > >>
I'm experiencing very serious delay issues >> in 2 production servers. >> >> The server are dual intel quad core >> o.s. : freebsd 7 stable (yesterday) amd64, mfi driver for 2 physical >> controllers: >> 1) two sas disks in mirror (mfi0) >> 2) one dell md1000 disk enclosure with 15 sata disks in raid 60 (total >> ~7 TB) (mfi1) >> >> I created a single partition for the second disk, I put the geom_journal >> on it, >> I formatted it with ufs and mounted it on /data directory. >> >> from /etc/fstab >> /dev/mfid1.journal /data ufs rw,async 2 0 >> >> > > Giulio, > > I'm wondering about the device name "mfid1.journal". Please show us your > partitioning scheme and output of ``gjournal list''. > > Volker > Sure. -------------------------------------------------------------------------------------------- > gjournal list Geom name: gjournal 4016494930 ID: 4016494930 Providers: 1. Name: mfid1.journal Mediasize: 7494986366464 (6.8T) Sectorsize: 512 Mode: r1w1e1 Consumers: 1. Name: mfid1 Mediasize: 7496060108800 (6.8T) Sectorsize: 512 Mode: r1w1e1 Jend: 7496060108288 Jstart: 7494986366464 Role: Data,Journal -------------------------------------------------------------------------------------------- Hope it helps... From owner-freebsd-geom@FreeBSD.ORG Mon Sep 29 11:06:50 2008 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E97A31065686 for ; Mon, 29 Sep 2008 11:06:50 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id CFB1D8FC17 for ; Mon, 29 Sep 2008 11:06:50 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m8TB6oMh040799 for ; Mon, 29 Sep 2008 11:06:50 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m8TB6o3c040795 for freebsd-geom@FreeBSD.org; Mon, 29 Sep 2008 11:06:50 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 29 Sep 2008 11:06:50 GMT Message-Id: <200809291106.m8TB6o3c040795@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-geom@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-geom@FreeBSD.org X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Sep 2008 11:06:51 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/126902 geom [geom] [geom_label] Kernel panic during install boot o kern/124973 geom [gjournal] [patch] boot order affects geom_journal con o kern/124969 geom gvinum(8): gvinum raid5 plex does not detect missing s o kern/124294 geom [geom] gmirror(8) have inappropriate logic when workin o kern/124130 geom [gmirror][usb] gmirror fails to start usb devices that o kern/123962 geom [panic] gjournal(8): gjournal (455Gb data, 8Gb journal o kern/123630 geom [patch] [gmirror] gmirror doesnt allow the original dr o kern/123122 geom [geom] GEOM / gjournal kernel lock f kern/122415 geom [geom] UFS labels are being constantly created and rem o kern/122067 geom [geom] [panic] Geom crashed during boot o kern/121559 geom [patch] [geom] geom label class allows to create inacc o kern/121364 geom [gmirror] Removing all providers create a "zombie" mir o kern/120231 geom [geom] GEOM_CONCAT error adding second drive o kern/120044 geom [msdosfs] [geom] incorrect MSDOSFS label fries adminis o kern/120021 geom [geom] [panic] net-p2p/qbittorrent crashes system when o kern/119743 geom [geom] geom label for cds is keeped after dismount and f kern/115547 geom [geom] [patch] [request] let GEOM Eli get password fro o kern/114532 geom [geom] GEOM_MIRROR shows up in kldstat even if compile o kern/113957 geom [gmirror] gmirror is intermittently reporting a degrad o kern/113885 geom [gmirror] [patch] improved gmirror balance algorithm o kern/113837 geom [geom] unable to access 1024 sector size storage o kern/113419 geom [geom] geom fox multipathing not failing back p bin/110705 geom gmirror(8) control utility does not exit with correct o kern/107707 geom [geom] [patch] [request] add new class geom_xbox360 to o kern/104389 geom [geom] [patch] sys/geom/geom_dump.c doesn't encode XML o kern/98034 geom [geom] dereference of NULL pointer in acd_geom_detach o kern/94632 geom [geom] Kernel output resets input while GELI asks for o kern/90582 geom [geom] [panic] Restore cause panic string (ffs_blkfree o bin/90093 geom fdisk(8) incapable of altering in-core geometry a kern/89660 geom [vinum] [patch] [panic] due to g_malloc returning null o kern/89546 geom [geom] GEOM error s kern/89102 geom [geom] [panic] panic when forced unmount FS from unplu o kern/87544 geom [gbde] mmaping large files on a gbde filesystem deadlo o kern/84556 geom [geom] GBDE-encrypted swap causes panic at shutdown o kern/79251 geom [2TB] newfs fails on 2.6TB gbde device o kern/79035 geom [vinum] gvinum unable to create a striped set of mirro o bin/78131 geom gbde(8) "destroy" not working. s kern/73177 geom kldload geom_* causes panic due to memory exhaustion 38 problems total. From owner-freebsd-geom@FreeBSD.ORG Wed Oct 1 07:05:54 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA0D81065699 for ; Wed, 1 Oct 2008 07:05:54 +0000 (UTC) (envelope-from lulf@freebsd.org) Received: from bene2.itea.ntnu.no (bene2.itea.ntnu.no [IPv6:2001:700:300:3::57]) by mx1.freebsd.org (Postfix) with ESMTP id 28A158FC12 for ; Wed, 1 Oct 2008 07:05:53 +0000 (UTC) (envelope-from lulf@freebsd.org) Received: from localhost (localhost [127.0.0.1]) by bene2.itea.ntnu.no (Postfix) with ESMTP id 7AFA990002; Wed, 1 Oct 2008 09:05:51 +0200 (CEST) Received: from carrot.studby.ntnu.no (unknown [IPv6:2001:700:300:3::184]) by bene2.itea.ntnu.no (Postfix) with ESMTP id 698B590001; Wed, 1 Oct 2008 09:05:50 +0200 (CEST) Date: Wed, 1 Oct 2008 09:09:12 +0200 From: Ulf Lilleengen To: "Rick C. Petty" Message-ID: <20081001070911.GA1626@carrot.studby.ntnu.no> References: <20080310052711.GA49676@keira.kiwi-computer.com> <20080313153551.82wlu8iio4088c44@webmail.ntnu.no> <20080313182257.GB14969@keira.kiwi-computer.com> <20080316090554.GA1230@carrot.studby.ntnu.no> <20080316185055.GA64920@keira.kiwi-computer.com> <20080407075843.GA1391@carrot.studby.ntnu.no> <20080407142713.GA62578@keira.kiwi-computer.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="GID0FwUMdk1T2AWN" Content-Disposition: inline In-Reply-To: <20080407142713.GA62578@keira.kiwi-computer.com> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: Debian amavisd-new at bene2.itea.ntnu.no X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-geom@freebsd.org Subject: Re: [patch] geom_vinum platform fixes X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Oct 2008 07:05:54 -0000 --GID0FwUMdk1T2AWN Content-Type: multipart/mixed; boundary="xHFwDpU9dbj6ez1V" Content-Disposition: inline --xHFwDpU9dbj6ez1V Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On man, apr 07, 2008 at 09:27:13am -0500, Rick C. Petty wrote: > On Mon, Apr 07, 2008 at 09:58:43AM +0200, Ulf Lilleengen wrote: > > On s=F8n, mar 16, 2008 at 12:50:55pm -0600, Rick C. Petty wrote: > > > On Sun, Mar 16, 2008 at 10:05:55AM +0100, Ulf Lilleengen wrote: > > > >=20 > > > > I've reviewed the patch and done some modifications to it. I'll nee= d some > > > > testing first though (I don't have a testbed right now since I'm tr= avelling). > > >=20 > > > I've reviewed your patch but haven't had time to test it yet (hopeful= ly > > > this week..) > > >=20 > > Have you been able to test it? I don't have any 64-bit hardware at the > > moment :) >=20 > Sorry, I've been busy with other projects. I'll try to get to it this > week, although I can't see why your patch would break anything. >=20 It has been a long time, but I was able to test the patch on i386, amd64 and sparc64, and unfortunately it did not work on sparc64. However, I have now modified it to support big/little endian 32/64-bit and it seems to work on both sparc64 and i386, and the testing is successful on amd64 as well (after I get home today), I will commit the attached patch. Sorry for the long delay :) --=20 Ulf Lilleengen --xHFwDpU9dbj6ez1V-- --GID0FwUMdk1T2AWN Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkjjIhcACgkQCILg8nMIdCVeXQCeNUNMKD5Itbzw5FYLXTQST3gC 0gwAnRLajbkLY6VJAJYHefNmFyBAGopy =4HYZ -----END PGP SIGNATURE----- --GID0FwUMdk1T2AWN-- From owner-freebsd-geom@FreeBSD.ORG Wed Oct 1 12:19:17 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40D801065688 for ; Wed, 1 Oct 2008 12:19:17 +0000 (UTC) (envelope-from crahman@gmail.com) Received: from hs-out-0708.google.com (hs-out-0708.google.com [64.233.178.248]) by mx1.freebsd.org (Postfix) with ESMTP id F3D058FC17 for ; Wed, 1 Oct 2008 12:19:16 +0000 (UTC) (envelope-from crahman@gmail.com) Received: by hs-out-0708.google.com with SMTP id 54so155493hsz.11 for ; Wed, 01 Oct 2008 05:19:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type:content-transfer-encoding :content-disposition; bh=rHtLhiHabfBoMyFrfFH4yJ5grHAc3mQElvh8mYh0vWU=; b=YIk0QrUeotUcUgEVYNNKI94InWMfHMH6QXh5iE6DV6mFUmfVjnLrMmHXycFS3EYL0W mtoyuYZFZempXdGpyN1jKsGnrIxDnGgvKbzOjoJkdPAnmhm/mAV7Ex7HSBFrqNuS9BZb nbQnHWWBClMhE5zV2tcp2oZPvJ7n5G0BRpAtI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type :content-transfer-encoding:content-disposition; b=AfS/wUrYI2RQ5dBIwZA4SaCpVsbHDJYSU7kyGENySWtFtL9SLeAXV6ZnVACCd3zcIp U6h42h7Kg4RZvjTCjA+9uaCzLrUJmDQWMOPQ2GVcAh7ai71zefS182W9LejZ1EPGuWBD 0PkoUYmxokyKZ58dnT7smfPEN9+mJvRdHZJhI= Received: by 10.65.75.2 with SMTP id c2mr13923972qbl.58.1222861972039; Wed, 01 Oct 2008 04:52:52 -0700 (PDT) Received: by 10.65.176.2 with HTTP; Wed, 1 Oct 2008 04:52:52 -0700 (PDT) Message-ID: <9e77bdb50810010452r3bd4a01bs14facb8fa9a97b4a@mail.gmail.com> Date: Wed, 1 Oct 2008 05:52:52 -0600 From: "Cyrus Rahman" To: freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: gjournal deadlock X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Oct 2008 12:19:17 -0000 I continue to experience deadlocks using gjournal with large files. In a previous message I mentioned that they occur frequently with snapshots. Although useful, it is certainly possible to do without snapshots, however, lately I have experienced them in another context, namely, building nanobsd images. The problem occurs when writing out the image file through the md(4) device. Writing 128MB images causes no trouble, but moving to a 2GB image causes the deadlock every time. In fact, I was only able to succeed by building the image on a non-journaled filesystem. The deadlock occurs while sleeping on wdrain - here's the ps(1) output of the processes involved in one such event: 0 51 0 0 -16 0 0 16 wdrain DL ?? 1:24.22 [g_journal switcher] 0 52022 52018 0 -16 0 4640 1152 wdrain D ?? 0:00.02 newsyslog 1001 52069 1725 0 -16 0 2596 636 wdrain D p3 0:00.01 sync 0 51935 51933 0 -16 0 4640 1124 wdrain T p7 0:00.38 cpio -dump /usr/obj/nanobsd.img 0 51924 0 0 75 0 0 16 suspfs DL ?? 0:00.12 [md0] These values are used when deciding to msleep in wdrain: vfs.hirunningspace: 1048576 vfs.lorunningspace: 524288 vfs.runningbufspace: 1956352 They remain static after the deadlock. The really unacceptable aspect of this is that if you don't notice the deadlock has occurred, you can continue to work for many hours on other projects. However, none of the changes made to the filesystem after the deadlock will be committed to the disk. So all your work, including any notes about the deadlock, will vanish when you reboot. It's strange seeing all those deleted files reappear and your code revert back ten hours to the instant the deadlock occurred, and this issue represents a serious danger to anyone using gjournal in a production environment. Furthermore, the problem affects all gjournaled filesystems, not just the one involved in the observed deadlock - so, for example, your successfully received mail and such will also vanish. I expect what happens is that all the changes after the deadlock pile up in the journals, and so remain visible until the inevitable reboot, at which time they are discarded. From owner-freebsd-geom@FreeBSD.ORG Wed Oct 1 13:05:48 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B0B3106568E for ; Wed, 1 Oct 2008 13:05:48 +0000 (UTC) (envelope-from lulf@freebsd.org) Received: from bene1.itea.ntnu.no (bene1.itea.ntnu.no [IPv6:2001:700:300:3::56]) by mx1.freebsd.org (Postfix) with ESMTP id 9ECF48FC0C for ; Wed, 1 Oct 2008 13:05:46 +0000 (UTC) (envelope-from lulf@freebsd.org) Received: from localhost (localhost [127.0.0.1]) by bene1.itea.ntnu.no (Postfix) with ESMTP id 34976176ADF; Wed, 1 Oct 2008 15:05:45 +0200 (CEST) Received: from carrot.studby.ntnu.no (unknown [IPv6:2001:700:300:3::184]) by bene1.itea.ntnu.no (Postfix) with ESMTP id 4A9742401B; Wed, 1 Oct 2008 15:05:42 +0200 (CEST) Date: Wed, 1 Oct 2008 15:09:00 +0200 From: Ulf Lilleengen To: "Rick C. Petty" Message-ID: <20081001130842.GA1419@carrot.studby.ntnu.no> References: <20080310052711.GA49676@keira.kiwi-computer.com> <20080313153551.82wlu8iio4088c44@webmail.ntnu.no> <20080313182257.GB14969@keira.kiwi-computer.com> <20080316090554.GA1230@carrot.studby.ntnu.no> <20080316185055.GA64920@keira.kiwi-computer.com> <20080407075843.GA1391@carrot.studby.ntnu.no> <20080407142713.GA62578@keira.kiwi-computer.com> <20081001070911.GA1626@carrot.studby.ntnu.no> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="a1QUDc0q7S3U7/Jg" Content-Disposition: inline In-Reply-To: <20081001070911.GA1626@carrot.studby.ntnu.no> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: Debian amavisd-new at bene1.itea.ntnu.no X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-geom@freebsd.org Subject: Re: [patch] geom_vinum platform fixes X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Oct 2008 13:05:48 -0000 --a1QUDc0q7S3U7/Jg Content-Type: multipart/mixed; boundary="LKTjZJSUETSlgu2t" Content-Disposition: inline --LKTjZJSUETSlgu2t Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 01, 2008 at 09:09:12AM +0200, Ulf Lilleengen wrote: > On man, apr 07, 2008 at 09:27:13am -0500, Rick C. Petty wrote: > > On Mon, Apr 07, 2008 at 09:58:43AM +0200, Ulf Lilleengen wrote: > > > On s=F8n, mar 16, 2008 at 12:50:55pm -0600, Rick C. Petty wrote: > > > > On Sun, Mar 16, 2008 at 10:05:55AM +0100, Ulf Lilleengen wrote: > > > > >=20 > > > > > I've reviewed the patch and done some modifications to it. I'll n= eed some > > > > > testing first though (I don't have a testbed right now since I'm = travelling). > > > >=20 > > > > I've reviewed your patch but haven't had time to test it yet (hopef= ully > > > > this week..) > > > >=20 > > > Have you been able to test it? I don't have any 64-bit hardware at the > > > moment :) > >=20 > > Sorry, I've been busy with other projects. I'll try to get to it this > > week, although I can't see why your patch would break anything. > >=20 >=20 > It has been a long time, but I was able to test the patch on i386, amd64 = and > sparc64, and unfortunately it did not work on sparc64. However, I have now > modified it to support big/little endian 32/64-bit and it seems to work on > both sparc64 and i386, and the testing is successful on amd64 as well (af= ter > I get home today), I will commit the attached patch. >=20 > Sorry for the long delay :) >=20 And the patch :) --=20 Ulf Lilleengen --LKTjZJSUETSlgu2t-- --a1QUDc0q7S3U7/Jg Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkjjdmsACgkQCILg8nMIdCVNOQCfSrqGptn3S4fEK3Xc1iClVjzB jb4An006E9KcXXaq2LRNCVqmBAI4hWWN =06xE -----END PGP SIGNATURE----- --a1QUDc0q7S3U7/Jg-- From owner-freebsd-geom@FreeBSD.ORG Wed Oct 1 13:07:56 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B71B1065687 for ; Wed, 1 Oct 2008 13:07:56 +0000 (UTC) (envelope-from lulf@freebsd.org) Received: from bene2.itea.ntnu.no (bene2.itea.ntnu.no [IPv6:2001:700:300:3::57]) by mx1.freebsd.org (Postfix) with ESMTP id A3AD38FC1E for ; Wed, 1 Oct 2008 13:07:54 +0000 (UTC) (envelope-from lulf@freebsd.org) Received: from localhost (localhost [127.0.0.1]) by bene2.itea.ntnu.no (Postfix) with ESMTP id 41FDD90001; Wed, 1 Oct 2008 15:07:53 +0200 (CEST) Received: from carrot.studby.ntnu.no (unknown [IPv6:2001:700:300:3::184]) by bene2.itea.ntnu.no (Postfix) with ESMTP id B1A8D90003; Wed, 1 Oct 2008 15:07:52 +0200 (CEST) Date: Wed, 1 Oct 2008 15:11:13 +0200 From: Ulf Lilleengen To: "Rick C. Petty" Message-ID: <20081001131113.GB1419@carrot.studby.ntnu.no> References: <20080310052711.GA49676@keira.kiwi-computer.com> <20080313153551.82wlu8iio4088c44@webmail.ntnu.no> <20080313182257.GB14969@keira.kiwi-computer.com> <20080316090554.GA1230@carrot.studby.ntnu.no> <20080316185055.GA64920@keira.kiwi-computer.com> <20080407075843.GA1391@carrot.studby.ntnu.no> <20080407142713.GA62578@keira.kiwi-computer.com> <20081001070911.GA1626@carrot.studby.ntnu.no> <20081001130842.GA1419@carrot.studby.ntnu.no> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20081001130842.GA1419@carrot.studby.ntnu.no> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: Debian amavisd-new at bene2.itea.ntnu.no Cc: freebsd-geom@freebsd.org Subject: Re: [patch] geom_vinum platform fixes X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "..."@carrot.studby.ntnu.no List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Oct 2008 13:07:56 -0000 On Wed, Oct 01, 2008 at 03:09:00PM +0200, Ulf Lilleengen wrote: > On Wed, Oct 01, 2008 at 09:09:12AM +0200, Ulf Lilleengen wrote: > > On man, apr 07, 2008 at 09:27:13am -0500, Rick C. Petty wrote: > > > On Mon, Apr 07, 2008 at 09:58:43AM +0200, Ulf Lilleengen wrote: > > > > On søn, mar 16, 2008 at 12:50:55pm -0600, Rick C. Petty wrote: > > > > > On Sun, Mar 16, 2008 at 10:05:55AM +0100, Ulf Lilleengen wrote: > > > > > > > > > > > > I've reviewed the patch and done some modifications to it. I'll need some > > > > > > testing first though (I don't have a testbed right now since I'm travelling). > > > > > > > > > > I've reviewed your patch but haven't had time to test it yet (hopefully > > > > > this week..) > > > > > > > > > Have you been able to test it? I don't have any 64-bit hardware at the > > > > moment :) > > > > > > Sorry, I've been busy with other projects. I'll try to get to it this > > > week, although I can't see why your patch would break anything. > > > > > > > It has been a long time, but I was able to test the patch on i386, amd64 and > > sparc64, and unfortunately it did not work on sparc64. However, I have now > > modified it to support big/little endian 32/64-bit and it seems to work on > > both sparc64 and i386, and the testing is successful on amd64 as well (after > > I get home today), I will commit the attached patch. > > > > Sorry for the long delay :) > > > > And the patch :) > Hrmf, does not seem to get through. Here then: http://people.freebsd.org/~lulf/patches/gvinum/gvinum_platformfix.diff -- Ulf Lilleengen From owner-freebsd-geom@FreeBSD.ORG Wed Oct 1 20:31:52 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D73DC1065693 for ; Wed, 1 Oct 2008 20:31:52 +0000 (UTC) (envelope-from crahman@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.31]) by mx1.freebsd.org (Postfix) with ESMTP id 94C9F8FC16 for ; Wed, 1 Oct 2008 20:31:52 +0000 (UTC) (envelope-from crahman@gmail.com) Received: by yw-out-2324.google.com with SMTP id 9so137343ywe.13 for ; Wed, 01 Oct 2008 13:31:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type:content-transfer-encoding :content-disposition; bh=UTUEieBXWOm4REWnVmXS/XQ7FRC72I9eX6K0gGb3P4w=; b=eG6ykDQcwPFhUuRJQWKkPmxj2tEbro2LUqZtgRyUyUO2Ss35DTy5ja1waZQ3VAmDJl RH+88PzNVg4ftLku1ug4hThzJ9E1A+l33Y6A3K3kTWfSzXZPBXMkvFHOCL6UHLIoXUGF C8AVQuIxKEoH2Fq9ZicFEDlvR7wkTJHDwO/Sg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type :content-transfer-encoding:content-disposition; b=rMuepeKQW0kRIsSS2gLIXwW7qldCzrmXJ32dcpg6sFRjU6AqO43AXvkHk7jpfiBLvL mAzC4Pw/B/c1ZZ0oQt67T/TFMFtfiR1aMh1NqNDyI9YR0Mqzzd41m9bV4npFaCFSuwyi n2BHTDCxrFvrMRt7b54CzdnWjn0iioIT+rbEs= Received: by 10.65.212.18 with SMTP id o18mr14897583qbq.53.1222893110894; Wed, 01 Oct 2008 13:31:50 -0700 (PDT) Received: by 10.65.176.2 with HTTP; Wed, 1 Oct 2008 13:31:50 -0700 (PDT) Message-ID: <9e77bdb50810011331y7216eac3yf85907f96f5e8370@mail.gmail.com> Date: Wed, 1 Oct 2008 14:31:50 -0600 From: "Cyrus Rahman" To: freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: Experiences with Gpart X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Oct 2008 20:31:52 -0000 I recently had occasion to set up a system that needed to contain numerous partitions, and I thought it would be a good occasion to try gpart to divide the disk. By and large it went very well, with the exception of bug in 'gpart bootcode' which made it necessary to manually dd in the bootstrap. I did discover a few oddities that I will mention here. My intent was to mirror the disks with gmirror. It didn't make sense to individually mirror each gpart partition and have some 30 simultaneous restores going should the mirror ever break. My first idea was to mirror the entire disks (ad4 and ad6) with gmirror and put gpart over the mirror. This sort of works, except that gpart itself if it finds itself at the beginning of the disk even when it doesn't properly taste itself on the last block of the disk. As a result, the same gpart partitions get attached to ad4 and ad6, as well as to the mirror. The gpart geoms attached directly to the disks complain about the last sector being incorrect (the backup table) and request that you fix it (which would not likely go well). But in addition to that, any further geoms contained within the mirror get attached to the outer gpart geoms on ad4 and ad6 and not the mirror. They don't work properly, because the mirror is already attached to ad4 and ad6, so you end up with a non-functional setup. In experimenting, I also discovered that if you directly nest gpart tables, things work fine until you reboot. Then the internal partitions vanish because gpart doesn't attach to them. Fortunately, if you put the inner gpart inside a gmirror inside the outer gpart, things do work fine. This I did for two reasons: First, to avoid the aforementioned trouble with an outermost gmirror containing a gpart geom, and second, so that I could better define how the disk was to be mirrored. I was able to put a number of smaller filesystems inside a single mirrored gpart partition, and then most of the really big ones inside a second mirrored gpart partition. This second partition used the -F option to gmirror so that it would not need to be re-synchonized in a crash. The filesystems within it are gjournal'd to make this possible. Booting was yet another issue, since the loader objects to nested gpart partitions. I handled this by simply making a third partition on the outermost gpt that contained the root. Since this is likely confusing, here's a diagram that will help if you use a fixed font: ad4 ad6 ------ ------ gpart 1 (ad4p1) gpart 1 (ad6p1) freebsd-boot freebsd-boot gpart 2 (ad4p2) gpart 2 (ad6p2) gmirror (gm0 = ad4p2 + ad6p2) root (gm0.p1) gpart 3 (ad4p3) gpart3 (ad6p3) gmirror (gm1 = ad4p3 + ad6p3) gpart 1 (gm1p1) gpart 2 (gm1p2 ... gpart n (gm1pn) gpart 3 (ad4p4) gpart 3 (ad6p4) gmirror (gm2 = ad4p4 + ad6p4) (mirrored with -F) gpart 1 (gm2p1) gjournal (gm2p1.journal) gpart 2 (gm2p2) gjournal (gm2p2.journal) ... Despite the intent of gpt's being to make such nesting unnecessary, as a means of defining the structure of gmirrors, which take up the entire extent of whatever encloses them, the nesting was very helpful. From owner-freebsd-geom@FreeBSD.ORG Thu Oct 2 22:16:08 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 201191065698 for ; Thu, 2 Oct 2008 22:16:08 +0000 (UTC) (envelope-from stickybit@gmx.net) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 7F0468FC17 for ; Thu, 2 Oct 2008 22:16:07 +0000 (UTC) (envelope-from stickybit@gmx.net) Received: (qmail invoked by alias); 02 Oct 2008 21:49:25 -0000 Received: from port-87-234-81-167.dynamic.qsc.de (EHLO localhost) [87.234.81.167] by mail.gmx.net (mp004) with SMTP; 02 Oct 2008 23:49:25 +0200 X-Authenticated: #23197544 X-Provags-ID: V01U2FsdGVkX19QxvF8yOpQOvD+IKKohcQPzmfUGWIuDoYOXr+HPN 498iZWdK4rWejw From: Sticky Bit To: freebsd-geom@freebsd.org Date: Thu, 2 Oct 2008 23:45:22 +0200 User-Agent: Mail User Agent X-Privacy-Policy: Consider OpenPGP/MIME signing and encryption. X-Anti-Spam-Policy: No HTML or Richtext! Plain text emails preferred. X-Security-Policy: No attachments! MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200810022345.22955.stickybit@gmx.net> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 X-FuHaFi: 0.5 Subject: Wrong warnings about rawoffset not 0 (geom_bsd or geom_journal related?) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: stickybit@gmx.net List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Oct 2008 22:16:08 -0000 Hello, after I recently setup and successful use gjournal I got following messages on every boot: WARNING: Expected rawoffset 0, found 63 WARNING: Expected rawoffset 0, found 120824865 I think the expectation is wrong here. Looks like geom_bsd (or another geom part) is not aware of the fact that there is more than one slice on disk? Slice 1 offset is 63. This of course should never be 0 because space before has to be reserved for boot managers etc. according to the specifications. So fdisk is right here to set offset at 63 for slice 1. Slice 2 offset is 120824865. This is also right because it is a new slice. So those warnings are not plausible and certainly wrong. Did I miss something? Can someone explain it? Slices 1 and 2 were created by sysinstall long time ago. BTW why starts the first partition within these slices at offset 0 and not at offset 16 which it should according to bsdlabel manual page? It works without problems but seems wrong. Slices 3 and 4 were recently created by hand using fdisk and then partitioned using bsdlabel. They hold the journal providers. BTW first partition within these slices starts at offset 16 which seems right. Looks like sysinstall created first partitions within a slice have a wrong offset beginning at 0 and not at 16. But that is probably not the cause of the warning messages at boot. This is my FreeBSD RELENG_7 amd64 setup (fdisk, bsdlabel, gjournal status): # fdisk /dev/ad6 ******* Working on device /dev/ad6 ******* parameters extracted from in-core disklabel are: cylinders=620181 heads=16 sectors/track=63 (1008 blks/cyl) Figures below won't work with BIOS for partitions not in cyl 1 parameters to be used for BIOS calculations are: cylinders=620181 heads=16 sectors/track=63 (1008 blks/cyl) Media sector size is 512 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 63, size 120824802 (58996 Meg), flag 80 (active) beg: cyl 0/ head 1/ sector 1; end: cyl 1023/ head 254/ sector 63 The data for partition 2 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 120824865, size 335533590 (163834 Meg), flag 80 (active) beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 254/ sector 63 The data for partition 3 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 456358455, size 83875365 (40954 Meg), flag 80 (active) beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 254/ sector 63 The data for partition 4 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 540233820, size 84903525 (41456 Meg), flag 80 (active) beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 254/ sector 63 # bsdlabel /dev/ad6s1: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 2097152 0 4.2BSD 2048 16384 28528 b: 22257664 2097152 swap c: 120824802 0 unused 0 0 # "raw" part, don't edit d: 2097152 24354816 4.2BSD 2048 16384 28528 e: 31457280 26451968 4.2BSD 2048 16384 28528 f: 20971520 57909248 4.2BSD 2048 16384 28528 g: 20971520 78880768 4.2BSD 2048 16384 28528 h: 20972514 99852288 4.2BSD 2048 16384 28528 # bsdlabel /dev/ad6s2: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 335533590 0 unused 0 0 # "raw" part, don't edit d: 41943040 0 4.2BSD 2048 16384 28528 e: 20971520 41943040 4.2BSD 2048 16384 28528 f: 62914560 62914560 4.2BSD 2048 16384 28528 g: 209704470 125829120 4.2BSD 2048 16384 28528 # bsdlabel /dev/ad6s3: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 83875365 0 unused 0 0 # "raw" part, don't edit d: 16777216 16 4.2BSD 0 0 0 e: 16777216 16777232 4.2BSD 0 0 0 f: 16777216 33554448 4.2BSD 0 0 0 g: 16777216 50331664 4.2BSD 0 0 0 h: 16766485 67108880 4.2BSD 0 0 0 # bsdlabel /dev/ad6s4: 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 84903525 0 unused 0 0 # "raw" part, don't edit d: 16777216 16 4.2BSD 0 0 0 e: 16777216 16777232 4.2BSD 0 0 0 f: 16777216 33554448 4.2BSD 0 0 0 g: 16777216 50331664 4.2BSD 0 0 0 h: 17794645 67108880 4.2BSD 0 0 0 # gjournal status Name Status Components ad6s1a.journal N/A ad6s1a ad6s3d ad6s1d.journal N/A ad6s1d ad6s3e ad6s1e.journal N/A ad6s1e ad6s3f ad6s1f.journal N/A ad6s1f ad6s3g ad6s1g.journal N/A ad6s1g ad6s3h ad6s1h.journal N/A ad6s1h ad6s4d ad6s2d.journal N/A ad6s2d ad6s4e ad6s2e.journal N/A ad6s2e ad6s4f ad6s2f.journal N/A ad6s2f ad6s4g ad6s2g.journal N/A ad6s2g ad6s4h Thanks! Regards