From owner-freebsd-geom@FreeBSD.ORG Mon Jan 1 06:52:07 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 239B216A40F for ; Mon, 1 Jan 2007 06:52:07 +0000 (UTC) (envelope-from awatts@pett.com.au) Received: from mail.equard.com.au (mail.equard.com.au [150.101.96.125]) by mx1.freebsd.org (Postfix) with ESMTP id 9290613C448 for ; Mon, 1 Jan 2007 06:52:06 +0000 (UTC) (envelope-from awatts@pett.com.au) Received: from pett.com.au ([172.24.169.85]) by mail.equard.com.au (8.13.8/8.13.8) with ESMTP id l016cTnc032136 for ; Mon, 1 Jan 2007 17:08:30 +1030 (CST) (envelope-from awatts@pett.com.au) Message-ID: <4598AC65.3080902@pett.com.au> Date: Mon, 01 Jan 2007 17:08:29 +1030 From: Alastair Watts User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.6) Gecko/20040113 X-Accept-Language: en-au, en-us, en MIME-Version: 1.0 To: freebsd-geom@freebsd.org References: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> In-Reply-To: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> X-Enigmail-Version: 0.83.6.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2007 06:52:07 -0000 Michael Knoll wrote: > I am planning on converting my FreeBSD machine to use gmirror on two > 40gig drives. Reading the handbook and other sites with instructions > on configuring gmirror, I notice they all disable swap. Is this > acceptable? Is it expected swap be on another drive? If so, is there > a solution which I can keep the swap on the mirror. as I don't have > another drive? I'm running the following setup: > gmirror status Name Status Components mirror/gm0 COMPLETE ad0 ad2 > swapinfo Device 1K-blocks Used Avail Capacity /dev/mirror/gm0s1b 1048576 784 1048576 0% > mount /dev/mirror/gm0s1a on / (ufs, local) devfs on /dev (devfs, local) /dev/mirror/gm0s1d on /tmp (ufs, local, soft-updates) /dev/mirror/gm0s1e on /usr (ufs, local, soft-updates) devfs on /usr/var/named/dev (devfs, local) Can someone confirm if this is not what should be done? It has appeared to work fine (having swap mirrored is a good thing from a keeping the system alive point of view), and I would be interested to hear why it shouldn't be done. Cheers, Al From owner-freebsd-geom@FreeBSD.ORG Mon Jan 1 11:08:40 2007 Return-Path: X-Original-To: freebsd-geom@FreeBSD.org Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6CE5716A52E for ; Mon, 1 Jan 2007 11:08:40 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [69.147.83.40]) by mx1.freebsd.org (Postfix) with ESMTP id 5CF6413C4C5 for ; Mon, 1 Jan 2007 11:08:40 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id l01B8eBY048856 for ; Mon, 1 Jan 2007 11:08:40 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id l01B8cqE048852 for freebsd-geom@FreeBSD.org; Mon, 1 Jan 2007 11:08:38 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 1 Jan 2007 11:08:38 GMT Message-Id: <200701011108.l01B8cqE048852@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: linimon set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-geom@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2007 11:08:40 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/73177 geom kldload geom_* causes panic due to memory exhaustion o kern/76538 geom [gbde] nfs-write on gbde partition stalls and continue o kern/83464 geom [geom] [patch] Unhandled malloc failures within libgeo o kern/84556 geom [geom] GBDE-encrypted swap causes panic at shutdown o kern/87544 geom [gbde] mmaping large files on a gbde filesystem deadlo o kern/89102 geom [geom_vfs] [panic] panic when forced unmount FS from u o bin/90093 geom fdisk(8) incapable of altering in-core geometry o kern/90582 geom [geom_mirror] [panic] Restore cause panic string (ffs_ o kern/98034 geom [geom] dereference of NULL pointer in acd_geom_detach o kern/104389 geom [geom] [patch] sys/geom/geom_dump.c doesn't encode XML 10 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o bin/78131 geom gbde "destroy" not working. o kern/79251 geom [2TB] newfs fails on 2.6TB gbde device o kern/94632 geom [geom] Kernel output resets input while GELI asks for o kern/105390 geom [geli] filesystem on a md backed by sparse file with s 4 problems total. From owner-freebsd-geom@FreeBSD.ORG Mon Jan 1 18:08:38 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B093816A407 for ; Mon, 1 Jan 2007 18:08:38 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 4104013C428 for ; Mon, 1 Jan 2007 18:08:38 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1H1RaC-0008Td-T9 for freebsd-geom@freebsd.org; Mon, 01 Jan 2007 19:08:24 +0100 Received: from 89-172-55-32.adsl.net.t-com.hr ([89.172.55.32]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 01 Jan 2007 19:08:24 +0100 Received: from ivoras by 89-172-55-32.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 01 Jan 2007 19:08:24 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Mon, 01 Jan 2007 19:07:59 +0100 Lines: 37 Message-ID: References: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig65FDBB6085CFF7DAC921DCD2" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-55-32.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> X-Enigmail-Version: 0.94.1.2 Sender: news Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2007 18:08:38 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig65FDBB6085CFF7DAC921DCD2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Michael Knoll wrote: > I am planning on converting my FreeBSD machine to use gmirror on two > 40gig drives. Reading the handbook and other sites with instructions > on configuring gmirror, I notice they all disable swap. Is this > acceptable? Is it expected swap be on another drive? If so, is there > a solution which I can keep the swap on the mirror. as I don't have > another drive? There's no technical problem with it, it's more a matter of organization and convenience. For example: you don't really need the swap to survive a crash, so there's no need to introduce the overhead of mirroring it - better to leave 2x the space configured as swap area. If you have more than one area, the system should take advantage of it and use both to maximize speed (in theory this should work as an implicit RAID0 across the swap areas). --------------enig65FDBB6085CFF7DAC921DCD2 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFmU4GldnAQVacBcgRAixsAKCQHtk9a80B6i7q9sjZMyH+EwhZqgCgjWkv mHrVTJ5zg1CVjtQ8N+QISa8= =BVHJ -----END PGP SIGNATURE----- --------------enig65FDBB6085CFF7DAC921DCD2-- From owner-freebsd-geom@FreeBSD.ORG Mon Jan 1 20:42:23 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DF28816A415 for ; Mon, 1 Jan 2007 20:42:23 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.187]) by mx1.freebsd.org (Postfix) with ESMTP id 7A79A13C43E for ; Mon, 1 Jan 2007 20:42:23 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: by nf-out-0910.google.com with SMTP id x37so6566161nfc for ; Mon, 01 Jan 2007 12:42:22 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:user-agent:mime-version:to:subject:x-enigmail-version:content-type:content-transfer-encoding; b=ov2V965BEp1zSsrZNrLiBUoLizhlwEe9UHg7OCGPu7ioljbq1ZFTa1Nf1+j54JJ+OYc3qH18ms4wOehiSdk/ScizraGC907ec4YykDcQ2kK6mm2qQtVMLCkIuNNwsf7QjrE6BJdS5tbaUruYBg6KK/pHnvJkDNgA5ZvrflVKUe4= Received: by 10.49.68.6 with SMTP id v6mr11214264nfk.1167682648050; Mon, 01 Jan 2007 12:17:28 -0800 (PST) Received: from ?192.168.123.201? ( [195.241.221.201]) by mx.google.com with ESMTP id h1sm82241947nfe.2007.01.01.12.17.26; Mon, 01 Jan 2007 12:17:27 -0800 (PST) Message-ID: <45996C4F.8070700@gmail.com> Date: Mon, 01 Jan 2007 21:17:19 +0100 From: Rene Ladan User-Agent: Thunderbird 1.5.0.9 (X11/20061224) MIME-Version: 1.0 To: freebsd-geom@freebsd.org X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: xbox360 extension for review/debug X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2007 20:42:24 -0000 Hi, I've written an extension to /sys/geom/geom_mbr.c to slice up xbox360 hard disks and memory units. The patch for revision 1.68 (i.e. CURRENT) is at http://home.tisali.nl/rladan/freebsd/geom_mbr.c.diff Memory units are sliced up into two file systems, hard disks into three (plus two non-filesystem slices). This would normally be handled by the existing geom_mbr code, but Mircosoft didn't put a mbr / partition table on the media :/ The code generates a new class XBOX360 which intends to detect and slice up the media. It is currently unusable, without an attached medium the kernel panics during boot at xbox360_taste+0x82 in process g_up. My geom foo could use some help here :) At least the 'geom' command in ddb shows this new class (without consumers/producers). (Microsoft also invented a new filesystem for the media, I'm currently debugging a clone of msdosfs to parse this fileystem, see the -fs archives of Nov/Dec 2006.) Regards, Rene -- GPG fingerprint = E738 5471 D185 7013 0EE0 4FC8 3C1D 6F83 12E1 84F6 (subkeys.pgp.net) "It won't fit on the line." -- me, 2001 From owner-freebsd-geom@FreeBSD.ORG Mon Jan 1 21:53:29 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B5C0B16A407 for ; Mon, 1 Jan 2007 21:53:29 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 7AE7813C442 for ; Mon, 1 Jan 2007 21:53:29 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 807F41747B; Mon, 1 Jan 2007 21:24:27 +0000 (UTC) To: Rene Ladan From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 01 Jan 2007 21:17:19 +0100." <45996C4F.8070700@gmail.com> Date: Mon, 01 Jan 2007 21:24:22 +0000 Message-ID: <74225.1167686662@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: freebsd-geom@freebsd.org Subject: Re: xbox360 extension for review/debug X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2007 21:53:29 -0000 In message <45996C4F.8070700@gmail.com>, Rene Ladan writes: >Hi, > >I've written an extension to /sys/geom/geom_mbr.c to slice up xbox360 >hard disks and memory units. The patch for revision 1.68 (i.e. CURRENT) >is at http://home.tisali.nl/rladan/freebsd/geom_mbr.c.diff This is wrong, you should make a geom_xbox360 class instead. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-geom@FreeBSD.ORG Mon Jan 1 22:08:26 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EC07416A58E for ; Mon, 1 Jan 2007 22:08:26 +0000 (UTC) (envelope-from tom.hurst@clara.net) Received: from spork.qfe3.net (spork.qfe3.net [212.13.207.101]) by mx1.freebsd.org (Postfix) with ESMTP id B08BA13C428 for ; Mon, 1 Jan 2007 22:08:26 +0000 (UTC) (envelope-from tom.hurst@clara.net) Received: from [81.104.144.87] (helo=voi.aagh.net) by spork.qfe3.net with esmtp (Exim 4.62 (FreeBSD)) (envelope-from ) id 1H1V2v-000B9G-7h; Mon, 01 Jan 2007 21:50:17 +0000 Received: from freaky by voi.aagh.net with local (Exim 4.63 (FreeBSD)) (envelope-from ) id 1H1V2u-000J3i-Rr; Mon, 01 Jan 2007 21:50:16 +0000 Date: Mon, 1 Jan 2007 21:50:16 +0000 From: Thomas Hurst To: freebsd-geom@freebsd.org Message-ID: <20070101215016.GA71635@voi.aagh.net> Mail-Followup-To: freebsd-geom@freebsd.org, michael.knoll@gmail.com References: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> Organization: Not much. User-Agent: Mutt/1.5.13 (2006-08-11) Sender: Thomas Hurst Cc: michael.knoll@gmail.com Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jan 2007 22:08:27 -0000 * Michael Knoll (michael.knoll@gmail.com) wrote: > I am planning on converting my FreeBSD machine to use gmirror on two > 40gig drives. Reading the handbook and other sites with instructions > on configuring gmirror, I notice they all disable swap. Is this > acceptable? Is it expected swap be on another drive? If so, is there > a solution which I can keep the swap on the mirror. as I don't have > another drive? On every gmirrorred system I've set up I've always included swap in the mirror; you don't have to, but it's a good idea if you want the system to stay up after a disk failure, which is surely part of if not the entire point. You can't put kernel crash dumps on gmirrored devices, but you can put them on one of the underlying providers fairly easily if that's important. Are you sure you're not confusing the swapoff="YES" line many guides suggest adding to rc.conf with disabling swap completely? This just makes the system issue swapoff -a at shutdown to make sure any mirrored swap devices are closed, which used to result in the mirror being rebuilt at startup because it was "dirty". -- Thomas 'Freaky' Hurst http://hur.st/ From owner-freebsd-geom@FreeBSD.ORG Tue Jan 2 05:25:42 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D215116A407 for ; Tue, 2 Jan 2007 05:25:42 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (noop.in-addr.com [208.58.23.51]) by mx1.freebsd.org (Postfix) with ESMTP id 9935413C44C for ; Tue, 2 Jan 2007 05:25:42 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by noop.in-addr.com with local (Exim 4.54 (FreeBSD)) id 1H1bmZ-000NV0-IG; Tue, 02 Jan 2007 00:01:51 -0500 Date: Tue, 2 Jan 2007 00:01:51 -0500 From: Gary Palmer To: Ivan Voras Message-ID: <20070102050151.GA90174@in-addr.com> Mail-Followup-To: Ivan Voras , freebsd-geom@freebsd.org References: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: freebsd-geom@freebsd.org Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2007 05:25:42 -0000 On Mon, Jan 01, 2007 at 07:07:59PM +0100, Ivan Voras wrote: > Michael Knoll wrote: > > I am planning on converting my FreeBSD machine to use gmirror on two > > 40gig drives. Reading the handbook and other sites with instructions > > on configuring gmirror, I notice they all disable swap. Is this > > acceptable? Is it expected swap be on another drive? If so, is there > > a solution which I can keep the swap on the mirror. as I don't have > > another drive? > > There's no technical problem with it, it's more a matter of organization > and convenience. For example: you don't really need the swap to survive > a crash, so there's no need to introduce the overhead of mirroring it - > better to leave 2x the space configured as swap area. If you have more > than one area, the system should take advantage of it and use both to > maximize speed (in theory this should work as an implicit RAID0 across > the swap areas). Except in the case where a drive holding some swapped out memory goes bad and the system panics or crashes as a result. It might not make as much sense for desktops, but if I were (still) building servers I'd mirror everything that the system depended on to run. From owner-freebsd-geom@FreeBSD.ORG Tue Jan 2 09:26:20 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5F2E916A40F for ; Tue, 2 Jan 2007 09:26:20 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 1F10D13C455 for ; Tue, 2 Jan 2007 09:26:19 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1H1fuR-0007hN-VI for freebsd-geom@freebsd.org; Tue, 02 Jan 2007 10:26:15 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 02 Jan 2007 10:26:15 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 02 Jan 2007 10:26:15 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Tue, 02 Jan 2007 10:26:08 +0100 Lines: 9 Message-ID: References: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> <20070102050151.GA90174@in-addr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 1.5.0.4 (X11/20060625) In-Reply-To: <20070102050151.GA90174@in-addr.com> Sender: news Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2007 09:26:20 -0000 Gary Palmer wrote: > Except in the case where a drive holding some swapped out memory goes bad > and the system panics or crashes as a result. It might not make as > much sense for desktops, but if I were (still) building servers I'd > mirror everything that the system depended on to run. Well, yes... though the "only" things lost in this case are the processes using the swap :) But you're right in the general case. From owner-freebsd-geom@FreeBSD.ORG Tue Jan 2 10:01:38 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2885016A407 for ; Tue, 2 Jan 2007 10:01:38 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30310.mail.mud.yahoo.com (web30310.mail.mud.yahoo.com [209.191.69.72]) by mx1.freebsd.org (Postfix) with SMTP id E616F13C457 for ; Tue, 2 Jan 2007 10:01:37 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 87142 invoked by uid 60001); 2 Jan 2007 10:01:37 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=viGP73ZQ27MJeAdnAMl63pcysxHmolB5+0nMhqAio6AEXwFsWMSt2QWDjqffz3JG8h/FfFWhqcaU5qVwSh7Fd8tlqP+xYrkukDR/ZE/6KHHuS5FxKV0IUqJETABr4ZrcvU7yDZ8VrF3rFT0wgBflh+40zkTw8trtYlF4PfnF0FU= ; Message-ID: <20070102100137.87140.qmail@web30310.mail.mud.yahoo.com> X-YMail-OSG: _3zPCXYVM1mJr8FskiQa1VeJUPbYsVbJJrJj.pygFxNtntUSZ_Ao3F2EfnRzbVke6eKMlsS4jrd9YsNKE_qM_6ivB11Z4ZtdSYj3tFf0NyPQD1V_atrWOtVQEkMICOwspLKLauJhXPJhh3w- Received: from [213.54.170.50] by web30310.mail.mud.yahoo.com via HTTP; Tue, 02 Jan 2007 02:01:37 PST Date: Tue, 2 Jan 2007 02:01:37 -0800 (PST) From: "R. B. Riddick" To: Ivan Voras , freebsd-geom@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2007 10:01:38 -0000 --- Ivan Voras wrote: > Gary Palmer wrote: > > > Except in the case where a drive holding some swapped out memory goes bad > > and the system panics or crashes as a result. It might not make as > > much sense for desktops, but if I were (still) building servers I'd > > mirror everything that the system depended on to run. > > Well, yes... though the "only" things lost in this case are the > processes using the swap :) But you're right in the general case. > Hmm... I wonder what gmirror is good for, when one of its consumers fails... I just setup this test setting: 1. gnop on ad0s1gd 2. gmirror on ad0s1gd.nop (hardcoded (-h)) and ad0s1ge 3. dd (writes from /dev/urandom to the mirror) 4. gnop configure -v -f 100 ad0s1gd.nop 5. dd becomes unresponsive; CTRL+t says: load: 0.78 cmd: dd 11034 [physwr] 0.01u 0.68s 0% 612k I think an infinite page/swap-out transaction is quite bad for a FreeBSD, so that we will need a reboot anyway... I think, graid5 (http://home.tiscali.de/cmdr_faako/geom_raid5.tbz) would handle that better... But I did not test that for some weeks... Now I will reboot, in order to get rid of the "dd"... :-) -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Tue Jan 2 19:33:24 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ECF3D16A47B for ; Tue, 2 Jan 2007 19:33:24 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id 7896513C442 for ; Tue, 2 Jan 2007 19:33:24 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 84060 invoked by uid 2001); 2 Jan 2007 19:38:59 -0000 Date: Tue, 2 Jan 2007 13:38:59 -0600 From: "Rick C. Petty" To: Michael Knoll Message-ID: <20070102193859.GA83431@keira.kiwi-computer.com> References: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-geom@freebsd.org Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2007 19:33:25 -0000 On Sun, Dec 31, 2006 at 12:13:28PM -0500, Michael Knoll wrote: > I am planning on converting my FreeBSD machine to use gmirror on two > 40gig drives. Reading the handbook and other sites with instructions > on configuring gmirror, I notice they all disable swap. Is this > acceptable? Is it expected swap be on another drive? If so, is there > a solution which I can keep the swap on the mirror. as I don't have > another drive? I've not had much luck using any of those instructions. Instead, I opted to mirror the entire drive, the equivalent of: gmirror label -vb round-robin gm0 ad0 gmirror insert -v gm0 ad1 followed by an "fdisk", a "bsdlabel", and a number of "newfs"es. This was relatively easy to do without using the boot CD, even if you don't have a third drive to install from: (assuming you've booted from ad0) gmirror label -vb round-robin gm0 ad1 fdisk -B mirror/gm0 bsdlabel -Bw mirror/gm0s1 bsdlabel -e mirror/gm0s1 * modify label accordingly, then: * newfs /dev/mirror/gm0s1a newfs -U /dev/mirror/gm0s1d ... followed by mounting the filesystems from the mirror (and creating the necessary mountpoints, of course), then I'd do an installworld as per the instructions in /usr/src/UPDATING onto the separate (mirrored) partition. I wouldn't disable all swap (for various reasons). Having the swap partition mirrored with the rest of the disk isn't a bad thing-- it ensures the integrity of crashdumps. -- Rick C. Petty From owner-freebsd-geom@FreeBSD.ORG Tue Jan 2 19:40:54 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 29CEA16A415 for ; Tue, 2 Jan 2007 19:40:54 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id A824F13C441 for ; Tue, 2 Jan 2007 19:40:53 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 84211 invoked by uid 2001); 2 Jan 2007 19:46:31 -0000 Date: Tue, 2 Jan 2007 13:46:31 -0600 From: "Rick C. Petty" To: Ivan Voras Message-ID: <20070102194631.GB83431@keira.kiwi-computer.com> References: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.1i Cc: freebsd-geom@freebsd.org Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2007 19:40:54 -0000 On Mon, Jan 01, 2007 at 07:07:59PM +0100, Ivan Voras wrote: > > There's no technical problem with it, it's more a matter of organization > and convenience. For example: you don't really need the swap to survive > a crash, so there's no need to introduce the overhead of mirroring it - > better to leave 2x the space configured as swap area. If you have more > than one area, the system should take advantage of it and use both to > maximize speed (in theory this should work as an implicit RAID0 across > the swap areas). Mirror the whole disk, but instead of having /etc/fstab use /dev/mirror/gm0s1b as a swap device, have it use both /dev/mirror/ad0s1b and /dev/mirror/ad1s1b.. Since gm0s1b isn't used, there shouldn't be a problem referencing the underlying device nodes =) I'm sure the current implementation does not allow for this, but I can't see any reason why this shouldn't be *allowed*. However, it may not be *possible* because AFAIK there's no way in GEOM for bsdlabel to tell its provider (gmirror) that certain portions aren't in use, so the whole devices (ad0 and ad1) are being fully consumed. An interesting idea though... -- Rick C. Petty From owner-freebsd-geom@FreeBSD.ORG Tue Jan 2 19:46:24 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1EC5016A403 for ; Tue, 2 Jan 2007 19:46:24 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id A05F613C465 for ; Tue, 2 Jan 2007 19:46:23 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 84347 invoked by uid 2001); 2 Jan 2007 19:52:01 -0000 Date: Tue, 2 Jan 2007 13:52:01 -0600 From: "Rick C. Petty" To: freebsd-geom@freebsd.org, michael.knoll@gmail.com Message-ID: <20070102195201.GC83431@keira.kiwi-computer.com> References: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> <20070101215016.GA71635@voi.aagh.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070101215016.GA71635@voi.aagh.net> User-Agent: Mutt/1.4.2.1i Cc: Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Jan 2007 19:46:24 -0000 On Mon, Jan 01, 2007 at 09:50:16PM +0000, Thomas Hurst wrote: > > > Are you sure you're not confusing the swapoff="YES" line many guides > suggest adding to rc.conf with disabling swap completely? This just > makes the system issue swapoff -a at shutdown to make sure any mirrored > swap devices are closed, which used to result in the mirror being > rebuilt at startup because it was "dirty". Where is that rc.conf option defined or used? Certainly not in 6.x: % grep -R swapoff /etc/ /etc/security/audit_event:355:AUE_DARWIN_SWAPOFF:swapoff():ad /etc/security/audit_event:43045:AUE_SWAPOFF:swapoff():ad -- Rick C. Petty From owner-freebsd-geom@FreeBSD.ORG Wed Jan 3 06:00:53 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D1A1916A40F for ; Wed, 3 Jan 2007 06:00:53 +0000 (UTC) (envelope-from grafan@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.250]) by mx1.freebsd.org (Postfix) with ESMTP id 9938213C428 for ; Wed, 3 Jan 2007 06:00:53 +0000 (UTC) (envelope-from grafan@gmail.com) Received: by an-out-0708.google.com with SMTP id c24so1641371ana for ; Tue, 02 Jan 2007 22:00:52 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=TUULr6z3qrh3IiraSPtqBF22tqHKQ+Um+tUlh16lihBCbUKY7ioSQ7wxwks9d8FgV9KOQM7HB0Mr3aGY2fwy5ZJ3GDHkezLsI2+0YN0iEx6RefrkKljzbL7RfSx+VJI7l4+2fq+SwbPyyVoejJMOqS7kxLCdCKVdJLL7dP03peo= Received: by 10.100.37.4 with SMTP id k4mr6422667ank.1167804052760; Tue, 02 Jan 2007 22:00:52 -0800 (PST) Received: by 10.100.136.16 with HTTP; Tue, 2 Jan 2007 22:00:52 -0800 (PST) Message-ID: <6eb82e0701022200i7a2cb356g358a1938d620e97a@mail.gmail.com> Date: Wed, 3 Jan 2007 14:00:52 +0800 From: "Rong-en Fan" To: freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: gconcat and gpt X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jan 2007 06:00:53 -0000 Hi, I'm running 6.2-RC1 on an i386 box. I'm testing gjournal patch. I found that something is annoying... I use gconcat label big da1 da2 da3 and then running gpt on the concat/big device. So, I got concat/bigp1, concat/bigp2, ... etc *AND* I also have da1p1, da1p2... Well, this is not a problem for normal use. However, when I enable gjournal, at boot time, gconcat first activates the big one then, gjournal finds there is journal data on da1p1, so it uses! Then, da1 is deactivated from gconcat... My question is, why I got da1p1 when da1 is used by gconcat? Regards, Rong-En Fan From owner-freebsd-geom@FreeBSD.ORG Wed Jan 3 08:35:39 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EE1DA16A403 for ; Wed, 3 Jan 2007 08:35:39 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 7616813C457 for ; Wed, 3 Jan 2007 08:35:39 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id DD69E45B26; Wed, 3 Jan 2007 09:35:36 +0100 (CET) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 7F1B045685; Wed, 3 Jan 2007 09:35:28 +0100 (CET) Date: Wed, 3 Jan 2007 09:35:05 +0100 From: Pawel Jakub Dawidek To: "R. B. Riddick" Message-ID: <20070103083505.GB6253@garage.freebsd.pl> References: <20070102100137.87140.qmail@web30310.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="5QAgd0e35j3NYeGe" Content-Disposition: inline In-Reply-To: <20070102100137.87140.qmail@web30310.mail.mud.yahoo.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: Ivan Voras , freebsd-geom@freebsd.org Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jan 2007 08:35:40 -0000 --5QAgd0e35j3NYeGe Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 02, 2007 at 02:01:37AM -0800, R. B. Riddick wrote: > --- Ivan Voras wrote: > > Gary Palmer wrote: > >=20 > > > Except in the case where a drive holding some swapped out memory goes= bad > > > and the system panics or crashes as a result. It might not make as > > > much sense for desktops, but if I were (still) building servers I'd > > > mirror everything that the system depended on to run. > >=20 > > Well, yes... though the "only" things lost in this case are the > > processes using the swap :) But you're right in the general case. > >=20 > Hmm... I wonder what gmirror is good for, when one of its consumers fails= =2E.. >=20 > I just setup this test setting: > 1. gnop on ad0s1gd > 2. gmirror on ad0s1gd.nop (hardcoded (-h)) and ad0s1ge > 3. dd (writes from /dev/urandom to the mirror) > 4. gnop configure -v -f 100 ad0s1gd.nop > 5. dd becomes unresponsive; CTRL+t says: > load: 0.78 cmd: dd 11034 [physwr] 0.01u 0.68s 0% 612k I can't reproduce it. What block size did you use for dd(1)? I did a lot of testing in the past with gmirror/graid3 and gnop(8), actually, this is why I implemented gnop(8) in the first place. If it doesn't work, it's a bug in gmirror, but I can't reproduce it with quite recent current. From what I see you're not using recent current, because there is no '-f' option for gnop(8) anymore, but it should also work with RELENG_6, so more info which will allow me to reproduce the problem would be helpful. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --5QAgd0e35j3NYeGe Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFm2q5ForvXbEpPzQRAmZHAJ4uXy2rF8H17S/sFLZ6ch196drhVwCcDaOk 3YFRwuB8TFkCp/x3LOWDNjo= =+ry4 -----END PGP SIGNATURE----- --5QAgd0e35j3NYeGe-- From owner-freebsd-geom@FreeBSD.ORG Wed Jan 3 09:41:54 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AF33B16A407 for ; Wed, 3 Jan 2007 09:41:54 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30306.mail.mud.yahoo.com (web30306.mail.mud.yahoo.com [209.191.69.68]) by mx1.freebsd.org (Postfix) with SMTP id 7C80813C465 for ; Wed, 3 Jan 2007 09:41:54 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 45527 invoked by uid 60001); 3 Jan 2007 09:41:50 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=xCipV/vcKrbCNK7JI+UVwPRMEuroou09N3ct2IcHbCZN6om+eXYeiDklBLldAtWn8s/EI8rgE1yReImbDAcIM7/gTkag/yCDOMWPUpUHTqx+flZZ+iiYXIcSiB/2K/+NZMhPrGIHbXRNoqcMgFeJEyOju4d4ItrjbsPIzMI4HCA= ; Message-ID: <20070103094150.45525.qmail@web30306.mail.mud.yahoo.com> X-YMail-OSG: VUq6tOEVM1m7.eDrkxTDPqYFcG5mDbqMe1C6A.mV Received: from [213.54.186.118] by web30306.mail.mud.yahoo.com via HTTP; Wed, 03 Jan 2007 01:41:49 PST Date: Wed, 3 Jan 2007 01:41:49 -0800 (PST) From: "R. B. Riddick" To: Pawel Jakub Dawidek In-Reply-To: <20070103083505.GB6253@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-geom@freebsd.org Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jan 2007 09:41:54 -0000 --- Pawel Jakub Dawidek wrote: > > I just setup this test setting: > > 1. gnop on ad0s1gd > > 2. gmirror on ad0s1gd.nop (hardcoded (-h)) and ad0s1ge > > 3. dd (writes from /dev/urandom to the mirror) > > 4. gnop configure -v -f 100 ad0s1gd.nop > > 5. dd becomes unresponsive; CTRL+t says: > > load: 0.78 cmd: dd 11034 [physwr] 0.01u 0.68s 0% 612k > > I can't reproduce it. What block size did you use for dd(1)? I did a > lot of testing in the past with gmirror/graid3 and gnop(8), actually, > this is why I implemented gnop(8) in the first place. If it doesn't > work, it's a bug in gmirror, but I can't reproduce it with quite recent > current. From what I see you're not using recent current, because there > is no '-f' option for gnop(8) anymore, but it should also work with > RELENG_6, so more info which will allow me to reproduce the problem > would be helpful. > > -- > Pawel Jakub Dawidek http://www.wheel.pl > pjd@FreeBSD.org http://www.FreeBSD.org > FreeBSD committer Am I Evil? Yes, I Am! > Oki Doke... It is a very sporadic error, that I was able to reproduce after an hour of desperate testing (I already thought, I had delusions yesterday): :-) In window number 1 we had this: neo# repeat 100 dd of=/dev/mirror/fook if=/dev/urandom bs=512 dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 15.717300 secs (3257524 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 14.194099 secs (3607097 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 18.362899 secs (2788203 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 17.022286 secs (3007792 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 14.196873 secs (3606392 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 20.076416 secs (2550230 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 20.647549 secs (2479688 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 14.124682 secs (3624824 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 17.892760 secs (2861464 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 18.004130 secs (2843763 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 15.764862 secs (3247697 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 16.149833 secs (3170280 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 18.820779 secs (2720370 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 15.577478 secs (3286764 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 18.791059 secs (2724673 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 24.638594 secs (2078020 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 24.352703 secs (2102415 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 24.075611 secs (2126612 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 24.332309 secs (2104177 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 24.459005 secs (2093278 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 24.170156 secs (2118294 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 24.209860 secs (2114820 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 24.144262 secs (2120565 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 18.784086 secs (2725684 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 20.041209 secs (2554711 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 14.122618 secs (3625354 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 14.088424 secs (3634153 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 17.949967 secs (2852344 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 16.368807 secs (3127869 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 14.365082 secs (3564163 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 14.138732 secs (3621222 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 20.906817 secs (2448937 bytes/sec) load: 0.92 cmd: dd 2352 [*Giant] 0.01u 1.08s 3% 696k 38596+0 records in 38595+0 records out 19760640 bytes transferred in 9.200829 secs (2147702 bytes/sec) load: 0.93 cmd: dd 2352 [physwr] 0.03u 1.52s 4% 696k 54424+0 records in 54423+0 records out 27864576 bytes transferred in 13.039194 secs (2136986 bytes/sec) load: 0.93 cmd: dd 2352 [physwr] 0.04u 1.80s 5% 696k 64547+0 records in 64546+0 records out 33047552 bytes transferred in 15.430266 secs (2141736 bytes/sec) load: 0.94 cmd: dd 2352 [physwr] 0.04u 1.94s 5% 696k 69424+0 records in 69423+0 records out 35544576 bytes transferred in 16.582293 secs (2143526 bytes/sec) load: 0.94 cmd: dd 2352 [runnable] 0.04u 1.99s 5% 696k 71177+0 records in 71177+0 records out 36442624 bytes transferred in 16.997564 secs (2143991 bytes/sec) load: 0.94 cmd: dd 2352 [physwr] 0.04u 2.02s 5% 696k 72332+0 records in 72331+0 records out 37033472 bytes transferred in 17.279865 secs (2143157 bytes/sec) load: 0.94 cmd: dd 2352 [physwr] 0.04u 2.04s 6% 696k 73318+0 records in 73317+0 records out 37538304 bytes transferred in 17.517803 secs (2142866 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 23.872245 secs (2144729 bytes/sec) load: 0.86 cmd: dd 2369 [physwr] 0.00u 0.05s 0% 696k 1870+0 records in 1869+0 records out 956928 bytes transferred in 0.439035 secs (2179617 bytes/sec) dd: /dev/mirror/fook: end of device 100000+0 records in 99999+0 records out 51199488 bytes transferred in 19.229316 secs (2662575 bytes/sec) load: 0.83 cmd: dd 2372 [physwr] 0.01u 1.70s 0% 696k load: 0.10 cmd: dd 2372 [physwr] 0.01u 1.70s 0% 696k load: 0.10 cmd: dd 2372 [physwr] 0.01u 1.70s 0% 696k load: 0.10 cmd: dd 2372 [physwr] 0.01u 1.70s 0% 696k load: 0.10 cmd: dd 2372 [physwr] 0.01u 1.70s 0% 696k load: 0.17 cmd: dd 2372 [physwr] 0.01u 1.70s 0% 696k [dd hangs here] In window #2 we have this: neo# gnop create ad0s1gd neo# gmirror label -h fook ad0s1gd.nop neo# gmirror insert fook ad0s1ge neo# gmirror status fook Name Status Components mirror/fook DEGRADED ad0s1gd.nop ad0s1ge (59%) neo# gmirror status fook Name Status Components mirror/fook COMPLETE ad0s1gd.nop ad0s1ge neo# echo start dd in another window start dd in another window neo# gnop configure -f 100 ad0s1gd.nop neo# gmirror status fook Name Status Components mirror/fook DEGRADED ad0s1ge neo# echo dd completed without premature error dd completed without premature error neo# gmirror forget fook neo# gnop configure -f 0 ad0s1gd.nop neo# gmirror insert fook ad0s1gd.nop neo# gmirror status Name Status Components mirror/sys COMPLETE ad0s1a ad1s1a mirror/home COMPLETE ad0s1d ad1s1d mirror/fook DEGRADED ad0s1ge ad0s1gd (39%) [...] [...hours later...] [...] neo# gmirror forget fook ; gmirror insert fook ad0s1gd neo# gmirror status Name Status Components mirror/sys COMPLETE ad0s1a ad1s1a mirror/home COMPLETE ad0s1d ad1s1d mirror/fook DEGRADED ad0s1ge ad0s1gd (48%) neo# gnop configure -f 100 ad0s1gd.nop neo# gmirror status Name Status Components mirror/sys COMPLETE ad0s1a ad1s1a mirror/home COMPLETE ad0s1d ad1s1d mirror/fook COMPLETE ad0s1ge ad0s1gd neo# gmirror remove fook ad0s1gd neo# gmirror status Name Status Components mirror/sys COMPLETE ad0s1a ad1s1a mirror/home COMPLETE ad0s1d ad1s1d mirror/fook COMPLETE ad0s1ge neo# gmirror forget fook ; gmirror insert -h fook ad0s1gd.nop Cannot store metadata on ad0s1gd.nop. neo# gnop configure -f 0 ad0s1gd.nop neo# gmirror forget fook ; gmirror insert -h fook ad0s1gd.nop neo# gmirror status Name Status Components mirror/sys COMPLETE ad0s1a ad1s1a mirror/home COMPLETE ad0s1d ad1s1d mirror/fook DEGRADED ad0s1ge ad0s1gd.nop (66%) neo# gmirror status Name Status Components mirror/sys COMPLETE ad0s1a ad1s1a mirror/home COMPLETE ad0s1d ad1s1d mirror/fook COMPLETE ad0s1ge ad0s1gd.nop neo# gmirror status Name Status Components mirror/sys COMPLETE ad0s1a ad1s1a mirror/home COMPLETE ad0s1d ad1s1d mirror/fook COMPLETE ad0s1ge ad0s1gd.nop neo# gnop configure -f 100 ad0s1gd.nop You have new mail. neo# gmirror status Name Status Components mirror/sys COMPLETE ad0s1a ad1s1a mirror/home COMPLETE ad0s1d ad1s1d mirror/fook COMPLETE ad0s1ge ad0s1gd.nop neo# gmirror list fook Geom name: fook State: COMPLETE Components: 2 Balance: split Slice: 4096 Flags: NONE GenID: 15 SyncID: 1 ID: 1003357718 Providers: 1. Name: mirror/fook Mediasize: 51199488 (49M) Sectorsize: 512 Mode: r1w1e0 Consumers: 1. Name: ad0s1ge Mediasize: 51200000 (49M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: NONE GenID: 15 SyncID: 1 ID: 1399415263 2. Name: ad0s1gd.nop Mediasize: 51200000 (49M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: HARDCODED GenID: 15 SyncID: 1 ID: 617307948 [gmirror does not detect the dead disk] -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Thu Jan 4 13:42:53 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B8DCC16A492 for ; Thu, 4 Jan 2007 13:42:53 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.freebsd.org (Postfix) with ESMTP id 4BEA113C465 for ; Thu, 4 Jan 2007 13:42:53 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.8/8.13.8) with ESMTP id l04DgpnT027874; Thu, 4 Jan 2007 07:42:51 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <459D045C.6020906@centtech.com> Date: Thu, 04 Jan 2007 07:42:52 -0600 From: Eric Anderson User-Agent: Thunderbird 1.5.0.9 (X11/20061223) MIME-Version: 1.0 To: Rong-en Fan References: <6eb82e0701022200i7a2cb356g358a1938d620e97a@mail.gmail.com> In-Reply-To: <6eb82e0701022200i7a2cb356g358a1938d620e97a@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/2413/Thu Jan 4 03:46:27 2007 on mh1.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh1.centtech.com Cc: freebsd-geom@freebsd.org Subject: Re: gconcat and gpt X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2007 13:42:53 -0000 On 01/03/07 00:00, Rong-en Fan wrote: > Hi, > > I'm running 6.2-RC1 on an i386 box. I'm testing gjournal patch. > I found that something is annoying... I use > > gconcat label big da1 da2 da3 > > and then running gpt on the concat/big device. So, I got > concat/bigp1, concat/bigp2, ... etc *AND* I also have > da1p1, da1p2... Well, this is not a problem for normal use. However, > when I enable gjournal, at boot time, gconcat first activates the > big one then, gjournal finds there is journal data on da1p1, so > it uses! Then, da1 is deactivated from gconcat... > > My question is, why I got da1p1 when da1 is used by gconcat? Did you use gjournal against the da* devices, or the gconcat device? If you ever used it on the da* devices, the meta data might still be there confusing things. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology An undefined problem has an infinite number of solutions. ------------------------------------------------------------------------ From owner-freebsd-geom@FreeBSD.ORG Thu Jan 4 14:21:43 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CF1EB16A403 for ; Thu, 4 Jan 2007 14:21:43 +0000 (UTC) (envelope-from grafan@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.240]) by mx1.freebsd.org (Postfix) with ESMTP id 944F813C45B for ; Thu, 4 Jan 2007 14:21:43 +0000 (UTC) (envelope-from grafan@gmail.com) Received: by an-out-0708.google.com with SMTP id c24so1758321ana for ; Thu, 04 Jan 2007 06:20:17 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=aZ47XqmSKg0GLFQ/gbJVnbjHU9UF2I1QpAcqOnohmoSWNPQ9zvm3KkEaoS22e66Uu8GBlJJJNejboQ8C+AjguqtWgxNgBRgwZNx1c95G54EyEUvCi5PlMc8VhCkwnsYFp2KfwJli81wFZ7zJyHH6vyGTNPAZ/NjpMkOd/N540yI= Received: by 10.100.37.4 with SMTP id k4mr7440133ank.1167920417971; Thu, 04 Jan 2007 06:20:17 -0800 (PST) Received: by 10.100.136.16 with HTTP; Thu, 4 Jan 2007 06:20:17 -0800 (PST) Message-ID: <6eb82e0701040620w2071876ar2f832775d7a8742a@mail.gmail.com> Date: Thu, 4 Jan 2007 22:20:17 +0800 From: "Rong-en Fan" To: "Eric Anderson" In-Reply-To: <459D045C.6020906@centtech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <6eb82e0701022200i7a2cb356g358a1938d620e97a@mail.gmail.com> <459D045C.6020906@centtech.com> Cc: freebsd-geom@freebsd.org Subject: Re: gconcat and gpt X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2007 14:21:43 -0000 On 1/4/07, Eric Anderson wrote: > On 01/03/07 00:00, Rong-en Fan wrote: > > Hi, > > > > I'm running 6.2-RC1 on an i386 box. I'm testing gjournal patch. > > I found that something is annoying... I use > > > > gconcat label big da1 da2 da3 > > > > and then running gpt on the concat/big device. So, I got > > concat/bigp1, concat/bigp2, ... etc *AND* I also have > > da1p1, da1p2... Well, this is not a problem for normal use. However, > > when I enable gjournal, at boot time, gconcat first activates the > > big one then, gjournal finds there is journal data on da1p1, so > > it uses! Then, da1 is deactivated from gconcat... > > > > My question is, why I got da1p1 when da1 is used by gconcat? > > Did you use gjournal against the da* devices, or the gconcat device? If > you ever used it on the da* devices, the meta data might still be there > confusing things. I use gjournal against gpt'ed gconcat device. From owner-freebsd-geom@FreeBSD.ORG Thu Jan 4 21:50:07 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 55E6416A415 for ; Thu, 4 Jan 2007 21:50:07 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.174]) by mx1.freebsd.org (Postfix) with ESMTP id D7A4413C46C for ; Thu, 4 Jan 2007 21:50:06 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: by ug-out-1314.google.com with SMTP id o2so4929928uge for ; Thu, 04 Jan 2007 13:50:05 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:user-agent:mime-version:to:cc:subject:references:in-reply-to:x-enigmail-version:content-type:content-transfer-encoding; b=sNxyVRIQ/fd68oR/EsEeQbOnvhjT66qPH3Zh8bDs7jHz2c6VSk2CzUL6MPsJYR/IyXlCK136JUXeNkb+HmSVmC03aotEiUsofqOXOiGEqfoH7oHPZukF0P+FIFmSXA4TFXmSALGXlKdBuACOtSAMregvkB2uh7Kiv2kSkxcQq4M= Received: by 10.67.106.3 with SMTP id i3mr11436155ugm.1167947373627; Thu, 04 Jan 2007 13:49:33 -0800 (PST) Received: from ?192.168.123.201? ( [195.241.221.201]) by mx.google.com with ESMTP id j33sm28828390ugc.2007.01.04.13.49.32; Thu, 04 Jan 2007 13:49:32 -0800 (PST) Message-ID: <459D766B.9050304@gmail.com> Date: Thu, 04 Jan 2007 22:49:31 +0100 From: Rene Ladan User-Agent: Thunderbird 1.5.0.9 (X11/20061224) MIME-Version: 1.0 To: Poul-Henning Kamp References: <74225.1167686662@critter.freebsd.dk> In-Reply-To: <74225.1167686662@critter.freebsd.dk> X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-geom@freebsd.org Subject: Re: xbox360 extension for review/debug X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jan 2007 21:50:07 -0000 Poul-Henning Kamp schreef: > In message <45996C4F.8070700@gmail.com>, Rene Ladan writes: >> Hi, >> >> I've written an extension to /sys/geom/geom_mbr.c to slice up xbox360 >> hard disks and memory units. The patch for revision 1.68 (i.e. CURRENT) >> is at http://home.tisali.nl/rladan/freebsd/geom_mbr.c.diff > > This is wrong, you should make a geom_xbox360 class instead. > Ok, I made a geom_xbox360 class and restored geom_mbr.c The new class consists of three files at the above website. geom_xbox360.c : the class, put it in /sys/geom geom_xbox360::Makefile : Makefile for /sys/modules/geom/geom_xbox360/ Makefile.diff : patch for /sys/modules/geom/Makefile Note that the kmod still panics when loading it, currently at g_xbox360_taste+0x78 Regards, Rene -- GPG fingerprint = E738 5471 D185 7013 0EE0 4FC8 3C1D 6F83 12E1 84F6 (subkeys.pgp.net) "It won't fit on the line." -- me, 2001 From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 01:19:58 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7CBE016A415; Fri, 5 Jan 2007 01:19:58 +0000 (UTC) (envelope-from lulf@stud.ntnu.no) Received: from merke.itea.ntnu.no (merke.itea.ntnu.no [129.241.7.61]) by mx1.freebsd.org (Postfix) with ESMTP id 0FF1713C457; Fri, 5 Jan 2007 01:19:58 +0000 (UTC) (envelope-from lulf@stud.ntnu.no) Received: from localhost (localhost [127.0.0.1]) by merke.itea.ntnu.no (Postfix) with ESMTP id B43ED13C5BA; Fri, 5 Jan 2007 01:58:00 +0100 (CET) Received: from webmail.ntnu.no (textus12.itea.ntnu.no [129.241.56.162]) by merke.itea.ntnu.no (Postfix) with ESMTP; Fri, 5 Jan 2007 01:58:00 +0100 (CET) Received: from 53.3.erx-lhm.eidsiva.net (53.3.erx-lhm.eidsiva.net [87.248.3.53]) by webmail.ntnu.no (Horde MIME library) with HTTP; Fri, 5 Jan 2007 01:58:00 +0100 Message-ID: <20070105015800.s3rqdzgm8k8owk4s@webmail.ntnu.no> Date: Fri, 5 Jan 2007 01:58:00 +0100 From: lulf@stud.ntnu.no To: freebsd-geom@freebsd.org, freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.3) X-Content-Scanned: with sophos and spamassassin at mailgw.ntnu.no. Cc: Subject: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 01:19:58 -0000 Hi, I was wondering if someone have started on the pluggable =20 disk-scheduler project on the "new ideas"-page yet. I was thinking on how one could implement this in GEOM by creating a =20 lightweight scheduler API/Framework integrated in GEOM. The framework would be in =20 charge of changing which schedulers that are to be used by the g_up and g_down threads= . I've put down some design goals for this: 1. Little/no overhead in I/O processing with default scheduling =20 compared to the "old" way. 2. Easily modifiable, preferable on-the-fly switching of schedulers. 3. Make it possible to many different schedulers to be implemented, without creating a too alien interface too them, but at the same time not restric= t them too much. More specifically my plan was to change the =20 g_up_procbody/g_down_procbody to ask the scheduler framework on which scheduler to use, and then further implement =20 procedures in that framework to handle the details of loading, switching and unloading =20 different schedulers for I/O. Then I would extract out the default I/O scheduler and try out some other ways to schedule I/O. Also, I'm not sure how I would handle each schedulers way to organize the queue. One should allow for different types of bioq's for the schedulers since they may have different needs of organizi= ng queues (like a heap maybe). I've started with some of my tampering in a p4 branch lulf_gpds. I have a DESCRIPTION document that would maybe explain some of my thoughts and proble= ms further. Some small code are written, but I want to hear some others =20 thoughts on this before I go crashing around doing stuff I might hate =20 later I did :) I was also thinking of an alternative way to implement this like a "gpds"-layer that could provide different schedulers to service I/O requests= , because that would make it to fine-grain more on scheduling, say telling tha= t the system-drive is used in a characteristic way and that one specific =20 scheduler algorithm is more appropriate there, and another drive is having a different =20 characteristic which then should use a different algorithm. However, this should be doable directly in geom as previously described, but with a bit more tampering with other code. This is probably the most efficie= nt way since it has no overhead of another GEOM class. I also have some questions about the GEOM layer in itself. Does the VM-manag= er actually swap pages out to disk via GEOM, or does it do that by itself (whic= h would make more sense in terms of efficiency). I'd like to hear from some of the GEOM gurus' view on this. Is the something that sounds doable and worth spending time on? Is there something I've overlooked? Have I completely lost my mind? I sometimes have the ability to write a bit different than what my mind is thinking sometimes :) Anyway, I'd like to research a bit on this topic to just see how much =20 it does matter with different I/O scheduling for different purposes. Comments are welcomed! --=20 Ulf Lilleengen From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 04:30:05 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B7B2516A412 for ; Fri, 5 Jan 2007 04:30:05 +0000 (UTC) (envelope-from tom.hurst@clara.net) Received: from spork.qfe3.net (spork.qfe3.net [212.13.207.101]) by mx1.freebsd.org (Postfix) with ESMTP id 6989613C448 for ; Fri, 5 Jan 2007 04:30:05 +0000 (UTC) (envelope-from tom.hurst@clara.net) Received: from [81.104.144.87] (helo=voi.aagh.net) by spork.qfe3.net with esmtp (Exim 4.62 (FreeBSD)) (envelope-from ) id 1H2giN-000ERA-Pm; Fri, 05 Jan 2007 04:29:59 +0000 Received: from freaky by voi.aagh.net with local (Exim 4.63 (FreeBSD)) (envelope-from ) id 1H2giN-0007sC-Gj; Fri, 05 Jan 2007 04:29:59 +0000 Date: Fri, 5 Jan 2007 04:29:59 +0000 From: Thomas Hurst To: "Rick C. Petty" Message-ID: <20070105042959.GA28563@voi.aagh.net> Mail-Followup-To: "Rick C. Petty" , freebsd-geom@freebsd.org, michael.knoll@gmail.com References: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> <20070101215016.GA71635@voi.aagh.net> <20070102195201.GC83431@keira.kiwi-computer.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070102195201.GC83431@keira.kiwi-computer.com> Organization: Not much. User-Agent: Mutt/1.5.13 (2006-08-11) Sender: Thomas Hurst Cc: michael.knoll@gmail.com, freebsd-geom@freebsd.org Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 04:30:05 -0000 * Rick C. Petty (rick-freebsd@kiwi-computer.com) wrote: > Where is that rc.conf option defined or used? Certainly not in 6.x: > > % grep -R swapoff /etc/ > /etc/security/audit_event:355:AUE_DARWIN_SWAPOFF:swapoff():ad > /etc/security/audit_event:43045:AUE_SWAPOFF:swapoff():ad Indeed; it didn't last long, but the legacy seems to remain in some guides: http://www.freebsd.org/cgi/cvsweb.cgi/src/etc/rc.d/swap1 Revision 1.6.2.4 Mon Jan 3 12:41:39 2005 UTC Gmirror now uses shutdown hooks to stop itself in a clean way, that's why we don't need stop method for rc.d/swap1 script anymore. Revision 1.6.2.3 Sat Oct 16 08:13:19 2004 UTC Do not remove swap partitions by default as it cause some problems. Instead, add 'swapoff' variable to rc.conf (turned off by default), so users who use gmirror and swap-on-mirror configuration still can turn it on and avoid problems with dirty mirror components on next boot. -- Thomas 'Freaky' Hurst http://hur.st/ From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 09:18:55 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8A1B616A403; Fri, 5 Jan 2007 09:18:55 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id 4087F13C45B; Fri, 5 Jan 2007 09:18:55 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A5D4F2.dip.t-dialin.net [84.165.212.242]) by redbull.bpaserver.net (Postfix) with ESMTP id 68A472E204; Fri, 5 Jan 2007 10:03:32 +0100 (CET) Received: from webmail.leidinger.net (webmail.Leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id B34745B497E; Fri, 5 Jan 2007 09:58:53 +0100 (CET) Received: (from www@localhost) by webmail.leidinger.net (8.13.8/8.13.8/Submit) id l058wrwH068816; Fri, 5 Jan 2007 09:58:53 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde MIME library) with HTTP; Fri, 05 Jan 2007 09:58:53 +0100 Message-ID: <20070105095853.705qf42sgkcoww4w@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Fri, 05 Jan 2007 09:58:53 +0100 From: Alexander Leidinger To: lulf@stud.ntnu.no References: <20070105015800.s3rqdzgm8k8owk4s@webmail.ntnu.no> In-Reply-To: <20070105015800.s3rqdzgm8k8owk4s@webmail.ntnu.no> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) H3 (4.1.3) / FreeBSD-7.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-14.864, required 6, autolearn=not spam, BAYES_00 -15.00, DK_POLICY_SIGNSOME 0.00, FORGED_RCVD_HELO 0.14) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: joel@freebsd.org, freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 09:18:55 -0000 Quoting lulf@stud.ntnu.no (from Fri, 5 Jan 2007 01:58:00 +0100): > Hi, > > I was wondering if someone have started on the pluggable disk-scheduler > project > on the "new ideas"-page yet. As long as there's not an entry saying that someone is looking at it we (Joel and me) are not aware of someone working on it (but in general there may be still an entry somewhere about someone but no progress... Joel, we should ping them I think). Bye, Alexander. -- I will not be briefed or debriefed, my underwear is my own. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 09:53:23 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AEDA516A494 for ; Fri, 5 Jan 2007 09:53:23 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30301.mail.mud.yahoo.com (web30301.mail.mud.yahoo.com [209.191.69.63]) by mx1.freebsd.org (Postfix) with SMTP id 740F313C441 for ; Fri, 5 Jan 2007 09:53:23 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 99250 invoked by uid 60001); 5 Jan 2007 09:53:22 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=hkSxbywgGMTVF+/GLvSWTSRCVUpCGqr/5pGoXvWmJK/AbUawkm8r5vMJf03rPoWCvGMKuwobaXbU+r72J3eNug8DBIKEKBX08SmUzksNVrmJLywFbwYHmqFkm6narCpPZ1OLIWmeGwNbXclVJjH+PGlfd4ZEjPdEtM3L0EUCRWg=; X-YMail-OSG: 8xhOjN0VM1luv79LkFTJ212crp4Q06yQC1ZYuJaJ.njYrlU3GzM0pNd49KqA4CHT.mncKHrbnbKFX9cx5HlYUaL7eG5WraWDnSo5xAuE8ZiczI7yvmxpbAxflzvLio4haCxnBT1CiVQ.j9kGoy11ySoFF57kQI.ln2KiflOQVGdXZilNEGaNI7Tcu7k6 Received: from [85.212.3.5] by web30301.mail.mud.yahoo.com via HTTP; Fri, 05 Jan 2007 01:53:22 PST Date: Fri, 5 Jan 2007 01:53:22 -0800 (PST) From: "R. B. Riddick" To: lulf@stud.ntnu.no, freebsd-geom@freebsd.org, freebsd-current@freebsd.org In-Reply-To: <20070105015800.s3rqdzgm8k8owk4s@webmail.ntnu.no> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <703692.97797.qm@web30301.mail.mud.yahoo.com> Cc: Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 09:53:23 -0000 --- lulf@stud.ntnu.no wrote: > for I/O. Then I would extract out the default I/O scheduler and try out some > other ways to schedule I/O. Also, I'm not sure how I would handle each > schedulers way to organize the queue. One should allow for different types > of bioq's for the schedulers since they may have different needs of > organizing queues (like a heap maybe). > I would _guess_ for features like in TCQ (Tagged Command Queuing) or NCQ (Native Command Queuing) the g_down thread is the wrong place, because: It would need knowledge about the underlying GEOM device, so that it can determine the correct moment to start a certain read request (currently g_down (g_io_schedule_down in src/sys/geom/geom_io.c) sees most likely just one request in its queue before it goes to sleep again, so that reordering this single one request would make no sense; if one delays read request this might lead to worse performance than before, because the disk handles simultaneous read requests better). For write requests this (TCQ/NCQ related scheduling) might be easier, because they can be delayed and delivered back up (g_up) early (I did that in graid5). A further enhancement might be the combination of requests (e.g. if they overlap or if they (nearly) adjoin to each other), which could be done in g_down, too (I do that in graid5, which leaded to a lot of bcopy-activity). -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 10:18:29 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 26DAA16A416; Fri, 5 Jan 2007 10:18:29 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id DA2C013C468; Fri, 5 Jan 2007 10:18:28 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 553951747B; Fri, 5 Jan 2007 10:18:27 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.8/8.13.8) with ESMTP id l05AIO6U003764; Fri, 5 Jan 2007 10:18:24 GMT (envelope-from phk@critter.freebsd.dk) To: lulf@stud.ntnu.no From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 05 Jan 2007 01:58:00 +0100." <20070105015800.s3rqdzgm8k8owk4s@webmail.ntnu.no> Date: Fri, 05 Jan 2007 10:18:24 +0000 Message-ID: <3763.1167992304@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 10:18:29 -0000 In message <20070105015800.s3rqdzgm8k8owk4s@webmail.ntnu.no>, lulf@stud.ntnu.no writes: >I was wondering if someone have started on the pluggable >disk-scheduler project >on the "new ideas"-page yet. > >I was thinking on how one could implement this in GEOM by [...] Sorting and scheduling should only be implemented at selected points in the GEOM mesh and it can even be argued that it should only be implemented at the bottom most level, right above the physical storage media. But given the increasing intelligence of disk drives in this area, I would also caution against expecting too much of a gain in reality. So before you embark on a major expedition, I would suggest that you do some benchmarking and experiments with the current disksorting, to shed light on the potential for improvement. Here are some ideas: Remove disksorting and see if if and how big a difference it makes today. Test both SCSI, ATA and USB media, and test both low-level benchmarks and "real-world" workloads. Change disksorting to reverse unidirectional elevator and bidirectional elevator and see if it makes a difference. (Modern disks store blocks in reverse sector order on the disk, discover and explain why) Capture an I/O trace from a suitably sensible realworld system, including the detailed timestamps of issuance and completion of the requests. Treat results statistically and try to determine a formula for predicting how long a given request is going to take for the disk. It's not that I think that all your ideas are bad, I am just not sure that the (traditional) view of the hardware they are based on, is still relevant, and I think your time would be much better spent addressing that question. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 11:31:30 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4593E16A407; Fri, 5 Jan 2007 11:31:30 +0000 (UTC) (envelope-from lulf@stud.ntnu.no) Received: from signal.itea.ntnu.no (signal.itea.ntnu.no [129.241.190.231]) by mx1.freebsd.org (Postfix) with ESMTP id F08FD13C455; Fri, 5 Jan 2007 11:31:29 +0000 (UTC) (envelope-from lulf@stud.ntnu.no) Received: from localhost (localhost [127.0.0.1]) by signal.itea.ntnu.no (Postfix) with ESMTP id AFF9333A89; Fri, 5 Jan 2007 12:31:27 +0100 (CET) Received: from webmail.ntnu.no (textus12.itea.ntnu.no [129.241.56.162]) by signal.itea.ntnu.no (Postfix) with ESMTP; Fri, 5 Jan 2007 12:31:27 +0100 (CET) Received: from 53.3.erx-lhm.eidsiva.net (53.3.erx-lhm.eidsiva.net [87.248.3.53]) by webmail.ntnu.no (Horde MIME library) with HTTP; Fri, 5 Jan 2007 12:31:27 +0100 Message-ID: <20070105123127.gnk0v58p44488g48@webmail.ntnu.no> Date: Fri, 5 Jan 2007 12:31:27 +0100 From: lulf@stud.ntnu.no To: Poul-Henning Kamp References: <3763.1167992304@critter.freebsd.dk> In-Reply-To: <3763.1167992304@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.3) X-Content-Scanned: with sophos and spamassassin at mailgw.ntnu.no. Cc: freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 11:31:30 -0000 Siterer Poul-Henning Kamp : > In message <20070105015800.s3rqdzgm8k8owk4s@webmail.ntnu.no>, =20 > lulf@stud.ntnu.no > writes: > >> I was wondering if someone have started on the pluggable >> disk-scheduler project >> on the "new ideas"-page yet. >> >> I was thinking on how one could implement this in GEOM by [...] > *snip* > > Here are some ideas: > > Remove disksorting and see if if and how big a difference > it makes today. Test both SCSI, ATA and USB media, and > test both low-level benchmarks and "real-world" workloads. > > Change disksorting to reverse unidirectional elevator > and bidirectional elevator and see if it makes a difference. > (Modern disks store blocks in reverse sector order on > the disk, discover and explain why) > > Capture an I/O trace from a suitably sensible realworld > system, including the detailed timestamps of issuance > and completion of the requests. Treat results statistically > and try to determine a formula for predicting how long > a given request is going to take for the disk. > > It's not that I think that all your ideas are bad, I am just > not sure that the (traditional) view of the hardware they > are based on, is still relevant, and I think your time would > be much better spent addressing that question. > I understand, and I clearly see the point about new hardware being =20 more intelligent in these matters. However, I will look into this a bit more just out of curiosity, and =20 do some actual test on how this can affect performance in the =20 scenarios you describe. And thanks for the tips! --=20 Ulf Lilleengen From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 11:37:33 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8ACBE16A415; Fri, 5 Jan 2007 11:37:33 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 4987313C465; Fri, 5 Jan 2007 11:37:33 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 17E881747C; Fri, 5 Jan 2007 11:37:32 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.8/8.13.8) with ESMTP id l05BbTaU004086; Fri, 5 Jan 2007 11:37:29 GMT (envelope-from phk@critter.freebsd.dk) To: lulf@stud.ntnu.no From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 05 Jan 2007 12:31:27 +0100." <20070105123127.gnk0v58p44488g48@webmail.ntnu.no> Date: Fri, 05 Jan 2007 11:37:29 +0000 Message-ID: <4085.1167997049@critter.freebsd.dk> Sender: phk@critter.freebsd.dk Cc: freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 11:37:33 -0000 In message <20070105123127.gnk0v58p44488g48@webmail.ntnu.no>, lulf@stud.ntnu.no writes: >However, I will look into this a bit more just out of curiosity, and =20 >do some actual test on how this can affect performance in the =20 >scenarios you describe. And thanks for the tips! Keep us posted, I am interested in your results even if they don't show anything remarkable. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 13:52:17 2007 Return-Path: X-Original-To: freebsd-geom@hub.freebsd.org Delivered-To: freebsd-geom@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AAA7516A407; Fri, 5 Jan 2007 13:52:17 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [69.147.83.40]) by mx1.freebsd.org (Postfix) with ESMTP id 807E713C442; Fri, 5 Jan 2007 13:52:17 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from freefall.freebsd.org (kib@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id l05DqHAF086929; Fri, 5 Jan 2007 13:52:17 GMT (envelope-from kib@freefall.freebsd.org) Received: (from kib@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id l05DqH5T086925; Fri, 5 Jan 2007 13:52:17 GMT (envelope-from kib) Date: Fri, 5 Jan 2007 13:52:17 GMT From: Konstantin Belousov Message-Id: <200701051352.l05DqH5T086925@freefall.freebsd.org> To: trasz@pin.if.uz.zgora.lp, kib@FreeBSD.org, freebsd-geom@FreeBSD.org Cc: Subject: Re: kern/105390: [geli] filesystem on a md backed by sparse file with softupdates enabled leads to wdrain livelock. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 13:52:17 -0000 Synopsis: [geli] filesystem on a md backed by sparse file with softupdates enabled leads to wdrain livelock. State-Changed-From-To: open->feedback State-Changed-By: kib State-Changed-When: Fri Jan 5 13:50:17 UTC 2007 State-Changed-Why: I committed a fix for md deadlocks, namely 1.153.2.8 +9 -0 src/sys/dev/md/md.c 1.491.2.10 +9 -1 src/sys/kern/vfs_bio.c 1.304.2.8 +1 -0 src/sys/sys/vnode.h Please, report whether it helps. http://www.freebsd.org/cgi/query-pr.cgi?pr=105390 From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 14:38:43 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 901C016A40F for ; Fri, 5 Jan 2007 14:38:43 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5022813C46A for ; Fri, 5 Jan 2007 14:38:43 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 07EE94C603; Fri, 5 Jan 2007 09:14:27 -0500 (EST) Date: Fri, 5 Jan 2007 14:14:26 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: lulf@stud.ntnu.no In-Reply-To: <20070105015800.s3rqdzgm8k8owk4s@webmail.ntnu.no> Message-ID: <20070105140941.B98541@fledge.watson.org> References: <20070105015800.s3rqdzgm8k8owk4s@webmail.ntnu.no> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 14:38:43 -0000 On Fri, 5 Jan 2007, lulf@stud.ntnu.no wrote: > Anyway, I'd like to research a bit on this topic to just see how much it > does matter with different I/O scheduling for different purposes. I think working on this is interesting, but the one caution I'd have is that it's possibly to introduce serious priority inversions through any complex scheduling scheme for I/O. In our VFS, I/O is frequently performed while holding locks or things that act like locks -- for example, during a directory lookup, while pulling an inode off the disk, etc. The I/O will be initiated by one thread, but then other threads will end up waiting for it also. If there is a naive mapping of initiating thread priority to I/O request priority, then you can end up with high priority threads being blocked on a low priority tasks, leading to nasty starvation effects, especially if the scheduler allows indefinite waiting for I/O at a low priority. This, at a rough approximation, is the problem that Kirk ran into when trying to rate limit bgfsck I/O in the kernel: key vnode locks, such as directory vnode locks, would be held across de-prioritized I/O, and high priority processes would then block on the vnode locks. There are various ways to address this, not least priority propagation (in which I/O priority is increased to match the priority of the highest priority thread waiting on the I/O request), but I wanted to make sure you had it on the list of design concerns. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 17:46:23 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7708816A40F; Fri, 5 Jan 2007 17:46:23 +0000 (UTC) (envelope-from lulf@stud.ntnu.no) Received: from fri.itea.ntnu.no (fri.itea.ntnu.no [129.241.7.60]) by mx1.freebsd.org (Postfix) with ESMTP id 339D613C45A; Fri, 5 Jan 2007 17:46:23 +0000 (UTC) (envelope-from lulf@stud.ntnu.no) Received: from localhost (localhost [127.0.0.1]) by fri.itea.ntnu.no (Postfix) with ESMTP id 2F4F580C2; Fri, 5 Jan 2007 18:46:22 +0100 (CET) Received: from webmail.ntnu.no (textus11.itea.ntnu.no [129.241.56.161]) by fri.itea.ntnu.no (Postfix) with ESMTP; Fri, 5 Jan 2007 18:46:21 +0100 (CET) Received: from 53.3.erx-lhm.eidsiva.net (53.3.erx-lhm.eidsiva.net [87.248.3.53]) by webmail.ntnu.no (Horde MIME library) with HTTP; Fri, 5 Jan 2007 18:46:21 +0100 Message-ID: <20070105184621.dh8kgoy7ko4gk4gc@webmail.ntnu.no> Date: Fri, 5 Jan 2007 18:46:21 +0100 From: lulf@stud.ntnu.no To: Ivan Voras References: <20070105123127.gnk0v58p44488g48@webmail.ntnu.no> <4085.1167997049@critter.freebsd.dk> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) H3 (4.1.3) X-Content-Scanned: with sophos and spamassassin at mailgw.ntnu.no. Cc: freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 17:46:23 -0000 Siterer Ivan Voras : > Poul-Henning Kamp wrote: >> In message <20070105123127.gnk0v58p44488g48@webmail.ntnu.no>, >> lulf@stud.ntnu.no >> writes: >> >> >>> However, I will look into this a bit more just out of curiosity, and =20 >>> do some actual test on how this can affect performance in the =20 >>> scenarios you describe. And thanks for the tips! >> >> Keep us posted, I am interested in your results even if they don't >> show anything remarkable. > > Have both of you seen this: > http://wikitest.freebsd.org/Hybrid Yes, this was what I was using as a basis. I forgot to paste the link, but I assumed people would know about it since it was referred to on the "ideas" page that I was referring to :) However, there was little actual test-result on that page, which would be nice to have. -- Ulf Lilleengen From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 18:46:57 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A379C16A412; Fri, 5 Jan 2007 18:46:57 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.freebsd.org (Postfix) with ESMTP id 2826213C441; Fri, 5 Jan 2007 18:46:57 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.13.6) with ESMTP id l05IYH2R091528; Fri, 5 Jan 2007 10:34:17 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id l05IYHUM091527; Fri, 5 Jan 2007 10:34:17 -0800 (PST) (envelope-from rizzo) Date: Fri, 5 Jan 2007 10:34:17 -0800 From: Luigi Rizzo To: Robert Watson Message-ID: <20070105103417.B91349@xorpc.icir.org> References: <20070105015800.s3rqdzgm8k8owk4s@webmail.ntnu.no> <20070105140941.B98541@fledge.watson.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20070105140941.B98541@fledge.watson.org>; from rwatson@freebsd.org on Fri, Jan 05, 2007 at 02:14:26PM +0000 Cc: freebsd-current@freebsd.org, lulf@stud.ntnu.no, freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 18:46:57 -0000 On Fri, Jan 05, 2007 at 02:14:26PM +0000, Robert Watson wrote: > > On Fri, 5 Jan 2007, lulf@stud.ntnu.no wrote: > > > Anyway, I'd like to research a bit on this topic to just see how much it > > does matter with different I/O scheduling for different purposes. > > I think working on this is interesting, but the one caution I'd have is that > it's possibly to introduce serious priority inversions through any complex > scheduling scheme for I/O. In our VFS, I/O is frequently performed while > holding locks or things that act like locks -- for example, during a directory > lookup, while pulling an inode off the disk, etc. The I/O will be initiated > by one thread, but then other threads will end up waiting for it also. If > there is a naive mapping of initiating thread priority to I/O request > priority, then you can end up with high priority threads being blocked on a > low priority tasks, leading to nasty starvation effects, especially if the > scheduler allows indefinite waiting for I/O at a low priority. This, at a that's a problem with priority based schedulers. neither the elevator nor the proportional-fair scheduler in Hybrid, nor a plain FIFO scheduler soffer from this 'indefinite waiting' problem. If i remember well the elevator code in freebsd had some support for 'prioritized' requests that could go in front of the queue no matter what, but i think that part was not really used. cheers luigi From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 18:46:57 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EDA0F16A47C; Fri, 5 Jan 2007 18:46:57 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.freebsd.org (Postfix) with ESMTP id A591213C44C; Fri, 5 Jan 2007 18:46:57 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.13.6) with ESMTP id l05IT5Po091469; Fri, 5 Jan 2007 10:29:05 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id l05IT5a9091468; Fri, 5 Jan 2007 10:29:05 -0800 (PST) (envelope-from rizzo) Date: Fri, 5 Jan 2007 10:29:05 -0800 From: Luigi Rizzo To: lulf@stud.ntnu.no Message-ID: <20070105102905.A91349@xorpc.icir.org> References: <20070105123127.gnk0v58p44488g48@webmail.ntnu.no> <4085.1167997049@critter.freebsd.dk> <20070105184621.dh8kgoy7ko4gk4gc@webmail.ntnu.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20070105184621.dh8kgoy7ko4gk4gc@webmail.ntnu.no>; from lulf@stud.ntnu.no on Fri, Jan 05, 2007 at 06:46:21PM +0100 Cc: freebsd-current@freebsd.org, Ivan Voras , freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 18:46:58 -0000 On Fri, Jan 05, 2007 at 06:46:21PM +0100, lulf@stud.ntnu.no wrote: > Siterer Ivan Voras : > > > Poul-Henning Kamp wrote: > >> In message <20070105123127.gnk0v58p44488g48@webmail.ntnu.no>, > >> lulf@stud.ntnu.no > >> writes: > >> > >> > >>> However, I will look into this a bit more just out of curiosity, and =20 > >>> do some actual test on how this can affect performance in the =20 > >>> scenarios you describe. And thanks for the tips! > >> > >> Keep us posted, I am interested in your results even if they don't > >> show anything remarkable. > > > > Have both of you seen this: > > http://wikitest.freebsd.org/Hybrid > > Yes, this was what I was using as a basis. > I forgot to paste the link, but I assumed people would know about it > since it was referred to on the "ideas" page that I was referring to :) > > However, there was little actual test-result on that page, which would > be nice to have. i can partly comment on the test results. Once you manage to make queues build up at the scheduler (using AIO and multiple outstanding requests per process etc.) we were able to achieve fairness whith the proportional-fair scheduler, whereas the elevator (standard scheduler) gave a lot more bandwidth to the process who started first. However there are a lot of conditions to make this happen: essentialy, anything that prevents queues to build up at the point where the scheduler is invoked will render the scheduler useless. Among these: - requests are made one at a time (e.g. without using AIO, or slower than the disk can serve them); - the ATA driver in 6.x/7.x does not queue bio's but moves them immediately to the layer below, where there is actually queueing; - locking at a higher level (e.g. the FS code) might enforce an ordering on requests before these are sent to the disk unit and so on. Also, there is a class of schedulers (non-work-conserving) that are not supported by this architecture, for the reasons explained in the wiki, and this is a shame as it is a very promising one judging by the literature. Essentially, a process doing e.g. a 'cvs checkout' on a large tree will still kill the performance of your disk no matter which scheduler you are using (elevator, hybrid, the dumb one that we wrote as an example). The reasons, i suspect, are a mixture of the ones described above. cheers luigi From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 19:26:45 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5D80C16A416 for ; Fri, 5 Jan 2007 19:26:45 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outR.internet-mail-service.net (outR.internet-mail-service.net [216.240.47.241]) by mx1.freebsd.org (Postfix) with ESMTP id 3B6B513C44C for ; Fri, 5 Jan 2007 19:26:45 +0000 (UTC) (envelope-from julian@elischer.org) Received: from shell.idiom.com (HELO idiom.com) (216.240.47.20) by out.internet-mail-service.net (qpsmtpd/0.32) with ESMTP; Fri, 05 Jan 2007 10:56:44 -0800 Received: from [10.251.18.229] (nat.ironport.com [63.251.108.100]) by idiom.com (8.12.11/8.12.11) with ESMTP id l05JEjdr048470; Fri, 5 Jan 2007 11:14:47 -0800 (PST) (envelope-from julian@elischer.org) Message-ID: <459EA3A1.6010407@elischer.org> Date: Fri, 05 Jan 2007 11:14:41 -0800 From: Julian Elischer User-Agent: Thunderbird 1.5.0.9 (Macintosh/20061207) MIME-Version: 1.0 To: Ivan Voras References: <20070105123127.gnk0v58p44488g48@webmail.ntnu.no> <4085.1167997049@critter.freebsd.dk> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 19:26:45 -0000 Ivan Voras wrote: > > Have both of you seen this: > http://wikitest.freebsd.org/Hybrid It has no performance numbers at all. Do you know of any? From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 20:14:21 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B52D216A407 for ; Fri, 5 Jan 2007 20:14:21 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 3A61913C441 for ; Fri, 5 Jan 2007 20:14:20 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1H2vSB-0008DI-6v for freebsd-geom@freebsd.org; Fri, 05 Jan 2007 21:14:15 +0100 Received: from 83-131-166-211.adsl.net.t-com.hr ([83.131.166.211]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 05 Jan 2007 21:14:15 +0100 Received: from ivoras by 83-131-166-211.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 05 Jan 2007 21:14:15 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Fri, 05 Jan 2007 21:13:56 +0100 Lines: 44 Message-ID: References: <20070105123127.gnk0v58p44488g48@webmail.ntnu.no> <4085.1167997049@critter.freebsd.dk> <459EA3A1.6010407@elischer.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig0DABB3A4A26A3B01F188FCA6" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 83-131-166-211.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: <459EA3A1.6010407@elischer.org> X-Enigmail-Version: 0.94.1.2 Sender: news Cc: freebsd-current@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 20:14:21 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig0DABB3A4A26A3B01F188FCA6 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Julian Elischer wrote: > Ivan Voras wrote: >=20 >> Have both of you seen this: >> http://wikitest.freebsd.org/Hybrid >=20 > It has no performance numbers at all. > Do you know of any? Nope, nothing except a vague (and unreliable) memory of reading somewhere there was no improvement, so it wasn't pursued further. Luigi Rizzo posted an interesting comment. The lack of results puzzled me at the time since I read Linux supposedly got decent improvements at the beginning of their 2.6 branch. Here's some links (admittedly, Googled as I don't have my own from that time): - http://linux.inet.hr/files/ols2005/seelam-reprint.pdf - http://kerneltrap.org/node/580 - http://www.redhat.com/magazine/008jun05/features/schedulers/ --------------enig0DABB3A4A26A3B01F188FCA6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFnrGLldnAQVacBcgRAsbIAJ4zD3xxbvohsiiKW0ujPzAw6QxAYACg2KjK +5YCoi79/4ajeHBQzmJ792g= =F0Tl -----END PGP SIGNATURE----- --------------enig0DABB3A4A26A3B01F188FCA6-- From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 21:52:23 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E49A216A407; Fri, 5 Jan 2007 21:52:23 +0000 (UTC) (envelope-from jd@ugcs.caltech.edu) Received: from heave.ugcs.caltech.edu (heave.ugcs.caltech.edu [131.215.176.104]) by mx1.freebsd.org (Postfix) with ESMTP id CF30113C441; Fri, 5 Jan 2007 21:52:23 +0000 (UTC) (envelope-from jd@ugcs.caltech.edu) Received: by heave.ugcs.caltech.edu (Postfix, from userid 3640) id D3E208F482; Fri, 5 Jan 2007 13:25:43 -0800 (PST) Date: Fri, 5 Jan 2007 13:25:43 -0800 From: Paul Allen To: Luigi Rizzo Message-ID: <20070105212543.GA8574@heave.ugcs.caltech.edu> References: <20070105123127.gnk0v58p44488g48@webmail.ntnu.no> <4085.1167997049@critter.freebsd.dk> <20070105184621.dh8kgoy7ko4gk4gc@webmail.ntnu.no> <20070105102905.A91349@xorpc.icir.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070105102905.A91349@xorpc.icir.org> Sender: jd@ugcs.caltech.edu Cc: freebsd-current@freebsd.org, lulf@stud.ntnu.no, Ivan Voras , freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 21:52:24 -0000 >From Luigi Rizzo , Fri, Jan 05, 2007 at 10:29:05AM -0800: > Essentially, a process doing e.g. a 'cvs checkout' on a large > tree will still kill the performance of your disk no matter > which scheduler you are using (elevator, hybrid, the dumb one that > we wrote as an example). The reasons, i suspect, are a mixture > of the ones described above. Interaction of the disk scheduler and softupdates? As I understand the softupdate code, the disk scheduler would be ignorant of I/O that depends on the completion of certain writes because issuing that I/O to the lower layers is delayed until the equiv of bio_done occurs. I don't see what use a disk scheduler would be unless it is possible to push information about any partial ordering requirements down to the level at which the scheduler can see and enforce them. As you alude to, the scheduler cannot do anything useful unless it sees a comprehensive picture of the pending I/O. -Paul From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 22:59:18 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A635B16A403; Fri, 5 Jan 2007 22:59:18 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.freebsd.org (Postfix) with ESMTP id 92A2D13C441; Fri, 5 Jan 2007 22:59:18 +0000 (UTC) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.11/8.13.6) with ESMTP id l05MxG5h094398; Fri, 5 Jan 2007 14:59:16 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.11/8.12.3/Submit) id l05MxFeV094397; Fri, 5 Jan 2007 14:59:15 -0800 (PST) (envelope-from rizzo) Date: Fri, 5 Jan 2007 14:59:15 -0800 From: Luigi Rizzo To: Paul Allen Message-ID: <20070105145915.B94175@xorpc.icir.org> References: <20070105123127.gnk0v58p44488g48@webmail.ntnu.no> <4085.1167997049@critter.freebsd.dk> <20070105184621.dh8kgoy7ko4gk4gc@webmail.ntnu.no> <20070105102905.A91349@xorpc.icir.org> <20070105212543.GA8574@heave.ugcs.caltech.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20070105212543.GA8574@heave.ugcs.caltech.edu>; from nospam@nospam.com on Fri, Jan 05, 2007 at 01:25:43PM -0800 Cc: freebsd-current@freebsd.org, lulf@stud.ntnu.no, Ivan Voras , freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 22:59:18 -0000 On Fri, Jan 05, 2007 at 01:25:43PM -0800, Paul Allen wrote: > >From Luigi Rizzo , Fri, Jan 05, 2007 at 10:29:05AM -0800: > > Essentially, a process doing e.g. a 'cvs checkout' on a large > > tree will still kill the performance of your disk no matter > > which scheduler you are using (elevator, hybrid, the dumb one that > > we wrote as an example). The reasons, i suspect, are a mixture > > of the ones described above. > > Interaction of the disk scheduler and softupdates? As I understand > the softupdate code, the disk scheduler would be ignorant of I/O that this was also on 4.11 which probably does not use softupdates. > depends on the completion of certain writes because issuing that I/O > to the lower layers is delayed until the equiv of bio_done occurs. > > I don't see what use a disk scheduler would be unless it is possible > to push information about any partial ordering requirements down to > the level at which the scheduler can see and enforce them. that does not mean that a scheduler is useless in general. in fact, what i find questionable is hardwiring unnecessary (in the sense that they are done only for performance reasons) ordering constraints in the layers above. I cannot comment for the disk, but e.g. for the process scheduler there are priority updates in many places depending on what the process is doing. cheers luigi From owner-freebsd-geom@FreeBSD.ORG Fri Jan 5 23:11:21 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 126B116A403 for ; Fri, 5 Jan 2007 23:11:21 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outQ.internet-mail-service.net (outQ.internet-mail-service.net [216.240.47.240]) by mx1.freebsd.org (Postfix) with ESMTP id D99CF13C458 for ; Fri, 5 Jan 2007 23:11:20 +0000 (UTC) (envelope-from julian@elischer.org) Received: from shell.idiom.com (HELO idiom.com) (216.240.47.20) by out.internet-mail-service.net (qpsmtpd/0.32) with ESMTP; Fri, 05 Jan 2007 14:53:15 -0800 Received: from [10.251.18.229] (nat.ironport.com [63.251.108.100]) by idiom.com (8.12.11/8.12.11) with ESMTP id l05NBFVL078286; Fri, 5 Jan 2007 15:11:15 -0800 (PST) (envelope-from julian@elischer.org) Message-ID: <459EDB0B.1050804@elischer.org> Date: Fri, 05 Jan 2007 15:11:07 -0800 From: Julian Elischer User-Agent: Thunderbird 1.5.0.9 (Macintosh/20061207) MIME-Version: 1.0 To: Luigi Rizzo References: <20070105123127.gnk0v58p44488g48@webmail.ntnu.no> <4085.1167997049@critter.freebsd.dk> <20070105184621.dh8kgoy7ko4gk4gc@webmail.ntnu.no> <20070105102905.A91349@xorpc.icir.org> <20070105212543.GA8574@heave.ugcs.caltech.edu> <20070105145915.B94175@xorpc.icir.org> In-Reply-To: <20070105145915.B94175@xorpc.icir.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: lulf@stud.ntnu.no, freebsd-current@freebsd.org, Paul Allen , Ivan Voras , freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 23:11:21 -0000 Luigi Rizzo wrote: > On Fri, Jan 05, 2007 at 01:25:43PM -0800, Paul Allen wrote: orant of I/O that > > this was also on 4.11 which probably does not use softupdates. yes it does.. Doesn't have UFS2 and snapshots however. > > that does not mean that a scheduler is useless in general. > in fact, what i find questionable is hardwiring unnecessary > (in the sense that they are done only for performance reasons) > ordering constraints in the layers above. I cannot comment for > the disk, but e.g. for the process scheduler there are priority > updates in many places depending on what the process is doing. > > cheers > luigi > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-geom@FreeBSD.ORG Sat Jan 6 00:59:43 2007 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DBD3316A403; Sat, 6 Jan 2007 00:59:43 +0000 (UTC) (envelope-from jd@ugcs.caltech.edu) Received: from heave.ugcs.caltech.edu (heave.ugcs.caltech.edu [131.215.176.104]) by mx1.freebsd.org (Postfix) with ESMTP id C784B13C442; Sat, 6 Jan 2007 00:59:43 +0000 (UTC) (envelope-from jd@ugcs.caltech.edu) Received: by heave.ugcs.caltech.edu (Postfix, from userid 3640) id 5B9048F482; Fri, 5 Jan 2007 16:59:43 -0800 (PST) Date: Fri, 5 Jan 2007 16:59:43 -0800 From: Paul Allen To: Luigi Rizzo Message-ID: <20070106005943.GB8574@heave.ugcs.caltech.edu> References: <20070105123127.gnk0v58p44488g48@webmail.ntnu.no> <4085.1167997049@critter.freebsd.dk> <20070105184621.dh8kgoy7ko4gk4gc@webmail.ntnu.no> <20070105102905.A91349@xorpc.icir.org> <20070105212543.GA8574@heave.ugcs.caltech.edu> <20070105145915.B94175@xorpc.icir.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070105145915.B94175@xorpc.icir.org> Sender: jd@ugcs.caltech.edu Cc: freebsd-current@freebsd.org, lulf@stud.ntnu.no, Ivan Voras , freebsd-geom@freebsd.org Subject: Re: Pluggable Disk Schedulers in GEOM X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2007 00:59:44 -0000 >From Luigi Rizzo , Fri, Jan 05, 2007 at 02:59:15PM -0800: > that does not mean that a scheduler is useless in general. > in fact, what i find questionable is hardwiring unnecessary > (in the sense that they are done only for performance reasons) > ordering constraints in the layers above. I cannot comment for I agree in so much as my point was strictly focused on ordering constraints imposed for consistency reasons. softupdates is not a performance mechanism per se; it is a mechanism to allow async writes of fs data occur in a consistent fashion. Anyways, you can easily resolve this issue by mounting async with softupdates disabled and then repeating the test.