From owner-freebsd-geom@FreeBSD.ORG Mon Oct 31 11:07:03 2011 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C6311065672 for ; Mon, 31 Oct 2011 11:07:03 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6B5AD8FC1D for ; Mon, 31 Oct 2011 11:07:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p9VB73b5056748 for ; Mon, 31 Oct 2011 11:07:03 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p9VB72wR056746 for freebsd-geom@FreeBSD.org; Mon, 31 Oct 2011 11:07:02 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 31 Oct 2011 11:07:02 GMT Message-Id: <201110311107.p9VB72wR056746@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-geom@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-geom@FreeBSD.org X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Oct 2011 11:07:03 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/162147 geom [geom] mirror GPT: GPT rejected -- may not be recovera o kern/162010 geom [geli] panic: Provider's error should be set (error=0) o kern/161979 geom [geom] glabel doesn't update after newfs, and glabel s o kern/161752 geom [geom] glabel(8) doesn't get gpt label change o bin/161677 geom gpart(8) Probably bug in gptboot o kern/160562 geom [geom][patch] Allow to insert new component to geom_ra o kern/160409 geom [geli] failed to attach provider o kern/159595 geom [geom] [panic] panic on gmirror unload in vbox [regres o kern/159091 geom [geom] GEOM fails to scan nested partitions to create p kern/158398 geom [headers] [patch] includes o kern/158197 geom [geom] geom_cache with size>1000 leads to panics o kern/157879 geom [libgeom] [regression] ABI change without version bump o kern/157863 geom [geli] kbdmux prevents geli passwords from being enter o kern/157739 geom [geom] GPT labels with geom_multipath o kern/157724 geom [geom] gpart(8) 'add' command must preserve gap for sc o kern/157723 geom [geom] GEOM should not process 'c' (raw) partitions fo o kern/157108 geom [gjournal] dumpon(8) fails on gjournal providers o kern/155994 geom [geom] Long "Suspend time" when reading large files fr o kern/154226 geom [geom] GEOM label does not change when you modify them o kern/150858 geom [geom] [geom_label] [patch] glabel(8) is not compatibl o kern/150626 geom [geom] [gjournal] gjournal(8) destroys label o kern/150555 geom [geom] gjournal unusable on GPT partitions o kern/150334 geom [geom] [udf] [patch] geom label does not support UDF o kern/149762 geom volume labels with rogue characters o bin/149215 geom [panic] [geom_part] gpart(8): Delete linux's slice via o kern/147667 geom [gmirror] Booting with one component of a gmirror, the o kern/145818 geom [geom] geom_stat_open showing cached information for n o kern/145042 geom [geom] System stops booting after printing message "GE f kern/144905 geom [geom][geom_part] panic in gpart_ctlreq when unpluggin o kern/143455 geom gstripe(8) in RELENG_8 (31st Jan 2010) broken o kern/142563 geom [geom] [hang] ioctl freeze in zpool o kern/141740 geom [geom] gjournal(8): g_journal_destroy concurrent error o kern/140352 geom [geom] gjournal + glabel not working o kern/135898 geom [geom] Severe filesystem corruption - large files or l o kern/134922 geom [gmirror] [panic] kernel panic when use fdisk on disk o kern/134113 geom [geli] Problem setting secondary GELI key o kern/133931 geom [geli] [request] intentionally wrong password to destr o bin/132845 geom [geom] [patch] ggated(8) does not close files opened a o bin/131415 geom [geli] keystrokes are unregulary sent to Geli when typ o kern/131353 geom [geom] gjournal(8) kernel lock o kern/129674 geom [geom] gjournal root did not mount on boot o kern/129645 geom gjournal(8): GEOM_JOURNAL causes system to fail to boo o kern/129245 geom [geom] gcache is more suitable for suffix based provid f kern/128276 geom [gmirror] machine lock up when gmirror module is used o kern/127420 geom [geom] [gjournal] [panic] Journal overflow on gmirrore o kern/124973 geom [gjournal] [patch] boot order affects geom_journal con o kern/124969 geom gvinum(8): gvinum raid5 plex does not detect missing s o kern/123962 geom [panic] [gjournal] gjournal (455Gb data, 8Gb journal), o kern/123122 geom [geom] GEOM / gjournal kernel lock o kern/122738 geom [geom] gmirror list "losts consumers" after gmirror de o kern/122067 geom [geom] [panic] Geom crashed during boot o kern/121364 geom [gmirror] Removing all providers create a "zombie" mir o kern/120091 geom [geom] [geli] [gjournal] geli does not prompt for pass o kern/115856 geom [geli] ZFS thought it was degraded when it should have o kern/115547 geom [geom] [patch] [request] let GEOM Eli get password fro o kern/114532 geom [geom] GEOM_MIRROR shows up in kldstat even if compile f kern/113957 geom [gmirror] gmirror is intermittently reporting a degrad o kern/113837 geom [geom] unable to access 1024 sector size storage o kern/113419 geom [geom] geom fox multipathing not failing back o kern/107707 geom [geom] [patch] [request] add new class geom_xbox360 to o kern/94632 geom [geom] Kernel output resets input while GELI asks for o kern/90582 geom [geom] [panic] Restore cause panic string (ffs_blkfree o bin/90093 geom fdisk(8) incapable of altering in-core geometry o kern/87544 geom [gbde] mmaping large files on a gbde filesystem deadlo o bin/86388 geom [geom] [geom_part] periodic(8) daily should backup gpa o kern/84556 geom [geom] [panic] GBDE-encrypted swap causes panic at shu o kern/79251 geom [2TB] newfs fails on 2.6TB gbde device o kern/79035 geom [vinum] gvinum unable to create a striped set of mirro o bin/78131 geom gbde(8) "destroy" not working. 69 problems total. From owner-freebsd-geom@FreeBSD.ORG Mon Oct 31 20:10:19 2011 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 102D11065674; Mon, 31 Oct 2011 20:10:19 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 700598FC12; Mon, 31 Oct 2011 20:10:18 +0000 (UTC) Received: by faar19 with SMTP id r19so8417043faa.13 for ; Mon, 31 Oct 2011 13:10:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :content-type:content-transfer-encoding; bh=vxcFKLjoZvxph4J+RwapjBEIpTuVlaiOHm8Kq/NL7dg=; b=NaacfmOZjBzusl1ZCqq4xSodUs/7/4svyRLS00fUqVBq95G8eJkwprpa0cbc5T7BX9 jNuQq3wi8S7tR/UIsxCyNHn+KCgrZ22cyU2xuZzFQ0IEvnooFcv7RS7CPhkgvfdg+lg7 WbIfh1O4Q0q8QG7ZgD12OCF+tAtIZxVjFOCsM= Received: by 10.223.61.138 with SMTP id t10mr31505566fah.20.1320091817314; Mon, 31 Oct 2011 13:10:17 -0700 (PDT) Received: from mavbook2.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id o16sm39868369fag.21.2011.10.31.13.10.15 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 31 Oct 2011 13:10:16 -0700 (PDT) Sender: Alexander Motin Message-ID: <4EAF00A6.5060903@FreeBSD.org> Date: Mon, 31 Oct 2011 22:10:14 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111003 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-geom@freebsd.org Subject: RFC: GEOM MULTIPATH rewrite X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Oct 2011 20:10:19 -0000 Hi. Attempt to fix some GEOM MULTIPATH issues made me almost rewrite it. So I would like to present my results and request for testing and feedback. The main changes: - Improved locking and destruction process to fix crashes in many cases. - Improved "automatic" configuration method to make it safe by reading metadata back from all specified paths after writing to one. - Added provider size check to reduce chance of conflict with other GEOM classes. - Added "manual" configuration method without using on-disk metadata. - Added "add" and "remove" commands to manage paths manually. - Failed paths no longer dropped from GEOM, but only marked as FAIL and excluded from I/O operations. - Automatically restore failed paths when all others paths are marked as failed, for example, because of device-caused (not transport) errors. - Added "fail" and "restore" commands to manually control FAIL flag. - GEOM is now destroyed on last provider disconnection. IMHO it is right to do if device was completely removed. - Added optional Active/Active mode support. Unlike Active/Passive mode, load evenly distributed between all working paths. If supported by device, it allows to significantly improve performance, utilizing bandwidth of all paths. It is controlled by -A option during creation. Disabled by default now. - Improved `status` and `list` commands output. Latest patch can be found here: http://people.freebsd.org/~mav/gmultipath4.patch Feedbacks are welcome! Sponsored by: iXsystems, Inc. -- Alexander Motin From owner-freebsd-geom@FreeBSD.ORG Tue Nov 1 03:42:37 2011 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08D7A106566B; Tue, 1 Nov 2011 03:42:37 +0000 (UTC) (envelope-from stephane.lapie@darkbsd.org) Received: from quasar.darkbsd.org (shinigami.darkbsd.org [82.227.96.182]) by mx1.freebsd.org (Postfix) with ESMTP id 736818FC08; Tue, 1 Nov 2011 03:42:35 +0000 (UTC) Received: from quasar.darkbsd.org (localhost [127.0.0.1]) by quasar.darkbsd.org (Postfix) with ESMTP id 6EC317711; Tue, 1 Nov 2011 04:42:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=darkbsd.org; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; s=selector1; bh=CGuO3uobQVaXkpqLi46slu4fnVM=; b=M qxEJxZtnQPUui6fYWWX45t4uLm9KZYPGQcl6uD4oKd0D5uX6d4zXyRkr6NgXbNvE rpsG/wpdRbxC/+gAZQbY657YKdm+amlGWSGyCbKdJN8wJFhvSswXLQoGVvkejGUf kHuizETFSmYC5D7STu7//mDkwJ3XqEijCnaf4aAxOU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=darkbsd.org; h=message-id :date:from:mime-version:to:cc:subject:references:in-reply-to :content-type; q=dns; s=selector1; b=dnRdQpHaK6yFeljtu0T1U46UAkk xEFJrsf/Y59YfM1w/5RrDgwDlQZ1CyWBsM2QcXQUfyN/MyJVH1VkSiAvoxx2xGhR j0OQ5nwtzI5bV1AoaVWFCdqMaI9TNPpt/l4L+zCReu1WLu8dZKVA9TYHFXBtCmHQ 1u79ohcV3chBKKhs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=darkbsd.org; h= content-type:content-type:in-reply-to:references:subject:subject :mime-version:user-agent:from:from:date:date:message-id:received :received; s=selector1; t=1320118955; bh=79SNKGBmqTfdv0oWWVt9DyW E8W+UKc5XGxjuCrQJp4o=; b=WONndy5+IaRlV17uCjNuYWlZ4mYMfY1fkRs0YsJ FZVazHoxhpg1KznjnaKNcdULOcxlFHCTFBfPAHkscn4Q8yB9QM0tV68eSYFZ6MMv t7HyjHFuzfwQ+qE+B7UAZK2IM+fdJEbXNZNQyItvjRk2jnrKjhVDBpVeQNbbG6HL kM7k= Received: from quasar.darkbsd.org ([127.0.0.1]) by quasar.darkbsd.org (quasar.darkbsd.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ro83RX-+tSAR; Tue, 1 Nov 2011 04:42:35 +0100 (CET) Received: from [192.168.166.168] (unknown [210.188.173.246]) (Authenticated sender: darksoul) by quasar.darkbsd.org (Postfix) with ESMTPSA id 5AA97770A; Tue, 1 Nov 2011 04:42:34 +0100 (CET) Message-ID: <4EAF6A99.5020609@darkbsd.org> Date: Tue, 01 Nov 2011 12:42:17 +0900 From: Stephane LAPIE User-Agent: Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: Alexander Motin References: <4EAF00A6.5060903@FreeBSD.org> In-Reply-To: <4EAF00A6.5060903@FreeBSD.org> X-Enigmail-Version: 1.4a1pre Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigB3C5D17B9A52540D296E9FEC" Cc: freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: RFC: GEOM MULTIPATH rewrite X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Nov 2011 03:42:37 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB3C5D17B9A52540D296E9FEC Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hello, First of all, many thanks. I am going to test your patch on 9.0-RC1, and try to backport it to 8.2 (which is the main version I am currently using at work, in the environment where I have a critical need for FC multipath redundancy...) Again, thanks for your efforts. I hope to be giving feedback soon. Cheers, On 11/01/2011 05:10 AM, Alexander Motin wrote: > Hi. >=20 > Attempt to fix some GEOM MULTIPATH issues made me almost rewrite it. So= > I would like to present my results and request for testing and feedback= =2E >=20 > The main changes: > - Improved locking and destruction process to fix crashes in many case= s. > - Improved "automatic" configuration method to make it safe by reading= > metadata back from all specified paths after writing to one. > - Added provider size check to reduce chance of conflict with other > GEOM classes. > - Added "manual" configuration method without using on-disk metadata. > - Added "add" and "remove" commands to manage paths manually. > - Failed paths no longer dropped from GEOM, but only marked as FAIL an= d > excluded from I/O operations. > - Automatically restore failed paths when all others paths are marked > as failed, for example, because of device-caused (not transport) errors= =2E > - Added "fail" and "restore" commands to manually control FAIL flag. > - GEOM is now destroyed on last provider disconnection. IMHO it is > right to do if device was completely removed. > - Added optional Active/Active mode support. Unlike Active/Passive > mode, load evenly distributed between all working paths. If supported b= y > device, it allows to significantly improve performance, utilizing > bandwidth of all paths. It is controlled by -A option during creation. > Disabled by default now. > - Improved `status` and `list` commands output. >=20 > Latest patch can be found here: > http://people.freebsd.org/~mav/gmultipath4.patch >=20 > Feedbacks are welcome! >=20 > Sponsored by: iXsystems, Inc. >=20 --=20 Stephane LAPIE, EPITA SRS, Promo 2005 "Even when they have digital readouts, I can't understand them." --MegaTokyo --------------enigB3C5D17B9A52540D296E9FEC Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6vaqMACgkQ24Ql8u6TF2OwKwCgod1huQlPNHd8P2VV0jvgXY8O jUAAn18c9LnMA8IRP3VBcNOaPOXVhXQX =UdJZ -----END PGP SIGNATURE----- --------------enigB3C5D17B9A52540D296E9FEC-- From owner-freebsd-geom@FreeBSD.ORG Tue Nov 1 07:43:00 2011 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C2A57106564A; Tue, 1 Nov 2011 07:43:00 +0000 (UTC) (envelope-from ae@FreeBSD.org) Received: from mail.kirov.so-ups.ru (ns.kirov.so-ups.ru [178.74.170.1]) by mx1.freebsd.org (Postfix) with ESMTP id 6A2298FC17; Tue, 1 Nov 2011 07:43:00 +0000 (UTC) Received: from kas30pipe.localhost (localhost.kirov.so-ups.ru [127.0.0.1]) by mail.kirov.so-ups.ru (Postfix) with SMTP id 14C49B8027; Tue, 1 Nov 2011 11:23:20 +0400 (SAMT) Received: from kirov.so-ups.ru (unknown [172.21.81.1]) by mail.kirov.so-ups.ru (Postfix) with ESMTP id 0A84CB801F; Tue, 1 Nov 2011 11:23:20 +0400 (SAMT) Received: by ns.kirov.so-ups.ru (Postfix, from userid 1010) id 00337B8433; Tue, 1 Nov 2011 11:23:19 +0400 (SAMT) Received: from [127.0.0.1] (elsukov.kirov.oduur.so [10.118.3.52]) by ns.kirov.so-ups.ru (Postfix) with ESMTP id B84DCB8422; Tue, 1 Nov 2011 11:23:19 +0400 (SAMT) Message-ID: <4EAF9E67.4040605@FreeBSD.org> Date: Tue, 01 Nov 2011 11:23:19 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla Thunderbird 1.5 (FreeBSD/20051231) MIME-Version: 1.0 To: sascha@trimind.de References: <8453E2A2-3219-4FAA-98CE-2F9D66EA1C39@FreeBSD.org> <20111028094828.GA1781@trimind.de> In-Reply-To: <20111028094828.GA1781@trimind.de> X-Enigmail-Version: 1.3.2 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit X-SpamTest-Version: SMTP-Filter Version 3.0.0 [0284], KAS30/Release X-SpamTest-Info: Not protected Cc: Garrett Cooper , antik@bsd.ee, Martin Wilke , current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: gmirror failed with error 19. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Nov 2011 07:43:00 -0000 On 28.10.2011 13:48, Sascha Klauder wrote: > I've got bitten by this as well when trying a source up- > grade of a freshly installed 8.2-RELEASE system that had > gmirror configured after installation according to the > procedure in the Handbook. > > The 9.0-RC1 kernel drops to the mountroot prompt when > booting with > > GEOM_MIRROR: Device mirror/gm0 launched (2/2). > GEOM_PART: partition 1 has end offset beyond last LBA: 490350671 > 490350670 > GEOM_PART: integrity check failed (mirror/gm0, MBR) This is the main problem. Your MBR' slice is bigger than actual space you have. The only way to fix this - recreate slice. You can break your mirror, recreate the slice (NOTE: you must preserve one sector for the gmirror's meta-data), then copy your data to the newly created slice, then reboot from the new slice and recreate mirror. -- WBR, Andrey V. Elsukov From owner-freebsd-geom@FreeBSD.ORG Tue Nov 1 12:40:39 2011 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F820106564A; Tue, 1 Nov 2011 12:40:39 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id E44A98FC0A; Tue, 1 Nov 2011 12:40:38 +0000 (UTC) Received: from localhost (host-89-230-170-58.ostrowmaz.mm.pl [89.230.170.58]) by mail.dawidek.net (Postfix) with ESMTPSA id 169C7A58; Tue, 1 Nov 2011 13:40:37 +0100 (CET) Date: Tue, 1 Nov 2011 13:39:48 +0100 From: Pawel Jakub Dawidek To: Alexander Motin Message-ID: <20111101123944.GC4567@garage.freebsd.pl> References: <4EAF00A6.5060903@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0vzXIDBeUiKkjNJl" Content-Disposition: inline In-Reply-To: <4EAF00A6.5060903@FreeBSD.org> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: RFC: GEOM MULTIPATH rewrite X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Nov 2011 12:40:39 -0000 --0vzXIDBeUiKkjNJl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 31, 2011 at 10:10:14PM +0200, Alexander Motin wrote: > Hi. >=20 > Attempt to fix some GEOM MULTIPATH issues made me almost rewrite it. So > I would like to present my results and request for testing and feedback. >=20 > The main changes: > - Improved locking and destruction process to fix crashes in many cases. > - Improved "automatic" configuration method to make it safe by reading > metadata back from all specified paths after writing to one. > - Added provider size check to reduce chance of conflict with other > GEOM classes. > - Added "manual" configuration method without using on-disk metadata. > - Added "add" and "remove" commands to manage paths manually. > - Failed paths no longer dropped from GEOM, but only marked as FAIL and > excluded from I/O operations. > - Automatically restore failed paths when all others paths are marked > as failed, for example, because of device-caused (not transport) errors. > - Added "fail" and "restore" commands to manually control FAIL flag. > - GEOM is now destroyed on last provider disconnection. IMHO it is > right to do if device was completely removed. > - Added optional Active/Active mode support. Unlike Active/Passive > mode, load evenly distributed between all working paths. If supported by > device, it allows to significantly improve performance, utilizing > bandwidth of all paths. It is controlled by -A option during creation. > Disabled by default now. > - Improved `status` and `list` commands output. >=20 > Latest patch can be found here: > http://people.freebsd.org/~mav/gmultipath4.patch >=20 > Feedbacks are welcome! >=20 > Sponsored by: iXsystems, Inc. There are two possible issues that comes to my mind, not sure if you address them. 1. When configuration is based on on-disk metadata, GEOM spoil/taste is not fully helpful - if you have two paths: da0 and da1 and I write to da0, gmultipath won't be informed by GEOM that da1 changed as well. One solution is to basically keep all paths open exclusively all the time, even if gmultipath provider is not open or emulate spoil/taste for other paths if any path was modified. 2. In active/active mode do you do anything to handle possible reordering? Ie. if you have overlapping writes and send both of them using different paths, you cannot be sure that order will be preserved. Most of the time that's not a problem, as file systems rarely if at all send overlapping writes to device, but this is weak assumption. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --0vzXIDBeUiKkjNJl Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk6v6JAACgkQForvXbEpPzQwSwCg7HoRkKI+8LdccEgUbpMFmcfM eQYAn3RWr/nGiRparSXh2LCHLgtu/fv7 =B1cU -----END PGP SIGNATURE----- --0vzXIDBeUiKkjNJl-- From owner-freebsd-geom@FreeBSD.ORG Tue Nov 1 13:05:43 2011 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0EE3C1065672; Tue, 1 Nov 2011 13:05:43 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2DE2F8FC12; Tue, 1 Nov 2011 13:05:42 +0000 (UTC) Received: by eyd10 with SMTP id 10so8038033eyd.13 for ; Tue, 01 Nov 2011 06:05:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=sRSKsN1WmuXYXd7bywp5jE1hRJRacNhtvinHkTUFlCw=; b=Thpbx2UQwdSs3Eo2LJnX7vuMiVmJR+9+Viev42J9iUi+zVrsoixH16cjhkmn5N6Gzv 8sM6cECvQKZXk3D10G7w+HoVq41NEiNTJo80E2vVRxVbv2Be8KWy1+XzQxP060uLIT0O ar4pe1NUCPuCTRvEmB8Pgpk9eqxXcU//9WmxQ= Received: by 10.14.8.136 with SMTP id 8mr4284eer.87.1320152741192; Tue, 01 Nov 2011 06:05:41 -0700 (PDT) Received: from mavbook2.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id t6sm29926393eeb.11.2011.11.01.06.05.39 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 01 Nov 2011 06:05:40 -0700 (PDT) Sender: Alexander Motin Message-ID: <4EAFEEA1.80500@FreeBSD.org> Date: Tue, 01 Nov 2011 15:05:37 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111003 Thunderbird/7.0.1 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <4EAF00A6.5060903@FreeBSD.org> <20111101123944.GC4567@garage.freebsd.pl> In-Reply-To: <20111101123944.GC4567@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-current@freebsd.org, freebsd-geom@freebsd.org Subject: Re: RFC: GEOM MULTIPATH rewrite X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Nov 2011 13:05:43 -0000 On 11/01/11 14:39, Pawel Jakub Dawidek wrote: > On Mon, Oct 31, 2011 at 10:10:14PM +0200, Alexander Motin wrote: >> Attempt to fix some GEOM MULTIPATH issues made me almost rewrite it. So >> I would like to present my results and request for testing and feedback. >> >> The main changes: >> - Improved locking and destruction process to fix crashes in many cases. >> - Improved "automatic" configuration method to make it safe by reading >> metadata back from all specified paths after writing to one. >> - Added provider size check to reduce chance of conflict with other >> GEOM classes. >> - Added "manual" configuration method without using on-disk metadata. >> - Added "add" and "remove" commands to manage paths manually. >> - Failed paths no longer dropped from GEOM, but only marked as FAIL and >> excluded from I/O operations. >> - Automatically restore failed paths when all others paths are marked >> as failed, for example, because of device-caused (not transport) errors. >> - Added "fail" and "restore" commands to manually control FAIL flag. >> - GEOM is now destroyed on last provider disconnection. IMHO it is >> right to do if device was completely removed. >> - Added optional Active/Active mode support. Unlike Active/Passive >> mode, load evenly distributed between all working paths. If supported by >> device, it allows to significantly improve performance, utilizing >> bandwidth of all paths. It is controlled by -A option during creation. >> Disabled by default now. >> - Improved `status` and `list` commands output. >> >> Latest patch can be found here: >> http://people.freebsd.org/~mav/gmultipath4.patch >> >> Feedbacks are welcome! >> >> Sponsored by: iXsystems, Inc. > > There are two possible issues that comes to my mind, not sure if you > address them. > > 1. When configuration is based on on-disk metadata, GEOM spoil/taste is > not fully helpful - if you have two paths: da0 and da1 and I write > to da0, gmultipath won't be informed by GEOM that da1 changed as well. > One solution is to basically keep all paths open exclusively all the > time, even if gmultipath provider is not open or emulate spoil/taste > for other paths if any path was modified. Now I am opening all underlying providers exclusively on attach to spoil other consumers. It is configurable via sysctl. > 2. In active/active mode do you do anything to handle possible > reordering? Ie. if you have overlapping writes and send both of them > using different paths, you cannot be sure that order will be > preserved. Most of the time that's not a problem, as file systems > rarely if at all send overlapping writes to device, but this is weak > assumption. No, I don't. I have doubt that it is sane to send even dependent I/O simultaneously without waiting for completion, not speaking about overlapping. When most of present devices support command queuing and so officially justify reordering simultaneous commands in custom way, I am not sure why above layers should be more strict, especially in cases when it is problematic. If somebody have ideas why and how to implement it, I am ready to discuss. -- Alexander Motin From owner-freebsd-geom@FreeBSD.ORG Tue Nov 1 17:15:00 2011 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6629B1065676; Tue, 1 Nov 2011 17:15:00 +0000 (UTC) (envelope-from jmg@h2.funkthat.com) Received: from h2.funkthat.com (gate.funkthat.com [70.36.235.232]) by mx1.freebsd.org (Postfix) with ESMTP id E0FC68FC18; Tue, 1 Nov 2011 17:14:59 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id pA1HExLG004262 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 1 Nov 2011 10:14:59 -0700 (PDT) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id pA1HExj3004261; Tue, 1 Nov 2011 10:14:59 -0700 (PDT) (envelope-from jmg) Date: Tue, 1 Nov 2011 10:14:59 -0700 From: John-Mark Gurney To: Alexander Motin Message-ID: <20111101171459.GY25601@funkthat.com> Mail-Followup-To: Alexander Motin , Pawel Jakub Dawidek , freebsd-current@freebsd.org, freebsd-geom@freebsd.org References: <4EAF00A6.5060903@FreeBSD.org> <20111101123944.GC4567@garage.freebsd.pl> <4EAFEEA1.80500@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EAFEEA1.80500@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Tue, 01 Nov 2011 10:14:59 -0700 (PDT) Cc: freebsd-current@freebsd.org, Pawel Jakub Dawidek , freebsd-geom@freebsd.org Subject: Re: RFC: GEOM MULTIPATH rewrite X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Nov 2011 17:15:00 -0000 Alexander Motin wrote this message on Tue, Nov 01, 2011 at 15:05 +0200: > > 2. In active/active mode do you do anything to handle possible > > reordering? Ie. if you have overlapping writes and send both of them > > using different paths, you cannot be sure that order will be > > preserved. Most of the time that's not a problem, as file systems > > rarely if at all send overlapping writes to device, but this is weak > > assumption. > > No, I don't. I have doubt that it is sane to send even dependent I/O > simultaneously without waiting for completion, not speaking about > overlapping. When most of present devices support command queuing and so > officially justify reordering simultaneous commands in custom way, I am > not sure why above layers should be more strict, especially in cases > when it is problematic. If somebody have ideas why and how to implement > it, I am ready to discuss. I know that phk and others have an idea what the contract for writes like this, but I just checked geom(4) and the IO section doesn't describe it.. I believe that you should not submit overlapping writes unless it's preceded or is an _ORDERED write, and that reads can be satisifed w/ stale data if it is submitted after a write of the same location, but it's not in geom(4). Hmm... turns out BIO_ORDERED isn't even documented in geom(4)... I'm willing to put some of this in the man page if someone comes up w/ a good list of points. Are the ones I listed above enough? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-geom@FreeBSD.ORG Tue Nov 1 19:24:10 2011 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 06FA41065672; Tue, 1 Nov 2011 19:24:10 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 653A58FC12; Tue, 1 Nov 2011 19:24:09 +0000 (UTC) Received: by faar19 with SMTP id r19so9829386faa.13 for ; Tue, 01 Nov 2011 12:24:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; bh=PTv7EGUW8kqZr06FiGa9RubFwC17uF1URowYwXehEwg=; b=RnJ+947bZ3/901dZvzVozHn7+UytrK6YVy6oh/IVtkGUvxclx2NVHhkjKU7YgcZ1aV af+GKFgqLi6R5QAyauTHw6QNTk7XPNZmxMacQfQAF8V6r+/GpIPIZe1ZZgYtQPsUGBvf kK9gsxzYXF1rXmiZze22fpV5WoO/hu6xMXelA= Received: by 10.223.76.27 with SMTP id a27mr2650763fak.12.1320175448305; Tue, 01 Nov 2011 12:24:08 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id o16sm395395fag.21.2011.11.01.12.24.06 (version=SSLv3 cipher=OTHER); Tue, 01 Nov 2011 12:24:07 -0700 (PDT) Sender: Alexander Motin Message-ID: <4EB05566.3060700@FreeBSD.org> Date: Tue, 01 Nov 2011 22:24:06 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:6.0.2) Gecko/20110910 Thunderbird/6.0.2 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Dennis_K=F6gel?= References: <4EAF00A6.5060903@FreeBSD.org> <05E0E64F-5EC4-425A-81E4-B6C35320608B@neveragain.de> In-Reply-To: <05E0E64F-5EC4-425A-81E4-B6C35320608B@neveragain.de> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: FreeBSD-Current , freebsd-geom@freebsd.org Subject: Re: RFC: GEOM MULTIPATH rewrite X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Nov 2011 19:24:10 -0000 On 01.11.2011 19:50, Dennis Kögel wrote: > Not sure if replying on-list or off-list makes more sense... Replying on-list could share experience to other users. > Anyway, some first impressions, on stable/9: > > The lab environment here is a EMC VNX / Clariion SAN, which has two Storage Processors, connected to different switches, connected to two isp(4)s on the test machine. So at any time, the machine sees four paths, but only two are available (depending on which SP owns the LUN). > > 580# camcontrol devlist > at scbus0 target 0 lun 0 (da0,pass0) > at scbus0 target 1 lun 0 (da1,pass1) > at scbus1 target 0 lun 0 (da2,pass2) > at scbus1 target 1 lun 0 (da3,pass3) > at scbus2 target 0 lun 0 (da4,pass4) > at scbus2 target 1 lun 0 (da5,pass5) > at scbus4 target 0 lun 0 (cd0,pass6) > > I miss the ability to "add" disks to automatic mode multipaths, but I (just now) realized this only makes sense when gmultipath has some kind of path checking facility (like periodically trying to read sector 0 of each configured device, this is was Linux' devicemapper-multipathd does). In automatic mode other paths supposed to be detected via metadata reading. If in your case some paths are not readable, automatic mode can't work as expected. By the way, could you describe how your configuration supposed to work, like when other paths will start working? Only when second storage processor itself detect that first one is dead or it suppose to be controlled somehow? If booted with verbose messages, what SCSI errors do you see on console/logs when trying to access second storage processor? Speaking about user-level daemons, it should be possible to do the same: write rc.d script to create multipath device in manual mode on boot and hook small script into devd.conf that will check models, serial numbers, or whatever else of newly detected devices and manually add them. At least if you are not booting from that device. Another possible way I see is to make geom_multipath to analyze device models and serial numbers on kernel level and attach devices based on it without using metadata. > First I did a test with active/active and manual mode (gmultipath create -A TESTBSD da{0,1,2,3}). > > This worked fine, and immediately kicked da{0,2} (which are not available) after writing something. Predictable. Errors caused not working paths to be marked as failed. If all working paths fail, these will be retried. > It's quite unexpected that act/act is apparently based on the writer's process id or CPU affinity (guessing) -- I needed multiple parallel dd(1) jobs to get gmultipath to use more than one path, but then it worked fine. (This also means that a dysfunct path isn't recognized before some I/O is attempted). Performance was similar to an identical Linux setup (for a very simple sequential I/O test at least). It is not an affinity, but just a dumb probe order. Now geom_multipath only balances immediate load, not average. So if you have only one request at a time, it will always use one path. If you have only one initiator, it should not be important, but if there are several, it should probably be improved. > Using automatic mode (gmultipath label -A TESTBSD da{1,3,0,2}) silently ignores da{0,2}, which are not available. "list" does not show them at all. I guess it should at least throw an error. Now "label" command only hints kernel to retaste other paths. If kernel is unable to read metadata from specified devices or there is no metadata, they just silently won't be added. I can add metadata checking in user-level. It will not guarantee success, but should at least handle basic mistakes. > Do any of the layers below gmultipath currently have information about metadata like the volume's WWN? This could be helpful in status / list output, maybe for other things, I guess... That information may be present inside CAM, but not above at the moment. Only device model and serial number exported to GEOM. > Another thought: As I guess "automatic" mode is meant to be the common case, the choice of "create" vs. "label" might be misleading. Users in a hurry will probably just look through the built-in help, see "create", and then use it. (I'm guilty as charged here, I didn't realize at first that I was using manual mode.) It is a somewhat unified among several GEOM classes that label is for automatic method and create is for manual. Thank you for your feedback! -- Alexander Motin From owner-freebsd-geom@FreeBSD.ORG Tue Nov 1 22:24:14 2011 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94AE7106566C for ; Tue, 1 Nov 2011 22:24:14 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 248718FC13 for ; Tue, 1 Nov 2011 22:24:14 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RLMke-0004rn-70 for freebsd-geom@freebsd.org; Tue, 01 Nov 2011 23:24:12 +0100 Received: from cpe-188-129-80-155.dynamic.amis.hr ([188.129.80.155]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 01 Nov 2011 23:24:12 +0100 Received: from ivoras by cpe-188-129-80-155.dynamic.amis.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 01 Nov 2011 23:24:12 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Tue, 01 Nov 2011 23:23:41 +0100 Lines: 51 Message-ID: References: <4EAF00A6.5060903@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cpe-188-129-80-155.dynamic.amis.hr User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 In-Reply-To: <4EAF00A6.5060903@FreeBSD.org> Cc: freebsd-current@freebsd.org Subject: Re: RFC: GEOM MULTIPATH rewrite X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Nov 2011 22:24:14 -0000 On 31.10.2011. 21:10, Alexander Motin wrote: > Hi. > > Attempt to fix some GEOM MULTIPATH issues made me almost rewrite it. So > I would like to present my results and request for testing and feedback. Good ideas, gmultipath robustness is a very good thing. Active/active support is also useful if ordering issues can be solved. There's a Linux paper linked at http://en.wikipedia.org/wiki/Multipath_I/O ... but apparently it doesn't say much about this problem. Could a simple policy such as "block all IO iff an BIO_ORDERED request is being processed" work well enough? I'd only add that you should be careful to be compatible with old gmultipath metadata (see e.g. recent user reports with 9.0 auto-updating gmirror metadata). > The main changes: > - Improved locking and destruction process to fix crashes in many cases. > - Improved "automatic" configuration method to make it safe by reading > metadata back from all specified paths after writing to one. > - Added provider size check to reduce chance of conflict with other > GEOM classes. > - Added "manual" configuration method without using on-disk metadata. > - Added "add" and "remove" commands to manage paths manually. > - Failed paths no longer dropped from GEOM, but only marked as FAIL and > excluded from I/O operations. > - Automatically restore failed paths when all others paths are marked > as failed, for example, because of device-caused (not transport) errors. > - Added "fail" and "restore" commands to manually control FAIL flag. > - GEOM is now destroyed on last provider disconnection. IMHO it is > right to do if device was completely removed. > - Added optional Active/Active mode support. Unlike Active/Passive > mode, load evenly distributed between all working paths. If supported by > device, it allows to significantly improve performance, utilizing > bandwidth of all paths. It is controlled by -A option during creation. > Disabled by default now. > - Improved `status` and `list` commands output. > > Latest patch can be found here: > http://people.freebsd.org/~mav/gmultipath4.patch > > Feedbacks are welcome! > > Sponsored by: iXsystems, Inc. > From owner-freebsd-geom@FreeBSD.ORG Wed Nov 2 19:37:33 2011 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE22F106566C; Wed, 2 Nov 2011 19:37:33 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4A6078FC1B; Wed, 2 Nov 2011 19:37:32 +0000 (UTC) Received: by faar19 with SMTP id r19so1159844faa.13 for ; Wed, 02 Nov 2011 12:37:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=zI4YI/bG7QndiOFEvqrQgZeeknyW3LUaay8fbSSJ2A8=; b=jtcnQ8kGvTfonYmdRFbCfdErMYKL8gasU5zaoLkSvqKzP+NsTAyxKc9yQzTzCFNzB4 Xa3HbBlzI1EqRg3oGu6hrTq7kRhAVwz432oNyncOwHHWCRWPJy9nTmVcO9yVh/DHIaGZ Xg9qljrXGpNF1Dfcea+Gdh+i5MmzwstSh0lg8= Received: by 10.223.61.72 with SMTP id s8mr3994602fah.27.1320262574879; Wed, 02 Nov 2011 12:36:14 -0700 (PDT) Received: from mavbook2.mavhome.dp.ua (pc.mavhome.dp.ua. [212.86.226.226]) by mx.google.com with ESMTPS id i3sm7320419faf.0.2011.11.02.12.36.13 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 02 Nov 2011 12:36:13 -0700 (PDT) Sender: Alexander Motin Message-ID: <4EB19BAB.90801@FreeBSD.org> Date: Wed, 02 Nov 2011 21:36:11 +0200 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:7.0.1) Gecko/20111003 Thunderbird/7.0.1 MIME-Version: 1.0 To: freebsd-current@freebsd.org References: <4EAF00A6.5060903@FreeBSD.org> In-Reply-To: <4EAF00A6.5060903@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Dennis Koegel , freebsd-geom@freebsd.org Subject: Re: RFC: GEOM MULTIPATH rewrite X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Nov 2011 19:37:34 -0000 On 10/31/11 22:10, Alexander Motin wrote: > Attempt to fix some GEOM MULTIPATH issues made me almost rewrite it. So > I would like to present my results and request for testing and feedback. > > The main changes: > - Improved locking and destruction process to fix crashes in many cases. > - Improved "automatic" configuration method to make it safe by reading > metadata back from all specified paths after writing to one. > - Added provider size check to reduce chance of conflict with other > GEOM classes. > - Added "manual" configuration method without using on-disk metadata. > - Added "add" and "remove" commands to manage paths manually. > - Failed paths no longer dropped from GEOM, but only marked as FAIL and > excluded from I/O operations. > - Automatically restore failed paths when all others paths are marked > as failed, for example, because of device-caused (not transport) errors. > - Added "fail" and "restore" commands to manually control FAIL flag. > - GEOM is now destroyed on last provider disconnection. IMHO it is > right to do if device was completely removed. > - Added optional Active/Active mode support. Unlike Active/Passive > mode, load evenly distributed between all working paths. If supported by > device, it allows to significantly improve performance, utilizing > bandwidth of all paths. It is controlled by -A option during creation. > Disabled by default now. > - Improved `status` and `list` commands output. Here is slightly updated version: http://people.freebsd.org/~mav/gmultipath5.patch Changes from previous: - "label" command was made to check metadata on other specified devices after writing to the first one and warn if something wrong there. - "add" command was allowed for devices created using automatic method. It may be useful in cases when some paths are temporary not operational and can't even read metadata. In such case they could be added manually and marked failed until their time will come. - Load balancing in Active/Active mode slightly improved: if all paths have equal load, do round-robin between them. -- Alexander Motin From owner-freebsd-geom@FreeBSD.ORG Thu Nov 3 11:51:00 2011 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8DF7106564A; Thu, 3 Nov 2011 11:51:00 +0000 (UTC) (envelope-from sascha@trimind.de) Received: from mikako.shopkeeper.de (mikako.shopkeeper.de [82.119.175.20]) by mx1.freebsd.org (Postfix) with ESMTP id 6F2E58FC15; Thu, 3 Nov 2011 11:50:59 +0000 (UTC) Received: from avalon.dobu.local (p508D4743.dip.t-dialin.net [80.141.71.67]) (authenticated bits=0) by mikako.shopkeeper.de (8.14.3/8.14.3) with ESMTP id pA3BosIW031316 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 3 Nov 2011 12:50:57 +0100 (CET) (envelope-from sascha@trimind.de) Received: from avalon.dobu.local (localhost [127.0.0.1]) by avalon.dobu.local (8.14.4/8.14.2) with ESMTP id pA3BosWg002210; Thu, 3 Nov 2011 12:50:54 +0100 (CET) (envelope-from sascha@avalon.dobu.local) Received: (from sascha@localhost) by avalon.dobu.local (8.14.4/8.14.4/Submit) id pA3Boso9002209; Thu, 3 Nov 2011 12:50:54 +0100 (CET) (envelope-from sascha) Date: Thu, 3 Nov 2011 12:50:54 +0100 From: Sascha Klauder To: "Andrey V. Elsukov" Message-ID: <20111103115054.GA2155@trimind.de> References: <8453E2A2-3219-4FAA-98CE-2F9D66EA1C39@FreeBSD.org> <20111028094828.GA1781@trimind.de> <4EAF9E67.4040605@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EAF9E67.4040605@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.97.2 at mikako.shopkeeper.de X-Virus-Status: Clean Cc: Garrett Cooper , antik@bsd.ee, Martin Wilke , current@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: gmirror failed with error 19. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Nov 2011 11:51:01 -0000 On Tue, 2011-11-01 11:23 +0400, Andrey V. Elsukov wrote: > On 28.10.2011 13:48, Sascha Klauder wrote: > > GEOM_MIRROR: Device mirror/gm0 launched (2/2). > > GEOM_PART: partition 1 has end offset beyond last LBA: 490350671 > 490350670 > > GEOM_PART: integrity check failed (mirror/gm0, MBR) > This is the main problem. Your MBR' slice is bigger than actual space > you have. The only way to fix this - recreate slice. I've partioned and labeled the disk with sysinstall(8) from 8.2-RELEASE media. Does 9.0 use different disk geometry cal- culation than 8.2 or is usage of gmirror the culprit? Both 8.2 and 9.0 kernels report the disk having 490350672 sectors. > You can break your mirror, recreate the slice (NOTE: you must preserve > one sector for the gmirror's meta-data), then copy your data to the > newly created slice, then reboot from the new slice and recreate mirror. I think I'd rather reinstall 9.0 from scratch, getting rid of MBR/disklabel as well. Cheers, -sascha From owner-freebsd-geom@FreeBSD.ORG Thu Nov 3 22:48:40 2011 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C9B8E106564A; Thu, 3 Nov 2011 22:48:40 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 770ED8FC0A; Thu, 3 Nov 2011 22:48:40 +0000 (UTC) Received: by iabz21 with SMTP id z21so3010686iab.13 for ; Thu, 03 Nov 2011 15:48:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=RWHdmeFDZbWUIJPgKJNseBYC4j9tYB+PcjCpp3DaQf8=; b=ig+V/8J+Kh+Gh1kJhA0hQ0QUUnA1anip8YVTEEaKAKnvag0WG3I7H2lsEFXWtYdxYI HDagcHmgIVtHDZNRKZQoqUYdB9luKhMYW2dsKn21UQZZOCJQZKj7AKq9qr2zSbkbm6K8 zjUx2aJ4iqyyWnmvjRgfHZYGM1cn1hRsqKafY= MIME-Version: 1.0 Received: by 10.231.28.209 with SMTP id n17mr2553578ibc.89.1320359213721; Thu, 03 Nov 2011 15:26:53 -0700 (PDT) Received: by 10.231.46.198 with HTTP; Thu, 3 Nov 2011 15:26:53 -0700 (PDT) In-Reply-To: <20111103115054.GA2155@trimind.de> References: <8453E2A2-3219-4FAA-98CE-2F9D66EA1C39@FreeBSD.org> <20111028094828.GA1781@trimind.de> <4EAF9E67.4040605@FreeBSD.org> <20111103115054.GA2155@trimind.de> Date: Thu, 3 Nov 2011 15:26:53 -0700 Message-ID: From: Kevin Oberman To: Sascha Klauder Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: antik@bsd.ee, current@freebsd.org, freebsd-geom@freebsd.org, Garrett Cooper , Martin Wilke , "Andrey V. Elsukov" Subject: Re: gmirror failed with error 19. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Nov 2011 22:48:40 -0000 On Thu, Nov 3, 2011 at 4:50 AM, Sascha Klauder wrote: > On Tue, 2011-11-01 11:23 +0400, Andrey V. Elsukov wrote: >> On 28.10.2011 13:48, Sascha Klauder wrote: >> > =A0GEOM_MIRROR: Device mirror/gm0 launched (2/2). >> > =A0GEOM_PART: partition 1 has end offset beyond last LBA: 490350671 > = 490350670 >> > =A0GEOM_PART: integrity check failed (mirror/gm0, MBR) >> This is the main problem. Your MBR' slice is bigger than actual space >> you have. The only way to fix this - recreate slice. > > =A0I've partioned and labeled the disk with sysinstall(8) from > 8.2-RELEASE media. =A0Does 9.0 use different disk geometry cal- > culation than 8.2 or is usage of gmirror the culprit? =A0Both > 8.2 and 9.0 kernels report the disk having 490350672 sectors. > >> You can break your mirror, recreate the slice (NOTE: you must preserve >> one sector for the gmirror's meta-data), then copy your data to the >> newly created slice, then reboot from the new slice and recreate mirror. > > =A0I think I'd rather reinstall 9.0 from scratch, getting rid > of MBR/disklabel as well. While using gpart and the 9.0 installer has some real benefits, it also has a few problems. I ave a disk that I partitioned with bsdinstall from the 9.0 distribution media and it worked fine for a STAT disk, but I am unable to boot te same disk when connected to a USB adapter. It shows up in the boot list, but when I select it, the screen blanks for a few seconds (<5) and the list re-appears. No errors, but my T520 BIOS clearly is not happy with it. I have no idea how common htis issue might be, but the T520 does have a UEFI BIOS. As noted, GEOM wants the final sector on the disk for its metadata, but GPT mandates that hte ast sector of the disk be used for a backup of the boot sector. The leads to the potential ofr all kinds if ugliness to develop, especially when BIOS, which knows nothing about GEOM, is accessing the disk. --=20 R. Kevin Oberman, Network Engineer E-mail: kob6558@gmail.com