From owner-freebsd-geom@FreeBSD.ORG Sun Dec 31 17:40:11 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B178E16A407 for ; Sun, 31 Dec 2006 17:40:11 +0000 (UTC) (envelope-from michael.knoll@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.189]) by mx1.freebsd.org (Postfix) with ESMTP id 4441913C442 for ; Sun, 31 Dec 2006 17:40:11 +0000 (UTC) (envelope-from michael.knoll@gmail.com) Received: by nf-out-0910.google.com with SMTP id x37so6202860nfc for ; Sun, 31 Dec 2006 09:40:10 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=BR3ilhcln8jchNXKCcnG+vZt20yPrSOgXpDIscngPyK2SoZSLZKVSGSzsxX8yNVpHenVoIXYBftrTSFRBgDeee+gvP7siwKmXI+32JVYGDx3W0aFWjCIUK4Gxhlar0eGIlWO1B3I1WnaszesRwhmVcGWjridt5TXeON3CEY+uIU= Received: by 10.78.171.13 with SMTP id t13mr1652063hue.1167585208488; Sun, 31 Dec 2006 09:13:28 -0800 (PST) Received: by 10.78.100.6 with HTTP; Sun, 31 Dec 2006 09:13:28 -0800 (PST) Message-ID: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> Date: Sun, 31 Dec 2006 12:13:28 -0500 From: "Michael Knoll" To: freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Dec 2006 17:40:11 -0000 I am planning on converting my FreeBSD machine to use gmirror on two 40gig drives. Reading the handbook and other sites with instructions on configuring gmirror, I notice they all disable swap. Is this acceptable? Is it expected swap be on another drive? If so, is there a solution which I can keep the swap on the mirror. as I don't have another drive? Thanks, Mike From owner-freebsd-geom@FreeBSD.ORG Sun Dec 31 17:47:04 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9915516A412 for ; Sun, 31 Dec 2006 17:47:04 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30304.mail.mud.yahoo.com (web30304.mail.mud.yahoo.com [209.191.69.66]) by mx1.freebsd.org (Postfix) with SMTP id 62AA013C428 for ; Sun, 31 Dec 2006 17:47:04 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 16912 invoked by uid 60001); 31 Dec 2006 17:46:59 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=MeoQAw2hUd5+o2xf1cClnusGcvQE7TAvFAV1g/K1baDaCi88udqE37XdSeTlZtRrAIW/1R2lgy7o2gPajYzw4kM0Rh3qnRVkQxBog+qGGj+HEiqXWne7QMLEevCzNT71tjbf0fQXv63A5N+ab2Q2jCuEVRZ7AkU8Q+JMKK7FPPc=; X-YMail-OSG: j0f0fHMVM1kK9EpLEXLvpmNutE7hBSkiOm0ICXfo3IgS_Ok3MMK01nS549HoCIiLf8NTO.vPzO9fUb9Y8iw2IlVb5XOhPZIHXaVuEn0khJjFEPgPtoeZrqzbmJtheikDXYceDUXAZHkS6Xi0GC9DeFuxEDCpTUTwXw-- Received: from [85.212.20.28] by web30304.mail.mud.yahoo.com via HTTP; Sun, 31 Dec 2006 09:46:59 PST Date: Sun, 31 Dec 2006 09:46:59 -0800 (PST) From: "R. B. Riddick" To: Michael Knoll , freebsd-geom@freebsd.org In-Reply-To: <975053160612310913t3dadcc02yfac58f6fbf0a49df@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <639505.15790.qm@web30304.mail.mud.yahoo.com> Cc: Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Dec 2006 17:47:04 -0000 --- Michael Knoll wrote: > I am planning on converting my FreeBSD machine to use gmirror on two > 40gig drives. Reading the handbook and other sites with instructions > on configuring gmirror, I notice they all disable swap. Is this > acceptable? Is it expected swap be on another drive? If so, is there > a solution which I can keep the swap on the mirror. as I don't have > another drive? > > Thanks, > Mike > _______________________________________________ > freebsd-geom@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-geom > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org" > I do it like this: % gmirror status Name Status Components mirror/sys COMPLETE ad0s1a ad1s1a mirror/home COMPLETE ad0s1d ad1s1d % graid5 status Name Status Components raid5/data COMPLETE HOT ad0s1e ad1s1e ad3e % swapinfo Device 1K-blocks Used Avail Capacity /dev/ad0s1b 250016 336 249680 0% % mount /dev/mirror/sys on / (ufs, local, soft-updates) devfs on /dev (devfs, local) /dev/mirror/home on /usr/home (ufs, local, soft-updates) /dev/raid5/data on /aux (ufs, local, soft-updates) /dev/ad3f on /opt (ufs, local, soft-updates) -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Sun Dec 31 17:49:04 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B5DA216A412 for ; Sun, 31 Dec 2006 17:49:04 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30306.mail.mud.yahoo.com (web30306.mail.mud.yahoo.com [209.191.69.68]) by mx1.freebsd.org (Postfix) with SMTP id 662FD13C448 for ; Sun, 31 Dec 2006 17:49:04 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 92072 invoked by uid 60001); 31 Dec 2006 17:49:03 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=XdpJEWgH324WeGWpK3vF0OBKDr2HpT5kAn1C/X82OinwAaQsQVLAc/LHtUJJum033cSgTGNdmOvB2yhyYnJPRZ4s2mjjjkBoNyrxbJUx2yN3MLmI4WFxzC3RLBJU+Hrn2uadlphbjTwuoNpfiy+x/ErVAvJ6xs+/V6z/0uUe3ws= ; Message-ID: <20061231174903.92070.qmail@web30306.mail.mud.yahoo.com> X-YMail-OSG: hctnn9wVM1lNqRA2JPqc.8H23r9ltdC3wTUj3sW0HqQZHEAR8iWgOcmrADT1Yt02d1_odWbKOFggbBQ59rd7QEtGjUbDDdo8g334arBEF1wTEQFmz4F2TW9pCeqCN7hiXCNeOA2GIbFTpU9ALLcOpPX8_yP4O7IB Received: from [85.212.20.28] by web30306.mail.mud.yahoo.com via HTTP; Sun, 31 Dec 2006 09:49:03 PST Date: Sun, 31 Dec 2006 09:49:03 -0800 (PST) From: "R. B. Riddick" To: Michael Knoll , freebsd-geom@freebsd.org In-Reply-To: <639505.15790.qm@web30304.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Recommended gmirror solution with swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Dec 2006 17:49:04 -0000 --- "R. B. Riddick" wrote: > % gmirror status > Name Status Components > mirror/sys COMPLETE ad0s1a > ad1s1a > mirror/home COMPLETE ad0s1d > ad1s1d > % graid5 status > Name Status Components > raid5/data COMPLETE HOT ad0s1e > ad1s1e > ad3e > Certainly this has a disadvantages in case of a rebuild, because: 2 disks have three concurrent streams of sequential rebuild-requests, so that the advantage of sequential-ness is destroyed, which makes the rebuild a very lengthly process... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Mon Dec 25 11:08:35 2006 Return-Path: X-Original-To: freebsd-geom@FreeBSD.org Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 51FBC16A54F for ; Mon, 25 Dec 2006 11:08:35 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [69.147.83.40]) by mx1.freebsd.org (Postfix) with ESMTP id 41ACE13C494 for ; Mon, 25 Dec 2006 11:08:35 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (linimon@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id kBPB8Zli034531 for ; Mon, 25 Dec 2006 11:08:35 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id kBPB8XfA034527 for freebsd-geom@FreeBSD.org; Mon, 25 Dec 2006 11:08:33 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 25 Dec 2006 11:08:33 GMT Message-Id: <200612251108.kBPB8XfA034527@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: linimon set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-geom@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Dec 2006 11:08:35 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/73177 geom kldload geom_* causes panic due to memory exhaustion o kern/76538 geom [gbde] nfs-write on gbde partition stalls and continue o kern/83464 geom [geom] [patch] Unhandled malloc failures within libgeo o kern/84556 geom [geom] GBDE-encrypted swap causes panic at shutdown o kern/87544 geom [gbde] mmaping large files on a gbde filesystem deadlo o kern/89102 geom [geom_vfs] [panic] panic when forced unmount FS from u o bin/90093 geom fdisk(8) incapable of altering in-core geometry o kern/90582 geom [geom_mirror] [panic] Restore cause panic string (ffs_ o kern/98034 geom [geom] dereference of NULL pointer in acd_geom_detach o kern/104389 geom [geom] [patch] sys/geom/geom_dump.c doesn't encode XML 10 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o bin/78131 geom gbde "destroy" not working. o kern/79251 geom [2TB] newfs fails on 2.6TB gbde device o kern/94632 geom [geom] Kernel output resets input while GELI asks for o kern/105390 geom [geli] filesystem on a md backed by sparse file with s 4 problems total. From owner-freebsd-geom@FreeBSD.ORG Tue Dec 26 03:11:15 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 262AF16A4D0 for ; Tue, 26 Dec 2006 03:11:15 +0000 (UTC) (envelope-from grafan@gmail.com) Received: from an-out-0708.google.com (an-out-0708.google.com [209.85.132.250]) by mx1.freebsd.org (Postfix) with ESMTP id BF7CD13C474 for ; Tue, 26 Dec 2006 03:11:14 +0000 (UTC) (envelope-from grafan@gmail.com) Received: by an-out-0708.google.com with SMTP id c24so1083779ana for ; Mon, 25 Dec 2006 19:11:13 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=hiV9CHkQKAGnkY/07dKlfhGRw7EQnO7nXaXHxSejuB0P2VqR0MBW++0Fjsq0yPu9ed3WPL0mu7XLiGfHqth3oCrm1z/hLFZhH0jW/anSSee2/4qLEEJzJXdKSHQecTLNHI+4/whgtvrA41/Yhesfhnn0eHabb7kecymKIPibSfc= Received: by 10.100.178.7 with SMTP id a7mr761761anf.1167101124827; Mon, 25 Dec 2006 18:45:24 -0800 (PST) Received: by 10.100.189.19 with HTTP; Mon, 25 Dec 2006 18:45:24 -0800 (PST) Message-ID: <6eb82e0612251845x21db27b7v9deffeb1215df293@mail.gmail.com> Date: Tue, 26 Dec 2006 10:45:24 +0800 From: "Rong-en Fan" To: freebsd-geom@freebsd.org In-Reply-To: <6eb82e0612190135n1d433873i61d432e6b165fcd1@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <6eb82e0612190135n1d433873i61d432e6b165fcd1@mail.gmail.com> Subject: gpt device node does not show at boot X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Dec 2006 03:11:15 -0000 I posted on stable@ few days ago. No replies. So I posted again here. ---------- Forwarded message ---------- From: Rong-en Fan Date: Dec 19, 2006 5:35 PM Subject: gpt device node does not show at boot To: "freebsd-stable@freebsd.org" I'm running 6.2-RC1 on i386. I use gpt(8) to partition my disk. After reboot, the device node, say da1p1, does not show up until 'gpt show da1' is issued. This prevents gpt partition being mounted from fstab, and therefore cannot be nfs exported at boot time! My kernel config is simply GENERIC+QUOTA+SMP. I also noticed that it is not possible to modify in-use disk's partition table. There is also a PR 85772 about it. Can someone comment on it? Thanks. Regards, Rong-En Fan From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 14:48:52 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6253816A416 for ; Wed, 27 Dec 2006 14:48:52 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from ferengi.borderworlds.dk (ferengi.borderworlds.dk [80.166.152.7]) by mx1.freebsd.org (Postfix) with ESMTP id 16C6413C479 for ; Wed, 27 Dec 2006 14:48:52 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from dominion.borderworlds.dk (dominion.borderworlds.dk [10.1.0.10]) by ferengi.borderworlds.dk (Postfix) with ESMTP id 90DC7B9B3 for ; Wed, 27 Dec 2006 15:18:44 +0100 (CET) Received: by dominion.borderworlds.dk (Postfix, from userid 2000) id CF9588C7; Wed, 27 Dec 2006 15:18:43 +0100 (CET) To: freebsd-geom@freebsd.org From: Christian Laursen Date: Wed, 27 Dec 2006 15:18:43 +0100 Message-ID: User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.50 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: geom_mirror delayed synchronizing question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 14:48:52 -0000 I am trying to add functionality to geom_mirror to prevent it from synchronizing mirrors until a given time even if the mirrors are configured for autosynchronization. The purpose of this is to be able to postpone the synchronization until fsck is done after an unclean shutdown. So far I have been succesful in doing part of this. I have added a tunable "kern.geom.mirror.sync" that tells geom_mirror whether sync is allowed. It is also available as a read-only sysctl. When this tunable is set to 0, mirrors are not synced. I have added a "gmirror allowsync" command that changes this variable to allow synchronization to happen. What is missing is automatic synchronization after running this command. If I manualle run "gmirror rebuild name prov" on the stale arrays, synchronization starts, but I need this to happen automatically. The small changes I have made so far: http://borderworlds.dk/~xi/gmirror.diff Any hints on how to write the code to find the relevant mirrors and initiate the rebuild would be much appreciated. Thanks. -- Christian Laursen From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 16:12:34 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9B14816A415 for ; Wed, 27 Dec 2006 16:12:34 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30308.mail.mud.yahoo.com (web30308.mail.mud.yahoo.com [209.191.69.70]) by mx1.freebsd.org (Postfix) with SMTP id 5FB3713C48F for ; Wed, 27 Dec 2006 16:12:34 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 81898 invoked by uid 60001); 27 Dec 2006 15:45:54 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=yygvyuXnP7SWhzPYCez1WPguAsR1rEjkw4GIKaaj8BqDHyQrFjKgwHUFhPNsGpP1qrxDFpYms1ix34rCHKh5WPtpJBt52QHt8Xwr92qJWaeO39V/6Vmjnnw/p1AVb/nRwJujAtRa0EqVwjTowDjeABQMe3nR+UzI+DEl1aDT2zs=; X-YMail-OSG: v3fhJUgVM1lVmWQ_ibqNOMGAMdCL87p.VoZuR6wXls9jSom8WKPAEjd9lowHjLeIlZ7KbOPUQe7EdTNBxAE3Iq5y2bhwf.UhJy54uLF158_sTIdbZjzhWAZZRC1n1WxZMMF3vbVtSG5owYU- Received: from [85.212.15.42] by web30308.mail.mud.yahoo.com via HTTP; Wed, 27 Dec 2006 07:45:54 PST Date: Wed, 27 Dec 2006 07:45:54 -0800 (PST) From: "R. B. Riddick" To: Christian Laursen , freebsd-geom@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <359629.81860.qm@web30308.mail.mud.yahoo.com> Cc: Subject: Re: geom_mirror delayed synchronizing question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 16:12:34 -0000 --- Christian Laursen wrote: > What is missing is automatic synchronization after running this > command. If I manualle run "gmirror rebuild name prov" on the stale > arrays, synchronization starts, but I need this to happen > automatically. > > The small changes I have made so far: > http://borderworlds.dk/~xi/gmirror.diff > > Any hints on how to write the code to find the relevant mirrors and > initiate the rebuild would be much appreciated. > Wasnt that feature already implemented? "graid5 configure -a" and "... -n"? I would be glad if it was possible for gmirror to detect an idle gmirror-device... If it is idle, requests r issued more frequent and if the device experiences concurrent requests (e. g. from fsck), gmirror does no or hardly any sync requests... But this strategy could lead to a very very lengthly rebuild process on a very busy server, which contracts a RAID0-like risk of (complete) device failure... Maybe this "nice feature" should be implemented somehow intelligently? Maybe phases of rebuild and phases where concurrent user-land requests r executed? E. g.: 10 second undisturbed user-land (if there r any pending requests) and then 1 second gmirror's rebuild requests... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 17:12:21 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 96AB516A403 for ; Wed, 27 Dec 2006 17:12:21 +0000 (UTC) (envelope-from fernan.aguero@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.226]) by mx1.freebsd.org (Postfix) with ESMTP id 5E10E13C478 for ; Wed, 27 Dec 2006 17:12:21 +0000 (UTC) (envelope-from fernan.aguero@gmail.com) Received: by nz-out-0506.google.com with SMTP id i11so1836905nzh for ; Wed, 27 Dec 2006 09:12:20 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=GaDrdzsbE26CDA9xGwwSWo12ooP2/R2dj9wns5KLCOy3m+b53z1AOSOQLjOClEPeu6aN+dfJThoQCZV9xFCvEV9Gj+z2W+xn89GPrgHoqkRYoP6BdBIVKP5p/BYpSugOPT/m7uNo46PmkkXVV6nZy5FmS4dWXRh3Rp3ukrrGey4= Received: by 10.35.60.15 with SMTP id n15mr26899818pyk.1167237982405; Wed, 27 Dec 2006 08:46:22 -0800 (PST) Received: by 10.35.51.11 with HTTP; Wed, 27 Dec 2006 08:46:22 -0800 (PST) Message-ID: <520894aa0612270846s89a58cdh4b2bacfe92f75dff@mail.gmail.com> Date: Wed, 27 Dec 2006 13:46:22 -0300 From: "Fernan Aguero" To: freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: gmirror issues (fdisk?, disklabel?, newfs?) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 17:12:21 -0000 Hi! I've got two identical disks (ad4, ad6) that I want to set up to use geom (swap in slice1, stripe slice2, mirror slice3). I'm following the article at http://people.freebsd.org/~rse/mirror/ and the geom chapter of the FreeBSD handbook, however the examples are not exactly what I want to achieve, and I'm confused by a bsdlabel error, and by newfs taken a very long time (hung?). ad4 has FreeBSD installed (single slice, simple partition scheme). I'm trying to set up the mirror with a single slice (ad6s3) first, then copy data from ad4 to ad6, then reformat ad4 and add ad4s3 to the mirror. But, I could not get it done. It seems like I'm able to make my box freeze by trying to newfs the partitions in gm0s3c (note the 'c'). This is what I did with ad6: dd if=/dev/zero of=/dev/ad6 bs=512 count=79 => SUCCESS next I created 3 slices using sysinstall (express install (experts), selected ad6 and created 3 slices (s1: 8Gb, s2: 40Gb, s3: all remaining space ~ 100 Gb), made s3 bootable, wrote, quit. Then sysinstall asked me about installing an MBR and I said yes, and then exit from sysinstall. fdisk now prints the following: gama# fdisk /dev/ad6 ******* Working on device /dev/ad6 ******* parameters extracted from in-core disklabel are: cylinders=310101 heads=16 sectors/track=63 (1008 blks/cyl) Figures below won't work with BIOS for partitions not in cyl 1 parameters to be used for BIOS calculations are: cylinders=310101 heads=16 sectors/track=63 (1008 blks/cyl) Media sector size is 512 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 63, size 16771797 (8189 Meg), flag 0 beg: cyl 0/ head 1/ sector 1; end: cyl 1023/ head 254/ sector 63 The data for partition 2 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 16771860, size 83875365 (40954 Meg), flag 0 beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 254/ sector 63 The data for partition 3 is: sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD) start 100647225, size 211929480 (103481 Meg), flag 80 (active) beg: cyl 1023/ head 255/ sector 63; end: cyl 1023/ head 254/ sector 63 The data for partition 4 is: Now I set up the mirror: gmirror label -n -v -b round-robin gm0s3 /dev/ad6s3 => SUCCESS Metadata value stored on /dev/ad6s3 Done. gmirror load => SUCCESS Next I try to create partitions on the newly created gm0s3: bsdlabel -w -B /dev/mirror/gm0s3 => FAIL bsdlabel: write /dev/mirror/gm0s3: Operation not permitted Why is this failing? when I list the mirror providers/consumers, I see that the provider is 'gm0s3' but the consumer is 'ad6s3c' and not 'ad6s3': gmirror list Geom name: gm0s3 State: COMPLETE Components: 1 Balance: round-robin Slice: 4096 Flags: NOAUTOSYNC GenID: 0 SyncID: 1 ID: 853713771 Providers: 1. Name: mirror/gm0s3 Mediasize: 108507893248 (101G) Sectorsize: 512 Mode: r0w0e0 Consumers: 1. Name: ad6s3c Mediasize: 108507893760 (101G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: BROKEN GenID: 0 SyncID: 1 ID: 538599761 So I figured out I could just try to partition gm0s3c instead ... bsdlabel -w -B /dev/mirror/gm0s3c => SUCCESS bsdlabel -e /dev/mirror/gm0s3c and this is the partition table: bsdlabel /dev/mirror/gm0s3c # /dev/mirror/gm0s3c: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 2097152 16 4.2BSD 0 0 0 c: 211929480 0 unused 0 0 # "raw" part, don't edit d: 2097152 2097168 4.2BSD 0 0 0 e: 20971520 4194320 4.2BSD 0 0 0 f: 41943040 25165840 4.2BSD 0 0 0 g: 62914560 67108880 4.2BSD 0 0 0 h: 81906040 130023440 4.2BSD 0 0 0 Now it's impossible to newfs the partitions. It seems like this is getting the box to freeze ... newfs -U /dev/mirror/gm0s3a would never finish even though it's only 1Gb ... even more, it's immune to Ctrl-C and kill -9. This is how top shows newfs: 836 root 1 -8 0 1464K 1020K gwrite 0:00 0.00% newfs after a while the system becomes unresponsive and needs a hard reset. This is all on FreeBSD-6.1p10, i386. What am I doing wrong? Thanks in advance, Fernan -- Fernan Aguero From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 17:29:32 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BD23B16A40F for ; Wed, 27 Dec 2006 17:29:32 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from ferengi.borderworlds.dk (ferengi.borderworlds.dk [80.166.152.7]) by mx1.freebsd.org (Postfix) with ESMTP id 6A7E313C46D for ; Wed, 27 Dec 2006 17:29:32 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from dominion.borderworlds.dk (dominion.borderworlds.dk [10.1.0.10]) by ferengi.borderworlds.dk (Postfix) with ESMTP id 46A77B80C for ; Wed, 27 Dec 2006 18:29:31 +0100 (CET) Received: by dominion.borderworlds.dk (Postfix, from userid 2000) id C6FE88C7; Wed, 27 Dec 2006 18:29:30 +0100 (CET) To: "R. B. Riddick" References: <359629.81860.qm@web30308.mail.mud.yahoo.com> From: Christian Laursen Date: Wed, 27 Dec 2006 18:29:30 +0100 In-Reply-To: <359629.81860.qm@web30308.mail.mud.yahoo.com> (R. B. Riddick's message of "Wed, 27 Dec 2006 07:45:54 -0800 (PST)") Message-ID: User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.50 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-geom@freebsd.org Subject: Re: geom_mirror delayed synchronizing question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 17:29:32 -0000 "R. B. Riddick" writes: > --- Christian Laursen wrote: >> What is missing is automatic synchronization after running this >> command. If I manualle run "gmirror rebuild name prov" on the stale >> arrays, synchronization starts, but I need this to happen >> automatically. >> >> The small changes I have made so far: >> http://borderworlds.dk/~xi/gmirror.diff >> >> Any hints on how to write the code to find the relevant mirrors and >> initiate the rebuild would be much appreciated. >> > Wasnt that feature already implemented? > "graid5 configure -a" and "... -n"? No, -a and -n enables and disables autosynchronization. I want autosynchronization but with a delay. > I would be glad if it was possible for gmirror to detect an idle > gmirror-device... If it is idle, requests r issued more frequent and if the > device experiences concurrent requests (e. g. from fsck), gmirror does no or > hardly any sync requests... That would probably work too. -- Christian Laursen From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 17:35:28 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7CDED16A412 for ; Wed, 27 Dec 2006 17:35:28 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30314.mail.mud.yahoo.com (web30314.mail.mud.yahoo.com [209.191.69.76]) by mx1.freebsd.org (Postfix) with SMTP id 2F4D513C487 for ; Wed, 27 Dec 2006 17:35:27 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 18081 invoked by uid 60001); 27 Dec 2006 17:35:27 -0000 Message-ID: <20061227173527.18079.qmail@web30314.mail.mud.yahoo.com> DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=mrr2dNj4kjDDCIwhl3FvmeYOVi5pj/1z7I2bA+bgj9xMg9Ly7miawBFYalQKaNd1licJ32S8lE9eJgfzsC0Dhw0HX1YZLVPraBp03CGrpAeoxkkLN0U7d+Q8AwOqy1ibicuTHMfkwzhmij3N0KOR+thPHw+2/4d6Gjh4eTDnQB8=; X-YMail-OSG: EryTZ50VM1mVuYewWcqPI0dyt4YF2LxvEJ0RdXDLVGgd3K84Cy5RVtMN0DDLlm1jcplnQuygAyHltR4RaGUj5q.RgGmHQ04fxiKtDuldfhu.OJFbWUwBj4kdxXDrNW82VUd24TROqtGKSA-- Received: from [85.212.15.42] by web30314.mail.mud.yahoo.com via HTTP; Wed, 27 Dec 2006 09:35:27 PST Date: Wed, 27 Dec 2006 09:35:27 -0800 (PST) From: "R. B. Riddick" To: Christian Laursen In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-geom@freebsd.org Subject: Re: geom_mirror delayed synchronizing question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 17:35:28 -0000 --- Christian Laursen wrote: > No, -a and -n enables and disables autosynchronization. I want > autosynchronization but with a delay. > Then just issue "gmirror configure -a" by need (delayed; e. g. from /usr/local/etc/rc.d/gmirror-chk.sh) and disable it again, when sync is done (or started? I would have to test that...)... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 17:43:46 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E335716A407 for ; Wed, 27 Dec 2006 17:43:46 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from ferengi.borderworlds.dk (ferengi.borderworlds.dk [80.166.152.7]) by mx1.freebsd.org (Postfix) with ESMTP id A43B213C46F for ; Wed, 27 Dec 2006 17:43:46 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from dominion.borderworlds.dk (dominion.borderworlds.dk [10.1.0.10]) by ferengi.borderworlds.dk (Postfix) with ESMTP id 9F55BB99E for ; Wed, 27 Dec 2006 18:43:45 +0100 (CET) Received: by dominion.borderworlds.dk (Postfix, from userid 2000) id 4AF408C7; Wed, 27 Dec 2006 18:43:44 +0100 (CET) To: "R. B. Riddick" References: <20061227173527.18079.qmail@web30314.mail.mud.yahoo.com> From: Christian Laursen Date: Wed, 27 Dec 2006 18:43:43 +0100 In-Reply-To: <20061227173527.18079.qmail@web30314.mail.mud.yahoo.com> (R. B. Riddick's message of "Wed, 27 Dec 2006 09:35:27 -0800 (PST)") Message-ID: User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.50 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-geom@freebsd.org Subject: Re: geom_mirror delayed synchronizing question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 17:43:47 -0000 "R. B. Riddick" writes: > --- Christian Laursen wrote: >> No, -a and -n enables and disables autosynchronization. I want >> autosynchronization but with a delay. >> > Then just issue "gmirror configure -a" by need (delayed; e. g. from > /usr/local/etc/rc.d/gmirror-chk.sh) and disable it again, when sync is done (or > started? I would have to test that...)... No, I would then need to disable autosync before the machine booted and enable it again late it the boot process. There are plenty of hacks that will do what I want but none of them are elegant. I didn't ask the question because I don't know how to hack around things but because I want a clean solution to the problem. -- Christian Laursen From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 17:57:26 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6A81316A403 for ; Wed, 27 Dec 2006 17:57:26 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30312.mail.mud.yahoo.com (web30312.mail.mud.yahoo.com [209.191.69.74]) by mx1.freebsd.org (Postfix) with SMTP id 33E5413C487 for ; Wed, 27 Dec 2006 17:57:26 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 44694 invoked by uid 60001); 27 Dec 2006 17:57:21 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=hcrNmaswJCopag4oxOsLOUy83FmTU1gAmYenUAiB/hthu6rwlwobihaYjaQXLoipjFAO8CFgjMiEeJIN15roW1ANapRHCqyxaGEp1N4MnmtgED5zbdIO3Xw3IPkHESOzNhTOvCivHTQUNk5MdDEqwzG6AXhOpcahKoBajhAJ+3g=; X-YMail-OSG: YVjsFScVM1kdIPDppeGfnPyRxhuCsDU7KyzeTm0xB87hWrmpgZ_aKnivONRzQq2JYXtA.I25HBGiJy4nqhJyuq_tR5Ruxb.ryowI_onkcHjt3_xYK3l439EobFfNWER6iPd6iU2ylK_s__k- Received: from [85.212.15.42] by web30312.mail.mud.yahoo.com via HTTP; Wed, 27 Dec 2006 09:57:21 PST Date: Wed, 27 Dec 2006 09:57:21 -0800 (PST) From: "R. B. Riddick" To: "R. B. Riddick" , Christian Laursen In-Reply-To: <20061227173527.18079.qmail@web30314.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <507320.43622.qm@web30312.mail.mud.yahoo.com> Cc: freebsd-geom@freebsd.org Subject: Re: geom_mirror delayed synchronizing question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 17:57:26 -0000 --- "R. B. Riddick" wrote: > --- Christian Laursen wrote: > > No, -a and -n enables and disables autosynchronization. I want > > autosynchronization but with a delay. > > > Then just issue "gmirror configure -a" by need (delayed; e. g. from > /usr/local/etc/rc.d/gmirror-chk.sh) and disable it again, when sync is done > (or > started? I would have to test that...)... > I have an even better idea (quick hack): 0. Turn off auto-sync: "gmirror configure -n " 1. we could schedule a gmirror-chk.sh execution every 5 minutes: */5 * * * * /root/gmirror-chk.sh (note: crond is started after fsck) 2. /root/gmirror-chk.sh would look like this: #!/bin/sh gmirror status | grep / | cut -d/ -f2 | cut -d\ -f1 | \ while read dn ;do na="" fs=1 gmirror list $dn | \ while read ln ;do if [ `echo $ln | grep -c "^[1-9]"` -ne 0 ]; then if [ $fs -eq 1 ]; then fs=0 else na=`echo $ln | cut -d\ -f3` fi elif [ ! -z "$na" ]; then if [ `echo $ln | grep -c "Flags: SYNCHRONIZING"` -ne 0 ]; then gmirror rebuild $dn $na break fi fi done done [TESTED OK R6.1] -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 17:58:16 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C3E9716A40F for ; Wed, 27 Dec 2006 17:58:16 +0000 (UTC) (envelope-from etc@fluffles.net) Received: from auriate.fluffles.net (cust.95.160.adsl.cistron.nl [195.64.95.160]) by mx1.freebsd.org (Postfix) with ESMTP id 7F72713C47A for ; Wed, 27 Dec 2006 17:58:16 +0000 (UTC) (envelope-from etc@fluffles.net) Received: from destiny ([10.0.0.21]) by auriate.fluffles.net with esmtpa (Exim 4.63 (FreeBSD)) (envelope-from ) id 1GzcOt-000NmM-Nc for freebsd-geom@freebsd.org; Wed, 27 Dec 2006 18:17:11 +0100 Message-ID: <4592AAA9.70008@fluffles.net> Date: Wed, 27 Dec 2006 18:17:29 +0100 From: Fluffles User-Agent: Thunderbird 1.5.0.8 (X11/20061114) MIME-Version: 1.0 To: freebsd-geom@freebsd.org References: <520894aa0612270846s89a58cdh4b2bacfe92f75dff@mail.gmail.com> In-Reply-To: <520894aa0612270846s89a58cdh4b2bacfe92f75dff@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: gmirror issues (fdisk?, disklabel?, newfs?) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 17:58:16 -0000 Fernan Aguero wrote: > gmirror label -n -v -b round-robin gm0s3 /dev/ad6s3 => SUCCESS > Metadata value stored on /dev/ad6s3 > Done. > gmirror load => SUCCESS > > Next I try to create partitions on the newly created gm0s3: > > bsdlabel -w -B /dev/mirror/gm0s3 => FAIL > bsdlabel: write /dev/mirror/gm0s3: Operation not permitted > > Why is this failing? Because the GEOM device is 'in use' by gmirror; you have labeled it and it's probably configured then; no direct write access to the geom device is possible then unless you play with debugvars. I suggest you label (install bootblock etc) before you give a "gmirror label" command. Make sure gmirror is unloaded, too, because if it's loaded it will look for the last sector on all geom providers for stored metadata; and configure a mirror device automatically thus denying you any direct write access to the device. - Veronica From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 17:58:32 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B090316A40F for ; Wed, 27 Dec 2006 17:58:32 +0000 (UTC) (envelope-from fernan.aguero@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.226]) by mx1.freebsd.org (Postfix) with ESMTP id 7403B13C47C for ; Wed, 27 Dec 2006 17:58:32 +0000 (UTC) (envelope-from fernan.aguero@gmail.com) Received: by nz-out-0506.google.com with SMTP id i11so1844257nzh for ; Wed, 27 Dec 2006 09:58:29 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=lzk8Fx/yMgIvgOKcwz47fT3+XfiVrWakTMjT++666wjJCY5uWOkAP5jGagRiwSR3Ox8oVwlXax4quAldlGJ8CAyd4cE/mywThGNxcTEr4I2xbBAft/xhb/iVnDj3mA3VlbLwU6kh7Fc3ImSwGmFyYc/jl30NG0qSToaXn2dWct0= Received: by 10.35.54.1 with SMTP id g1mr27015309pyk.1167242309372; Wed, 27 Dec 2006 09:58:29 -0800 (PST) Received: by 10.35.51.11 with HTTP; Wed, 27 Dec 2006 09:58:28 -0800 (PST) Message-ID: <520894aa0612270958g5eecb6aetad8c0a4f8decd207@mail.gmail.com> Date: Wed, 27 Dec 2006 14:58:28 -0300 From: "Fernan Aguero" To: Fluffles In-Reply-To: <4592AA91.5080105@fluffles.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <520894aa0612270846s89a58cdh4b2bacfe92f75dff@mail.gmail.com> <4592AA91.5080105@fluffles.net> Cc: freebsd-geom@freebsd.org Subject: Re: gmirror issues (fdisk?, disklabel?, newfs?) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 17:58:32 -0000 On 12/27/06, Fluffles wrote: > Fernan Aguero wrote: > > gmirror label -n -v -b round-robin gm0s3 /dev/ad6s3 => SUCCESS > > Metadata value stored on /dev/ad6s3 > > Done. > > gmirror load => SUCCESS > > > > Next I try to create partitions on the newly created gm0s3: > > > > bsdlabel -w -B /dev/mirror/gm0s3 => FAIL > > bsdlabel: write /dev/mirror/gm0s3: Operation not permitted > > > > Why is this failing? > > Because the GEOM device is 'in use' by gmirror; you have labeled it and > it's probably configured then; no direct write access to the geom device > is possible then unless you play with debugvars. I suggest you label > (install bootblock etc) before you give a "gmirror label" command. Veronica, thanks for the help. I just did all of fdisk+disklabel+newfs/tunefs before creating (labeling) and activating the mirror and it worked fine! Actually now that I think about it, what you say sounds pretty reasonable, but none of the articles I read mention anything about being unable to label or partition after loading (activating) the gmirror. In fact, they all activate gmirror before either bsdlabel/newfs or even before fdisk: http://people.freebsd.org/~rse/mirror/ (see second example to mirror slices) http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/geom-mirror.html I have just but guess that none of the examples in those articles can be reproduced :| Thanks again, Fernan > Make > sure gmirror is unloaded, too, because if it's loaded it will look for > the last sector on all geom providers for stored metadata; and configure > a mirror device automatically thus denying you any direct write access > to the device. > > - Veronica > -- Fernan Aguero From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 18:01:00 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A1B2A16A492 for ; Wed, 27 Dec 2006 18:01:00 +0000 (UTC) (envelope-from simon@zaphod.nitro.dk) Received: from mx.nitro.dk (zarniwoop.nitro.dk [83.92.207.38]) by mx1.freebsd.org (Postfix) with ESMTP id 6174913C48E for ; Wed, 27 Dec 2006 18:00:58 +0000 (UTC) (envelope-from simon@zaphod.nitro.dk) Received: from zaphod.nitro.dk (unknown [192.168.3.39]) by mx.nitro.dk (Postfix) with ESMTP id 80343386C02; Wed, 27 Dec 2006 17:32:16 +0000 (UTC) Received: by zaphod.nitro.dk (Postfix, from userid 3000) id 62FC31141E; Wed, 27 Dec 2006 18:32:41 +0100 (CET) Date: Wed, 27 Dec 2006 18:32:41 +0100 From: "Simon L. Nielsen" To: Christian Laursen Message-ID: <20061227173240.GD1063@zaphod.nitro.dk> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.11 Cc: freebsd-geom@freebsd.org Subject: Re: geom_mirror delayed synchronizing question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 18:01:00 -0000 On 2006.12.27 15:18:43 +0100, Christian Laursen wrote: > I am trying to add functionality to geom_mirror to prevent it from > synchronizing mirrors until a given time even if the mirrors are > configured for autosynchronization. The purpose of this is to be able > to postpone the synchronization until fsck is done after an unclean > shutdown. Is this mainly for forground or background fsck? For background fsck at least, wouldn't it be simpler to just postpone fsck until the sync is done? -- Simon L. Nielsen From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 18:03:05 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 58C9816A407 for ; Wed, 27 Dec 2006 18:03:05 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30304.mail.mud.yahoo.com (web30304.mail.mud.yahoo.com [209.191.69.66]) by mx1.freebsd.org (Postfix) with SMTP id 0AFC913C46D for ; Wed, 27 Dec 2006 18:03:04 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 53323 invoked by uid 60001); 27 Dec 2006 18:03:04 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=Lw8DFQay23m1p5NjIu8+9IHqe8aGiB8/HPOKz3sLj7vAEmeUwQHfd2mRrDnCN4jq6GOKsnDefBRydBE2iHeK464VPByeSaRQTsRztg43Ih37/IwTtc8+tHom/ITGYHjQjjTIs5RGOByVJmGk7TgLcaxYxZkcq1kYB5JV0FP++eI=; X-YMail-OSG: gF4.jbAVM1kZYGDeD0PEK49qMPbU4tN8ZX_KhlC7U4oOOHoEG.NlcFkN_AGiQjuRId1WgJgjA8W2uzu8cjqK7SuR3zvz5mxS5uWcFEgX7J3GUMdQ4z7.RUH2Pagy96R5n1U9SGc0dCeHJHM- Received: from [85.212.15.42] by web30304.mail.mud.yahoo.com via HTTP; Wed, 27 Dec 2006 10:03:04 PST Date: Wed, 27 Dec 2006 10:03:04 -0800 (PST) From: "R. B. Riddick" To: Christian Laursen In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <485248.53145.qm@web30304.mail.mud.yahoo.com> Cc: freebsd-geom@freebsd.org Subject: Re: geom_mirror delayed synchronizing question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 18:03:05 -0000 --- Christian Laursen wrote: > No, I would then need to disable autosync before the machine booted > and enable it again late it the boot process. > Yup! Sure. Turn off auto-sync and check the status of ur gmirror-s via crontab... > There are plenty of hacks that will do what I want but none of them > are elegant. > Hmm... That intelligent solution, that would (aa) try not to hinder user-land programs or that would (bb) try to find a compromise between user-land and those sequential re-sync requests, is something, that pawel could do best (since he is the author of gmirror)... > I didn't ask the question because I don't know how to hack around > things but because I want a clean solution to the problem. > Sounded like u wanted a hack since fsck isnt the only user-land program, whose performance can be hurt by rebuilding a gmirror... Sorry for all the traffic! -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 18:11:49 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5B4D116A407; Wed, 27 Dec 2006 18:11:49 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from ferengi.borderworlds.dk (ferengi.borderworlds.dk [80.166.152.7]) by mx1.freebsd.org (Postfix) with ESMTP id 1B29F13C47E; Wed, 27 Dec 2006 18:11:49 +0000 (UTC) (envelope-from xi@borderworlds.dk) Received: from dominion.borderworlds.dk (dominion.borderworlds.dk [10.1.0.10]) by ferengi.borderworlds.dk (Postfix) with ESMTP id 38813B80C; Wed, 27 Dec 2006 19:11:48 +0100 (CET) Received: by dominion.borderworlds.dk (Postfix, from userid 2000) id 111A38C7; Wed, 27 Dec 2006 19:11:46 +0100 (CET) To: "Simon L. Nielsen" References: <20061227173240.GD1063@zaphod.nitro.dk> From: Christian Laursen Date: Wed, 27 Dec 2006 19:11:46 +0100 In-Reply-To: <20061227173240.GD1063@zaphod.nitro.dk> (Simon L. Nielsen's message of "Wed, 27 Dec 2006 18:32:41 +0100") Message-ID: User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.50 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: freebsd-geom@freebsd.org Subject: Re: geom_mirror delayed synchronizing question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 18:11:49 -0000 "Simon L. Nielsen" writes: > On 2006.12.27 15:18:43 +0100, Christian Laursen wrote: >> I am trying to add functionality to geom_mirror to prevent it from >> synchronizing mirrors until a given time even if the mirrors are >> configured for autosynchronization. The purpose of this is to be able >> to postpone the synchronization until fsck is done after an unclean >> shutdown. > > Is this mainly for forground or background fsck? For background fsck > at least, wouldn't it be simpler to just postpone fsck until the sync > is done? I think it makes sense for both foreground and background fsck but I agree that for background fsck it would work to just postpone the fsck. However, we use foreground fsck everywhere so that is my primary concern. -- Christian Laursen From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 18:18:13 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 84AE316A47C for ; Wed, 27 Dec 2006 18:18:13 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30310.mail.mud.yahoo.com (web30310.mail.mud.yahoo.com [209.191.69.72]) by mx1.freebsd.org (Postfix) with SMTP id 306F913C498 for ; Wed, 27 Dec 2006 18:18:13 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 86069 invoked by uid 60001); 27 Dec 2006 18:18:12 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=zo1Yqsa1+idWo5dy236ydi53zzgyvwr4RIcf618ZQTRtIwTeP0DbqmX+kWNhTll+HTr3HosdDKF1zdId/tw9+0xXgM9AO7nDmkVO2NAueyuwMhBk9XzKK4pOMPWAF/BenCk0F3qeSwfrYBq8SNRa9BQ5GBNOMGaXbmDX+5wsEnc= ; Message-ID: <20061227181812.86067.qmail@web30310.mail.mud.yahoo.com> X-YMail-OSG: THT4by8VM1kMq9.ofNG2s2BFLAGyQRbxfur0CD0ELtvIFoRb9V614ImeG_dAlHsp_tEPXnL1X.q687_qUo4E14_ctXqpyniTiIVseAvF2mVlNNSbgCEhXklIHHDOiqkcFRAROaKyIrM- Received: from [85.212.15.42] by web30310.mail.mud.yahoo.com via HTTP; Wed, 27 Dec 2006 10:18:12 PST Date: Wed, 27 Dec 2006 10:18:12 -0800 (PST) From: "R. B. Riddick" To: Fluffles , freebsd-geom@freebsd.org In-Reply-To: <4592AAA9.70008@fluffles.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: gmirror issues (fdisk?, disklabel?, newfs?) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 18:18:13 -0000 Fernan Aguero wrote: > gmirror label -n -v -b round-robin gm0s3 /dev/ad6s3 => SUCCESS > Metadata value stored on /dev/ad6s3 > Done. > gmirror load => SUCCESS > > Next I try to create partitions on the newly created gm0s3: > > bsdlabel -w -B /dev/mirror/gm0s3 => FAIL > bsdlabel: write /dev/mirror/gm0s3: Operation not permitted > > Why is this failing? > On R6.1 it works like it should: neo# gmirror label -n -v -b round-robin fook md0 Metadata value stored on md0. Done. neo# bsdlabel -w -B /dev/mirror/fook neo# gmirror status Name Status Components mirror/fook COMPLETE md0 neo# bsdlabel /dev/mirror/fook # /dev/mirror/fook: 8 partitions: # size offset fstype [fsize bsize bps/cpg] a: 32751 16 unused 0 0 c: 32767 0 unused 0 0 # "raw" part, don't edit -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 20:05:04 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5F76516A415 for ; Wed, 27 Dec 2006 20:05:04 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 1896D13C479 for ; Wed, 27 Dec 2006 20:05:04 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from root by ciao.gmane.org with local (Exim 4.43) id 1Gzehy-0000nd-Fa for freebsd-geom@freebsd.org; Wed, 27 Dec 2006 20:45:02 +0100 Received: from homer.cload.net ([213.41.241.56]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 27 Dec 2006 20:45:02 +0100 Received: from ohmer by homer.cload.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 27 Dec 2006 20:45:02 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Matthieu Michaud Date: Wed, 27 Dec 2006 20:26:11 +0100 Lines: 12 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: homer.cload.net User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) Sender: news Subject: vmware + gconcat X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 20:05:04 -0000 hello, did someone success on mounting a gconcat slice under freebsd running as guest os of vmware ? host is vmware server running on windows xp. freebsd version is 6.2RC1. i succeeded on mounting simple ufs slice. thanks for any help ! -- Matthieu Michaud From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 23:43:36 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6245216A40F for ; Wed, 27 Dec 2006 23:43:36 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30304.mail.mud.yahoo.com (web30304.mail.mud.yahoo.com [209.191.69.66]) by mx1.freebsd.org (Postfix) with SMTP id 39D8C13C46D for ; Wed, 27 Dec 2006 23:43:36 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 64429 invoked by uid 60001); 27 Dec 2006 23:43:35 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=xD2uTU0Tzd5+IrwNoArapIREf666I3bJ3RL4y0NHXHeQzFGfMb73z1ImrBGCY/mVv0U2z0+56+6qBj2lcJIUxXA5w01atxZfWSs09Sf1P2O4tiVu+4of6VThm2I/6TweY3gQUKqumANcnkRq7dim/ehZQhTxQOdAVbBX+JKp3I4=; X-YMail-OSG: a4l7v1MVM1mMC43kBxiqifmzzK6_pYzIpufKQoZkiLs9ztmkMb5ny5sPtYa6na8bJjP0J3bOxtxYlvFRIMdTGzGnt7Usg95Et7gjOdfPJPEf_VY4ogW_EKtKuJ.V_wVVgH5IzQOO.RlLRQcm21nT7dwuAed44ESR_Q-- Received: from [85.212.15.42] by web30304.mail.mud.yahoo.com via HTTP; Wed, 27 Dec 2006 15:43:35 PST Date: Wed, 27 Dec 2006 15:43:35 -0800 (PST) From: "R. B. Riddick" To: rick-freebsd@kiwi-computer.com In-Reply-To: <20061227232712.GA90336@keira.kiwi-computer.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <631303.64130.qm@web30304.mail.mud.yahoo.com> Cc: freebsd-geom@freebsd.org Subject: Re: gmirror issues (fdisk?, disklabel?, newfs?) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 23:43:36 -0000 --- "Rick C. Petty" wrote: > On Wed, Dec 27, 2006 at 10:18:12AM -0800, R. B. Riddick wrote: > > > > On R6.1 it works like it should: > > I've seen just the opposite behavior, in 6.2-R. Once geom_mirror is > loaded, I can't use bsdlabel or fdisk: > Ohoh... [Tubby play with pudding-machine...]^2 Since I do not want to use my DSL flat-rate too much, I did not test 6.2-RC1... But I will try R6.2, as soon as possible... > # boot0cfg -B ad1 > boot0cfg: /dev/ad1: Geom not found > boot0cfg: write_mbr: /dev/ad1: No such file or directory > Hmm... The error message is misleading... But it is OK, that /dev/ad1 cannot be opened for writing as soon as gmirror uses it as a consumer, because: gmirror has no chance to notice changes that go directly to /dev/ad1, so that /dev/ad0 would stay unchanged (and so possibly the mirror is not sync'ed and gmirror does not mention that immediately). > # gmirror status > Name Status Components > mirror/gm0 COMPLETE ad0 > ad1 > see above > # boot0cfg -B /dev/mirror/gm0 > boot0cfg: /dev/mirror/gm0: Geom not found > boot0cfg: /dev/mirror/gm0: ioctl DIOCSMBR: Operation not permitted > This is strange, because: gmirror should certainly allow write access to its devices (providers)... Sounds really strange... Luckily someone mentioned that before R6.2... :-) > So, once you enable any geom layer, you no longer have write access to the > boot blocks or otherwise. This has been broken ever since GEOM was > introduced in 5.0. I've been bitten many times by this in: gmirror, > gvinum, gpt, glabel, etc. > Hmm... But why does it work here on my box with R6.1? I just used boot0cfg on my /dev/ad0, which has geom_bsd (/dev/ad0s1) and gmirror (e.g.: /dev/ad0s1a and /dev/ad1s1a build one gmirror) on top (I could successfully change the ticks and the default choice)... > What's worse is that these modules don't have a way to "untaste". If you > load the module, all providers get tasted and there's no way back. I got > burned last week when gmirror decided to flip out and it whacked ad1. > The system panicked on every boot. The only remedy was to boot a 5.5-R > media (IIRC) and do "gmirror clear ad1" *before* geom_mirror was loaded. > IIRC, 6.0, 6.1, and all the other 5.x media I had didn't work. What a > PITA! > Yup - some special cases r not handled as gracefully as they could be handled... I had another one: gmirror out of ad1s1a and ad0s1a, where ad0s1a was rebuilding... in order to stop rebuild I decided to remove ad1s1a which left a unusable gmirror device and a panic (certainly after reboot, too). Unfortunately I do not remember the kind of panic... If it was a bad memory access or just an ASSERT... But it crashed repeatedly until I disconnected the disk that had that bad gmirror-meta-info... :-) > It would be nice if the tools had a "load but don't taste" command, an > "untaste" command, and a "taste" command. Until then, it all just feels > so incomplete, like it was hacked together. > *sob* :-) I would say this "untaste" command wouldnt be necessary, if the geom classes would handle every single special case as gracefully as possible... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Wed Dec 27 23:49:37 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BAEF016A403 for ; Wed, 27 Dec 2006 23:49:37 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id 600AD13C480 for ; Wed, 27 Dec 2006 23:49:37 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 91459 invoked by uid 2001); 27 Dec 2006 23:27:12 -0000 Date: Wed, 27 Dec 2006 17:27:12 -0600 From: "Rick C. Petty" To: "R. B. Riddick" Message-ID: <20061227232712.GA90336@keira.kiwi-computer.com> References: <4592AAA9.70008@fluffles.net> <20061227181812.86067.qmail@web30310.mail.mud.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20061227181812.86067.qmail@web30310.mail.mud.yahoo.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-geom@freebsd.org Subject: Re: gmirror issues (fdisk?, disklabel?, newfs?) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2006 23:49:37 -0000 On Wed, Dec 27, 2006 at 10:18:12AM -0800, R. B. Riddick wrote: > > On R6.1 it works like it should: I've seen just the opposite behavior, in 6.2-R. Once geom_mirror is loaded, I can't use bsdlabel or fdisk: # boot0cfg -B ad1 boot0cfg: /dev/ad1: Geom not found boot0cfg: write_mbr: /dev/ad1: No such file or directory # ls /dev/ad* /dev/ad0 /dev/ad2 /dev/ad2s1a /dev/ad2s1c /dev/ad2s1e /dev/ad2s1g /dev/ad1 /dev/ad2s1 /dev/ad2s1b /dev/ad2s1d /dev/ad2s1f # gmirror status Name Status Components mirror/gm0 COMPLETE ad0 ad1 I have the same problem with "fdisk -B". Once geom opens the device, none of the fdisk/bsdlabel/boot0cfg tools work anymore. However, I wouldn't expect them to, but this also fails: # boot0cfg -B /dev/mirror/gm0 boot0cfg: /dev/mirror/gm0: Geom not found boot0cfg: /dev/mirror/gm0: ioctl DIOCSMBR: Operation not permitted So, once you enable any geom layer, you no longer have write access to the boot blocks or otherwise. This has been broken ever since GEOM was introduced in 5.0. I've been bitten many times by this in: gmirror, gvinum, gpt, glabel, etc. What's worse is that these modules don't have a way to "untaste". If you load the module, all providers get tasted and there's no way back. I got burned last week when gmirror decided to flip out and it whacked ad1. The system panicked on every boot. The only remedy was to boot a 5.5-R media (IIRC) and do "gmirror clear ad1" *before* geom_mirror was loaded. IIRC, 6.0, 6.1, and all the other 5.x media I had didn't work. What a PITA! It would be nice if the tools had a "load but don't taste" command, an "untaste" command, and a "taste" command. Until then, it all just feels so incomplete, like it was hacked together. -- Rick C. Petty From owner-freebsd-geom@FreeBSD.ORG Thu Dec 28 03:51:39 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2459116A40F for ; Thu, 28 Dec 2006 03:51:39 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id A744913C46D for ; Thu, 28 Dec 2006 03:51:38 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 98424 invoked by uid 2001); 28 Dec 2006 03:55:57 -0000 Date: Wed, 27 Dec 2006 21:55:57 -0600 From: "Rick C. Petty" To: "R. B. Riddick" Message-ID: <20061228035557.GA97647@keira.kiwi-computer.com> References: <20061227232712.GA90336@keira.kiwi-computer.com> <631303.64130.qm@web30304.mail.mud.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <631303.64130.qm@web30304.mail.mud.yahoo.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-geom@freebsd.org Subject: Re: gmirror issues (fdisk?, disklabel?, newfs?) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2006 03:51:39 -0000 On Wed, Dec 27, 2006 at 03:43:35PM -0800, R. B. Riddick wrote: > > Since I do not want to use my DSL flat-rate too much, I did not test 6.2-RC1... > But I will try R6.2, as soon as possible... Understandable. My testing has been with using csup/cvsup and daily buildworlds / installworlds. Much less bandwidth than ISOs =) > > # boot0cfg -B ad1 > > boot0cfg: /dev/ad1: Geom not found > > boot0cfg: write_mbr: /dev/ad1: No such file or directory > > > Hmm... The error message is misleading... Very. > But it is OK, that /dev/ad1 cannot be opened for writing as soon as gmirror > uses it as a consumer, because: gmirror has no chance to notice changes that go > directly to /dev/ad1, so that /dev/ad0 would stay unchanged (and so possibly > the mirror is not sync'ed and gmirror does not mention that immediately). Exactly, and I agree with this behavior. > > # boot0cfg -B /dev/mirror/gm0 > > boot0cfg: /dev/mirror/gm0: Geom not found > > boot0cfg: /dev/mirror/gm0: ioctl DIOCSMBR: Operation not permitted > > > This is strange, because: gmirror should certainly allow write access to its > devices (providers)... Sounds really strange... It is, and it's been a bug since 5.0. > Luckily someone mentioned that before R6.2... :-) It has been discussed/ignored for years. Search the lists for my name as well as others who've noticed this behavior. It's not a matter of release dates or deadlines but a matter of willing manpower. A good argument-- yet it seems manpower is quite willing to nerf other perfectly good software (*cough* vinum *cough*). In this case, I think it's more a matter of developers not knowing the depths of GEOM well.. it certainly was a barrier for me, or I would have submitted patches myself. I spent enough of my time trying to patch arla (AFS) to work with the numerous (perhaps unnecessary??) VFS changes, but those APIs keep shifting and I've given up until the APIs settle, which is never. Hopefully other people have more motivation/time than I. > Hmm... But why does it work here on my box with R6.1? No idea. (Bad) Luck? > I just used boot0cfg on my /dev/ad0, which has geom_bsd (/dev/ad0s1) and You shouldn't be able to "fdisk -B" or "boot0cfg -B" directly onto ad0 !!! Not if a GEOM provider has ad0 as a consumer! Why is the device entry even visible? (Granted, I'd rather no device entries were "hidden".) > gmirror (e.g.: /dev/ad0s1a and /dev/ad1s1a build one gmirror) on top (I could > successfully change the ticks and the default choice)... Ah, you've introduced an extra layer between the device and the mirror. You're not using gmirror on the whole device, as I am. Probably bsdlabel isn't GEOM-ready and thus it allows you to modify ad0 directly. Yet another bug. > Yup - some special cases r not handled as gracefully as they could be > handled... One of the understatements of the century. > I had another one: > gmirror out of ad1s1a and ad0s1a, where ad0s1a was rebuilding... in order to > stop rebuild I decided to remove ad1s1a which left a unusable gmirror device > and a panic (certainly after reboot, too). Unfortunately I do not remember the > kind of panic... If it was a bad memory access or just an ASSERT... But it > crashed repeatedly until I disconnected the disk that had that bad > gmirror-meta-info... :-) I certainly wouldn't expect any good behavior from doing that, but the fbsd kernel sure likes to panic a lot when there's another perfectly good option. IMO, panics should only happen whenever there's no possible way the system will ever become usable again until a reboot. For example, if you discover an ECC memory error in kernel space, or the device housing your root partition disappears (as in your example). But certainly not if your swap partition fails on pagein or trying to reload an already-loaded device. In my case, the system was working perfectly well and all of the sudden gmirror decided ad0 had the wrong quantum phase to be operating in December under a new moon, so it clobbered the metadata and immediately panicked. Successive reboots left many weird messages on the screen. I've never seen so many sequential "c"s on the screen at a time. > > It would be nice if the tools had a "load but don't taste" command, an > > "untaste" command, and a "taste" command. Until then, it all just feels > > so incomplete, like it was hacked together. > > > *sob* :-) > I would say this "untaste" command wouldnt be necessary, if the geom classes > would handle every single special case as gracefully as possible... Unfortunately with shifting APIs and error-prone programmers writing device drivers, I think all three of these commands are an absolute necessity. Otherwise, why would we need kernel debuggers at all? -- Rick C. Petty From owner-freebsd-geom@FreeBSD.ORG Thu Dec 28 06:10:00 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 159AE16A407 for ; Thu, 28 Dec 2006 06:10:00 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30307.mail.mud.yahoo.com (web30307.mail.mud.yahoo.com [209.191.69.69]) by mx1.freebsd.org (Postfix) with SMTP id B8CDC13C481 for ; Thu, 28 Dec 2006 06:09:59 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 69943 invoked by uid 60001); 28 Dec 2006 06:09:59 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=ymT7P71bCZHxnvO7+YHF4r00P6Vw34o4gh2Fexr5dvTMI4iX6SDUhU49DY3DPB6KO1Ou4qwXh70IFaiLhWV/pY5azsLu5BxXfe7+c9p96q2qB0z4HFhUMHI4Po7jwbmqupIJrJNs+qGg40z3dT4U3k27GDcnG5JjIEkXm5gQO4g=; X-YMail-OSG: 9cqjevsVM1k.wjhsWCy_zfLWbx1L6DsoVRGePQs8581dhKl0kzqchyGz3O3bYCmnWg-- Received: from [85.212.9.184] by web30307.mail.mud.yahoo.com via HTTP; Wed, 27 Dec 2006 22:09:59 PST Date: Wed, 27 Dec 2006 22:09:59 -0800 (PST) From: "R. B. Riddick" To: rick-freebsd@kiwi-computer.com In-Reply-To: <20061228035557.GA97647@keira.kiwi-computer.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <174658.68549.qm@web30307.mail.mud.yahoo.com> Cc: freebsd-geom@freebsd.org Subject: Re: gmirror issues (fdisk?, disklabel?, newfs?) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2006 06:10:00 -0000 --- "Rick C. Petty" wrote: > Ah, you've introduced an extra layer between the device and the mirror. > You're not using gmirror on the whole device, as I am. Probably bsdlabel > isn't GEOM-ready and thus it allows you to modify ad0 directly. Yet > another bug. > Hmm I think, slices (fdisk) and partitions (bsdlabel) are both GEOM classes now, so that bsdlabel is able to work with GEOM classes. Furthermore I did this gmirror/bsdlabel test with a md0 device and it worked fine for me (on R6.1) (I posted that some hours ago)... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Thu Dec 28 17:37:50 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D8CD716A47E for ; Thu, 28 Dec 2006 17:37:50 +0000 (UTC) (envelope-from vd@datamax.bg) Received: from jengal.datamax.bg (jengal.datamax.bg [82.103.104.21]) by mx1.freebsd.org (Postfix) with ESMTP id 98AE213C487 for ; Thu, 28 Dec 2006 17:37:50 +0000 (UTC) (envelope-from vd@datamax.bg) Received: from qlovarnika.bg.datamax (qlovarnika.bg.datamax [192.168.10.2]) by jengal.datamax.bg (Postfix) with SMTP id C93F8B833; Thu, 28 Dec 2006 19:18:58 +0200 (EET) Received: (nullmailer pid 11696 invoked by uid 1002); Thu, 28 Dec 2006 17:18:58 -0000 Date: Thu, 28 Dec 2006 19:18:58 +0200 From: Vasil Dimov To: freebsd-geom@freebsd.org Message-ID: <20061228171858.GA11296@qlovarnika.bg.datamax> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ew6BAiZeqk4r7MaW" Content-Disposition: inline Subject: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vd@FreeBSD.org List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2006 17:37:50 -0000 --ew6BAiZeqk4r7MaW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I wanted to do some measuring of gstripe performance and I created 17 devices each of which can read with approximately 1MiB/sec. (For the purpose I use ggate[dc] with bandwidth limitation between client and server. Any suggestions for simpler setup are welcome :-) So here are the results: Read speed is measured with "dd of=3D/dev/null bs=3D1m". It is imperfect and not very close to reality but it is simple and can easily be changed if necessary. First of all I ensure that devices do not collide when used simultaneously (numbers are in bytes/sec): % ./simple_read.sh single read: 1056381 parallel read: min: 1056164 max: 1056836 avg: 1056599 % (parallel means simultaneously reading from all 17 disks). Then I run this script I crafted. I hope the output is self-explanatory ("exp" stands for "expected" and is calculated by NUMBER_OF_DISKS * SINGLE_DISK_READ_SPEED): % ./stripe_test.sh 2 disks: 2080579 b/s (exp 2113778), avg disk load: 1040289 (98.4%) 3 disks: 3047572 b/s (exp 3170667), avg disk load: 1015857 (96.1%) 4 disks: 3970992 b/s (exp 4227556), avg disk load: 992748 (93.9%) 5 disks: 4679840 b/s (exp 5284445), avg disk load: 935968 (88.5%) 6 disks: 5460233 b/s (exp 6341334), avg disk load: 910038 (86.1%) 7 disks: 6390730 b/s (exp 7398223), avg disk load: 912961 (86.3%) 8 disks: 7654336 b/s (exp 8455112), avg disk load: 956792 (90.5%) 9 disks: 7707020 b/s (exp 9512001), avg disk load: 856335 (81.0%) 10 disks: 8188495 b/s (exp 10568890), avg disk load: 818849 (77.4%) 11 disks: 9478435 b/s (exp 11625779), avg disk load: 861675 (81.5%) 12 disks: 9457988 b/s (exp 12682668), avg disk load: 788165 (74.5%) 13 disks: 9653010 b/s (exp 13739557), avg disk load: 742539 (70.2%) 14 disks: 9649053 b/s (exp 14796446), avg disk load: 689218 (65.2%) 15 disks: 10162721 b/s (exp 15853335), avg disk load: 677514 (64.1%) 16 disks: 12659054 b/s (exp 16910224), avg disk load: 791190 (74.8%) 17 disks: 12506097 b/s (exp 17967113), avg disk load: 735652 (69.6%) % Can someone explain this? The tendency is for performace drop when increasing the number of disks in a stripe but there are some local peaks/extremums when using 8, 11 and 16 disks. Yes, I have read http://lists.freebsd.org/pipermail/freebsd-geom/2006-November/001705.html kern.geom.stripe.fast is set to 1. The scripts can be downloaded from http://people.freebsd.org/~vd/geom_test/ I intend to extend this test by: * test graid3 * measure with something other than dd(1) * measure write speed --=20 Vasil Dimov gro.DSBeerF@dv % Laugh at your problems: everybody else does. --ew6BAiZeqk4r7MaW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFFk/yCFw6SP/bBpCARAteTAJwJ2TpuDDmqaG9SQ5O0Be3aDSgS1QCg2AZV Om0G3pliqpOO8V4pTuwkBCI= =ASce -----END PGP SIGNATURE----- --ew6BAiZeqk4r7MaW-- From owner-freebsd-geom@FreeBSD.ORG Thu Dec 28 18:25:22 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 38FCF16A417 for ; Thu, 28 Dec 2006 18:25:16 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id D1D5813C475 for ; Thu, 28 Dec 2006 18:25:15 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1Gzzw3-0000il-HL for freebsd-geom@freebsd.org; Thu, 28 Dec 2006 19:24:59 +0100 Received: from 89-172-61-42.adsl.net.t-com.hr ([89.172.61.42]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 28 Dec 2006 19:24:59 +0100 Received: from ivoras by 89-172-61-42.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 28 Dec 2006 19:24:59 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Thu, 28 Dec 2006 19:24:45 +0100 Lines: 49 Message-ID: References: <20061228171858.GA11296@qlovarnika.bg.datamax> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig9A7097DADAAF2593CDBF794C" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-61-42.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: <20061228171858.GA11296@qlovarnika.bg.datamax> X-Enigmail-Version: 0.94.1.2 Sender: news Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2006 18:25:22 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig9A7097DADAAF2593CDBF794C Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Vasil Dimov wrote: > Can someone explain this? > The tendency is for performace drop when increasing the number of disks= > in a stripe but there are some local peaks/extremums when using 8, 11 > and 16 disks. I'll take a shot at this: Since maximum kernel reads are still limited to 128 KB/s, by adding more drives you're making individual requests shorter. I.e. with one drive, it gets 128 KB requests, with two, each gets 64 KB, with 16, each gets 8 KB. So network & kernel latency becomes visible. AFAIK there's unofficial (still?) GEOM_CACHE class which tries to get around this by requesting & caching 128K from each drive. Search the lists, it's mentioned somewhere. >=20 > Yes, I have read > http://lists.freebsd.org/pipermail/freebsd-geom/2006-November/001705.ht= ml >=20 > kern.geom.stripe.fast is set to 1. While you're playing with this, you could set vfs.read_max to 32 or higher and see if it helps. --------------enig9A7097DADAAF2593CDBF794C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFlAvzldnAQVacBcgRApOtAKDPqJ/HJfmf6M6jvctjitjzfkHVsgCgv59d ftyB+tS4eELnv1k02rWxpTM= =sgLO -----END PGP SIGNATURE----- --------------enig9A7097DADAAF2593CDBF794C-- From owner-freebsd-geom@FreeBSD.ORG Thu Dec 28 18:30:04 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6394316A403 for ; Thu, 28 Dec 2006 18:30:04 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 15EAC13C48E for ; Thu, 28 Dec 2006 18:30:04 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from root by ciao.gmane.org with local (Exim 4.43) id 1H000w-0001jN-6l for freebsd-geom@freebsd.org; Thu, 28 Dec 2006 19:30:02 +0100 Received: from 89-172-61-42.adsl.net.t-com.hr ([89.172.61.42]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 28 Dec 2006 19:30:02 +0100 Received: from ivoras by 89-172-61-42.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 28 Dec 2006 19:30:02 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Thu, 28 Dec 2006 19:27:50 +0100 Lines: 34 Message-ID: References: <20061228171858.GA11296@qlovarnika.bg.datamax> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigC0CDD9C5C2B97BF14E2645A8" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-61-42.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: X-Enigmail-Version: 0.94.1.2 Sender: news Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2006 18:30:04 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigC0CDD9C5C2B97BF14E2645A8 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Ivan Voras wrote: > I'll take a shot at this: Since maximum kernel reads are still limited > to 128 KB/s, by adding more drives you're making individual requests > shorter. I.e. with one drive, it gets 128 KB requests, with two, each > gets 64 KB, with 16, each gets 8 KB. So network & kernel latency become= s > visible. Btw. if I'm right, you should be able to observe this with gstat and/or iostat. If you don't see it, then it's possible I'm wrong :) --------------enigC0CDD9C5C2B97BF14E2645A8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFlAymldnAQVacBcgRAht9AKCo4K2oMISgKKOTQeUeQpNToEhhRwCglpAc hQpiTQrnYBwjCOXyG8l/oyA= =UuL+ -----END PGP SIGNATURE----- --------------enigC0CDD9C5C2B97BF14E2645A8-- From owner-freebsd-geom@FreeBSD.ORG Thu Dec 28 21:38:54 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 450AE16A40F for ; Thu, 28 Dec 2006 21:38:54 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30311.mail.mud.yahoo.com (web30311.mail.mud.yahoo.com [209.191.69.73]) by mx1.freebsd.org (Postfix) with SMTP id ECA5913C46D for ; Thu, 28 Dec 2006 21:38:53 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 68160 invoked by uid 60001); 28 Dec 2006 21:38:53 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=cfqKVktgvnu8J5dXDZAu8+xEHZ5O9yaEx9vOnNojlDNIaSMULUTOq0oLLCFLonLk13I/b3dWlh2Ukw8o8+d5RkUVz2+4K5t8ckt2/TCSSNXrycurRIOAmpmj2srfzk94hCawddDQ9AQkQMKgdSysUqlPr4G0BopFbMJFmpRblkY=; X-YMail-OSG: z2oRZgYVM1lJ.2twD_aq1SEQShvBFl5bKlgD70kfkjUtNQBTtHJVOAV6jsNpGGQArsAERGWkqp_gkCzEs1gI4kmVCa7QNpwLKrbruOwv4bf5fAYdbdIIQdymUYLKUGqD2Fuu7xdpZLCwL8pC4J0Q3S5yO.c1XJl36g-- Received: from [85.212.9.184] by web30311.mail.mud.yahoo.com via HTTP; Thu, 28 Dec 2006 13:38:52 PST Date: Thu, 28 Dec 2006 13:38:52 -0800 (PST) From: "R. B. Riddick" To: Ivan Voras , freebsd-geom@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <332760.68142.qm@web30311.mail.mud.yahoo.com> Cc: Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2006 21:38:54 -0000 I found this static intmax_t default_stripesize = 4096; in src/sbin/geom/class/stripe/geom_stripe.c The stripe-fast feature will help less and less (just as Ivan described it). A solution would be to increase stripe size to something bigger (like 64KB), so that always two drives r busy (128KB/64KB), but then u would not see speeds higher than 2 times single-drive-speed, because dd does not read ahead on a raw device (UFS would possibly). You could try your own "dd", that issues 20 read requests nearly at the same time, so that this 128KB limitations does not have an impact... Or u could try raidtest (from ports), which can already do multiple simultaneous requests (but random read and not sequential read; but ur read performance is not limited by the disk but by ipfw, IIRC)... gcache certainly would do some read-ahead, too... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 12:05:21 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5816716A512; Fri, 29 Dec 2006 12:05:21 +0000 (UTC) (envelope-from vd@datamax.bg) Received: from jengal.datamax.bg (jengal.datamax.bg [82.103.104.21]) by mx1.freebsd.org (Postfix) with ESMTP id 26A2713C48D; Fri, 29 Dec 2006 12:05:19 +0000 (UTC) (envelope-from vd@datamax.bg) Received: from qlovarnika.bg.datamax (qlovarnika.bg.datamax [192.168.10.2]) by jengal.datamax.bg (Postfix) with SMTP id 0E00DB84C; Fri, 29 Dec 2006 14:05:18 +0200 (EET) Received: (nullmailer pid 15882 invoked by uid 1002); Fri, 29 Dec 2006 12:05:18 -0000 Date: Fri, 29 Dec 2006 14:05:18 +0200 From: Vasil Dimov To: freebsd-geom@freebsd.org Message-ID: <20061229120517.GA12877@qlovarnika.bg.datamax> References: <20061228171858.GA11296@qlovarnika.bg.datamax> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tKW2IUtsqtDRztdT" Content-Disposition: inline In-Reply-To: <20061228171858.GA11296@qlovarnika.bg.datamax> Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vd@FreeBSD.org List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 12:05:21 -0000 --tKW2IUtsqtDRztdT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, Thank you very much for your answers! Here is what further tests showed: First of all I switched from dd to raidtest. I had to tune raidtest in order to use it, see ports/107311. I am testing only reading with 8 concurrent processes (raidtest test -n 8). 2 disks: 2103646 b/s (exp 2107584), avg disk load: 1051823 ( 99.8%) 3 disks: 3134534 b/s (exp 3161376), avg disk load: 1044844 ( 99.1%) 4 disks: 4153974 b/s (exp 4215168), avg disk load: 1038493 ( 98.5%) 5 disks: 5199917 b/s (exp 5268960), avg disk load: 1039983 ( 98.6%) 6 disks: 6141678 b/s (exp 6322752), avg disk load: 1023613 ( 97.1%) 7 disks: 7193116 b/s (exp 7376544), avg disk load: 1027588 ( 97.5%) 8 disks: 8219609 b/s (exp 8430336), avg disk load: 1027451 ( 97.5%) 9 disks: 9080762 b/s (exp 9484128), avg disk load: 1008973 ( 95.7%) 10 disks: 10241349 b/s (exp 10537920), avg disk load: 1024134 ( 97.1%) 11 disks: 11077983 b/s (exp 11591712), avg disk load: 1007089 ( 95.5%) 12 disks: 11851009 b/s (exp 12645504), avg disk load: 987584 ( 93.7%) 13 disks: 12663548 b/s (exp 13699296), avg disk load: 974119 ( 92.4%) 14 disks: 13821213 b/s (exp 14753088), avg disk load: 987229 ( 93.6%) 15 disks: 14283895 b/s (exp 15806880), avg disk load: 952259 ( 90.3%) 16 disks: 15057168 b/s (exp 16860672), avg disk load: 941073 ( 89.3%) 17 disks: 16171889 b/s (exp 17914464), avg disk load: 951287 ( 90.2%) It shows the same tendency as with dd(1). Changing vfs.read_max from 8 to 32 does not produce different results. This are gstat screen shots during the test: 1 drive: dT: 0.501s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 8 12 12 1003 548.1 0 0 0.0 96.6| ggate100 8 drives: dT: 0.501s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 6 116 116 1028 63.4 0 0 0.0 97.6| ggate100 2 118 118 1080 50.6 0 0 0.0 102.0| ggate101 4 114 114 1042 45.9 0 0 0.0 100.8| ggate102 4 110 110 982 39.3 0 0 0.0 96.0| ggate103 4 116 116 1027 43.7 0 0 0.0 98.6| ggate104 6 116 116 1056 58.4 0 0 0.0 101.6| ggate105 8 124 124 1029 60.0 0 0 0.0 98.0| ggate107 7 122 122 1051 61.9 0 0 0.0 106.6| ggate106 8 130 130 8230 62.8 0 0 0.0 99.6| stripe/stest 17 drives: dT: 0.563s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 5 185 178 848 18.9 0 0 0.0 88.8| ggate100 4 183 176 831 19.8 0 0 0.0 87.9| ggate101 4 181 174 822 18.8 0 0 0.0 87.0| ggate102 4 165 165 801 17.7 0 0 0.0 86.6| ggate103 2 167 167 820 17.3 0 0 0.0 85.0| ggate104 1 176 176 840 18.8 0 0 0.0 86.4| ggate105 4 167 167 812 23.5 0 0 0.0 85.5| ggate107 3 167 167 802 19.9 0 0 0.0 85.9| ggate108 1 172 172 841 19.9 0 0 0.0 86.5| ggate109 2 170 170 852 28.5 0 0 0.0 87.4| ggate110 4 172 172 847 21.4 0 0 0.0 87.1| ggate111 4 174 174 838 22.3 0 0 0.0 87.0| ggate112 5 163 163 789 16.2 0 0 0.0 86.8| ggate113 4 162 162 802 15.7 0 0 0.0 86.4| ggate114 3 162 162 823 17.0 0 0 0.0 86.2| ggate115 3 170 170 825 18.2 0 0 0.0 86.6| ggate116 5 169 169 810 25.7 0 0 0.0 86.1| ggate106 8 245 238 13573 27.5 0 0 0.0 90.1| stripe/stest I have not tried geom_cache... --=20 Vasil Dimov gro.DSBeerF@dv % If the code and the comments disagree, then both are probably wrong. -- Norm Schryer --tKW2IUtsqtDRztdT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFFlQR9Fw6SP/bBpCARAoBIAJ4srn9bRpyLZyy8lJxgAIqJUinDBQCg2o07 1qCl6ArTT4kcuA014CGGZ2o= =oSEe -----END PGP SIGNATURE----- --tKW2IUtsqtDRztdT-- From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 12:55:56 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 294E716A651 for ; Fri, 29 Dec 2006 12:55:56 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30305.mail.mud.yahoo.com (web30305.mail.mud.yahoo.com [209.191.69.67]) by mx1.freebsd.org (Postfix) with SMTP id 844D113C4C2 for ; Fri, 29 Dec 2006 12:55:45 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 12937 invoked by uid 60001); 29 Dec 2006 12:55:45 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=i3SnTYMcWWm1ANXiqNeaVISafadQX/vTgH7UwQA9/MMUagGy3hLC2B+/iBYXql3f2OEZo3oTqHKBmudLE9VEUIGEmlA3SN+eSnJ1fTvEq1Je0RNCN+KNb6LI0qRMlx8hfd318avoer7c0mre1TJUmZSsgTyqOLnqOspUeIJ2wsI= ; Message-ID: <20061229125545.12935.qmail@web30305.mail.mud.yahoo.com> X-YMail-OSG: 1.Jx2qMVM1mTzola_PPbxg4eVAsdShJOxu8KtjkiQTOi8Cba9LLYAj0y.WU0tk6IUJoVBRovC.BNmhr5.hTWPPVtSjJFJvg8z9pwDcfy7U5cJm9ml29lVfmFK9sMXKQe8i9.pkrVuFG8vDw- Received: from [85.212.8.201] by web30305.mail.mud.yahoo.com via HTTP; Fri, 29 Dec 2006 04:55:44 PST Date: Fri, 29 Dec 2006 04:55:44 -0800 (PST) From: "R. B. Riddick" To: vd@FreeBSD.org, freebsd-geom@freebsd.org In-Reply-To: <20061229120517.GA12877@qlovarnika.bg.datamax> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 12:55:56 -0000 --- Vasil Dimov wrote: > Here is what further tests showed: > > First of all I switched from dd to raidtest. I had to tune raidtest in > order to use it, see ports/107311. I am testing only reading with 8 > concurrent processes (raidtest test -n 8). > Interesting... But the degradation is much less now... Not 25% (or what was it) but just 10%... Did u try a different stripe size (-s 65536) with more concurrency (-n 20), too? -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 14:14:51 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1F77B16A407 for ; Fri, 29 Dec 2006 14:14:51 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30304.mail.mud.yahoo.com (web30304.mail.mud.yahoo.com [209.191.69.66]) by mx1.freebsd.org (Postfix) with SMTP id C36FD13C461 for ; Fri, 29 Dec 2006 14:14:50 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 89681 invoked by uid 60001); 29 Dec 2006 14:14:50 -0000 Message-ID: <20061229141450.89679.qmail@web30304.mail.mud.yahoo.com> DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=rQzv1YEYj+flRvwkjgiLxuCjMmTFNik6o+a1W1dF84cSHCt8Hl4fsrzr6QWGKDCt3E+c9uLBfNwqgPkSQSuMurGztqPoHQGv38BfFfcLiYUreJfSUf4fgpSeUP7agMYfAu6DjgP6INquVqfvDrh7s2k2dCjwlDmd571ixoXo2A8=; X-YMail-OSG: kEZId_cVM1kUckp_fWsrW7sl8g0lY3yrgXpsF7p_I0IhSw08de4uIrTX.ThaZA_GGQ-- Received: from [85.212.8.201] by web30304.mail.mud.yahoo.com via HTTP; Fri, 29 Dec 2006 06:14:50 PST Date: Fri, 29 Dec 2006 06:14:50 -0800 (PST) From: "R. B. Riddick" To: Christian Laursen MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-geom@freebsd.org Subject: Re: geom_mirror delayed synchronizing question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 14:14:51 -0000 --- Christian Laursen wrote: > There are plenty of hacks that will do what I want but none of them > are elegant. > I changed now my graid5 quite elegantly, so that it waits 0.1 seconds (100 milli seconds; 100000 micro secs) after a read request, before it issues the next internal verify-request. This nice-time is configurable via kern.geom.raid5.veri_nice (value is in ms and must be in this range: 0..999 (or else it is ignored)). A graid5 out of 2 disks behaves like gmirror with 2 disks and balance read-strategy. But graid5 is still experimental (not so many stress tests) and if install instructions are not so easy, because u have to go into /sys/modules/geom/geom_raid5 and /usr/src/sbin/geom/class/raid5 and do a "make && make install" there)... U can download the sources here: http://home.tiscali.de/cmdr_faako/geom_raid5.tbz -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 14:17:09 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CC52A16A415 for ; Fri, 29 Dec 2006 14:17:09 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 5B84E13C428 for ; Fri, 29 Dec 2006 14:17:09 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1H0IXX-0005uW-7m for freebsd-geom@freebsd.org; Fri, 29 Dec 2006 15:16:55 +0100 Received: from 89-172-40-174.adsl.net.t-com.hr ([89.172.40.174]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 29 Dec 2006 15:16:55 +0100 Received: from ivoras by 89-172-40-174.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 29 Dec 2006 15:16:55 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Fri, 29 Dec 2006 15:16:36 +0100 Lines: 50 Message-ID: References: <20061228171858.GA11296@qlovarnika.bg.datamax> <20061229120517.GA12877@qlovarnika.bg.datamax> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigBBA288B29013A069B0246382" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-40-174.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: <20061229120517.GA12877@qlovarnika.bg.datamax> X-Enigmail-Version: 0.94.1.2 Sender: news Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 14:17:09 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigBBA288B29013A069B0246382 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Vasil Dimov wrote: > Here is what further tests showed: > This are gstat screen shots during the test: >=20 > 1 drive: > dT: 0.501s w: 0.500s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name > 8 12 12 1003 548.1 0 0 0.0 96.6| ggate100= > 8 drives: > dT: 0.501s w: 0.500s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name > 6 116 116 1028 63.4 0 0 0.0 97.6| ggate100= > 17 drives: > dT: 0.563s w: 0.500s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name > 5 185 178 848 18.9 0 0 0.0 88.8| ggate100= Ok, so ops/s is increasing but it's not conclusive. Does anyone know how to get KB/t stats for synthetic geom devices? --------------enigBBA288B29013A069B0246382 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFlSNKldnAQVacBcgRAoqfAKCpIgxW/5r6cnk48aTp9X8TCpOe3wCgoqz7 YPA8voRLRr4vu9e4eZVWyfA= =SWRB -----END PGP SIGNATURE----- --------------enigBBA288B29013A069B0246382-- From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 14:24:02 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id EB5E216A40F; Fri, 29 Dec 2006 14:24:02 +0000 (UTC) (envelope-from vd@datamax.bg) Received: from jengal.datamax.bg (jengal.datamax.bg [82.103.104.21]) by mx1.freebsd.org (Postfix) with ESMTP id 52C2A13C46C; Fri, 29 Dec 2006 14:24:02 +0000 (UTC) (envelope-from vd@datamax.bg) Received: from qlovarnika.bg.datamax (qlovarnika.bg.datamax [192.168.10.2]) by jengal.datamax.bg (Postfix) with SMTP id 06639B833; Fri, 29 Dec 2006 16:24:01 +0200 (EET) Received: (nullmailer pid 24346 invoked by uid 1002); Fri, 29 Dec 2006 14:24:00 -0000 Date: Fri, 29 Dec 2006 16:24:00 +0200 From: Vasil Dimov To: "R\. B\. Riddick" Message-ID: <20061229142400.GA17217@qlovarnika.bg.datamax> References: <20061229120517.GA12877@qlovarnika.bg.datamax> <20061229125545.12935.qmail@web30305.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YiEDa0DAkWCtVeE4" Content-Disposition: inline In-Reply-To: <20061229125545.12935.qmail@web30305.mail.mud.yahoo.com> Cc: freebsd-geom@freebsd.org Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vd@FreeBSD.org List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 14:24:03 -0000 --YiEDa0DAkWCtVeE4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 29, 2006 at 04:55:44AM -0800, R. B. Riddick wrote: > --- Vasil Dimov wrote: > > Here is what further tests showed: > >=20 > > First of all I switched from dd to raidtest. I had to tune raidtest in > > order to use it, see ports/107311. I am testing only reading with 8 > > concurrent processes (raidtest test -n 8). > > > Interesting... > But the degradation is much less now... Not 25% (or what was it) but just > 10%... Yes, that seems to be the case. And the oscillations are less expressive. > Did u try a different stripe size (-s 65536) with more concurrency (-n 20= ), > too? Here it is: stripe size: 4096, read processes: 8 (this was my initial test with raidtest) 2 disks: 2109917 b/s (exp 2110906), avg disk load: 1054958 ( 99.9%) 3 disks: 3139464 b/s (exp 3166359), avg disk load: 1046488 ( 99.1%) 4 disks: 4194968 b/s (exp 4221812), avg disk load: 1048742 ( 99.3%) 5 disks: 5146921 b/s (exp 5277265), avg disk load: 1029384 ( 97.5%) 6 disks: 6226682 b/s (exp 6332718), avg disk load: 1037780 ( 98.3%) 7 disks: 7187536 b/s (exp 7388171), avg disk load: 1026790 ( 97.2%) 8 disks: 8145568 b/s (exp 8443624), avg disk load: 1018196 ( 96.4%) 9 disks: 9179785 b/s (exp 9499077), avg disk load: 1019976 ( 96.6%) 10 disks: 10065401 b/s (exp 10554530), avg disk load: 1006540 ( 95.3%) 11 disks: 11006498 b/s (exp 11609983), avg disk load: 1000590 ( 94.8%) 12 disks: 11878842 b/s (exp 12665436), avg disk load: 989903 ( 93.7%) 13 disks: 12905593 b/s (exp 13720889), avg disk load: 992737 ( 94.0%) 14 disks: 13670094 b/s (exp 14776342), avg disk load: 976435 ( 92.5%) 15 disks: 14474347 b/s (exp 15831795), avg disk load: 964956 ( 91.4%) 16 disks: 15474211 b/s (exp 16887248), avg disk load: 967138 ( 91.6%) 17 disks: 16021676 b/s (exp 17942701), avg disk load: 942451 ( 89.2%) stripe size: 64*1024, read processes: 8 2 disks: 2037981 b/s (exp 2112580), avg disk load: 1018990 ( 96.4%) 3 disks: 2792740 b/s (exp 3168870), avg disk load: 930913 ( 88.1%) 4 disks: 3686512 b/s (exp 4225160), avg disk load: 921628 ( 87.2%) 5 disks: 4133447 b/s (exp 5281450), avg disk load: 826689 ( 78.2%) 6 disks: 4370325 b/s (exp 6337740), avg disk load: 728387 ( 68.9%) 7 disks: 5241010 b/s (exp 7394030), avg disk load: 748715 ( 70.8%) 8 disks: 5249938 b/s (exp 8450320), avg disk load: 656242 ( 62.1%) 9 disks: 5458054 b/s (exp 9506610), avg disk load: 606450 ( 57.4%) 10 disks: 6381395 b/s (exp 10562900), avg disk load: 638139 ( 60.4%) 11 disks: 6409845 b/s (exp 11619190), avg disk load: 582713 ( 55.1%) 12 disks: 6539793 b/s (exp 12675480), avg disk load: 544982 ( 51.5%) 13 disks: 7261850 b/s (exp 13731770), avg disk load: 558603 ( 52.8%) 14 disks: 6814684 b/s (exp 14788060), avg disk load: 486763 ( 46.0%) 15 disks: 7535144 b/s (exp 15844350), avg disk load: 502342 ( 47.5%) 16 disks: 6971418 b/s (exp 16900640), avg disk load: 435713 ( 41.2%) 17 disks: 7880572 b/s (exp 17956930), avg disk load: 463563 ( 43.8%) stripe size: 4096, read processes: 20 2 disks: 2107385 b/s (exp 2112176), avg disk load: 1053692 ( 99.7%) 3 disks: 3143703 b/s (exp 3168264), avg disk load: 1047901 ( 99.2%) 4 disks: 4206919 b/s (exp 4224352), avg disk load: 1051729 ( 99.5%) 5 disks: 5167176 b/s (exp 5280440), avg disk load: 1033435 ( 97.8%) 6 disks: 6262062 b/s (exp 6336528), avg disk load: 1043677 ( 98.8%) 7 disks: 7271021 b/s (exp 7392616), avg disk load: 1038717 ( 98.3%) 8 disks: 8260114 b/s (exp 8448704), avg disk load: 1032514 ( 97.7%) 9 disks: 9238876 b/s (exp 9504792), avg disk load: 1026541 ( 97.2%) 10 disks: 10147589 b/s (exp 10560880), avg disk load: 1014758 ( 96.0%) 11 disks: 11063027 b/s (exp 11616968), avg disk load: 1005729 ( 95.2%) 12 disks: 12298836 b/s (exp 12673056), avg disk load: 1024903 ( 97.0%) 13 disks: 12893838 b/s (exp 13729144), avg disk load: 991833 ( 93.9%) 14 disks: 13927065 b/s (exp 14785232), avg disk load: 994790 ( 94.1%) 15 disks: 14851486 b/s (exp 15841320), avg disk load: 990099 ( 93.7%) 16 disks: 15630142 b/s (exp 16897408), avg disk load: 976883 ( 92.5%) 17 disks: 16685858 b/s (exp 17953496), avg disk load: 981521 ( 92.9%) stripe size: 64*1024, read processes: 20 2 disks: 2089750 b/s (exp 2111630), avg disk load: 1044875 ( 98.9%) 3 disks: 3081869 b/s (exp 3167445), avg disk load: 1027289 ( 97.2%) 4 disks: 3866341 b/s (exp 4223260), avg disk load: 966585 ( 91.5%) 5 disks: 4541626 b/s (exp 5279075), avg disk load: 908325 ( 86.0%) 6 disks: 5529365 b/s (exp 6334890), avg disk load: 921560 ( 87.2%) 7 disks: 6311299 b/s (exp 7390705), avg disk load: 901614 ( 85.3%) 8 disks: 6363864 b/s (exp 8446520), avg disk load: 795483 ( 75.3%) 9 disks: 6934731 b/s (exp 9502335), avg disk load: 770525 ( 72.9%) 10 disks: 7622329 b/s (exp 10558150), avg disk load: 762232 ( 72.1%) 11 disks: 7806745 b/s (exp 11613965), avg disk load: 709704 ( 67.2%) 12 disks: 8921822 b/s (exp 12669780), avg disk load: 743485 ( 70.4%) 13 disks: 9380174 b/s (exp 13725595), avg disk load: 721551 ( 68.3%) 14 disks: 9453859 b/s (exp 14781410), avg disk load: 675275 ( 63.9%) 15 disks: 10319599 b/s (exp 15837225), avg disk load: 687973 ( 65.1%) 16 disks: 10074550 b/s (exp 16893040), avg disk load: 629659 ( 59.6%) 17 disks: 10527268 b/s (exp 17948855), avg disk load: 619251 ( 58.6%) In summary increasing the stripe size results in performance drop while increasing the number of read processes results in performance raise. (I also tested with stripe size 128*1024). My guess is that this is because raidtest often generates read requests with size less than the stripe size and then it reads from just one disk, or at most from several ones. It rarely (never?) generates a request that has read size 17*stripesize or greater so that all disks are used in parallel. Btw I found strange behavior in graid3, I will post this in a separate thread... --=20 Vasil Dimov gro.DSBeerF@dv % Death liked black. It went with anything. It went with everything, sooner or later. -- (Terry Pratchett, Soul Music) --YiEDa0DAkWCtVeE4 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFFlSUAFw6SP/bBpCARAqGjAJ9nR3gsGoTGQG1E3dwIyCA3HOkusgCgkC9N HicWOKEA6W5QykBT7b01Hmw= =k2ig -----END PGP SIGNATURE----- --YiEDa0DAkWCtVeE4-- From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 14:26:39 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CDBCB16A407 for ; Fri, 29 Dec 2006 14:26:39 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30315.mail.mud.yahoo.com (web30315.mail.mud.yahoo.com [209.191.69.77]) by mx1.freebsd.org (Postfix) with SMTP id 7E90213C442 for ; Fri, 29 Dec 2006 14:26:39 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 45846 invoked by uid 60001); 29 Dec 2006 14:26:38 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=I0DnGD5sRm5VPUYYslI3JONnqDR864/Q97g+cQeYe2AVUVq00rwL6qCvtuQxS/pihbxJkeIYTdwOOW8FXwnq1MlwVEEgTwYU02dXu3tJTUwy7crXxvH4hHRd0uoCEgaD8DBDrVuWP3RmMtDixuEfoTlwVvu5Uk7I/IWGab3DsxM= ; Message-ID: <20061229142638.45844.qmail@web30315.mail.mud.yahoo.com> X-YMail-OSG: Y_xEjMkVM1lgKN_ImfDKldWKS8oatDAC8ACC2.hGqneqeRYTQVDNexhbD9EKWHVRYln8y7JlF67ew3fo5N_WNnobthggJ6g9inc.vPBRwZBQ7U4_Y0AHnsBrszeGPbYV8lSkJzAqMNbFoM0d7Tp1R36TM1AvNE0v Received: from [85.212.8.201] by web30315.mail.mud.yahoo.com via HTTP; Fri, 29 Dec 2006 06:26:38 PST Date: Fri, 29 Dec 2006 06:26:38 -0800 (PST) From: "R. B. Riddick" To: Ivan Voras , freebsd-geom@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 14:26:39 -0000 --- Ivan Voras wrote: > Vasil Dimov wrote: > > 17 drives: > > dT: 0.563s w: 0.500s > > L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name > > 5 185 178 848 18.9 0 0 0.0 88.8| ggate100 > > Ok, so ops/s is increasing but it's not conclusive. Does anyone know how > to get KB/t stats for synthetic geom devices? > I would say: transaction size = (volume per second) / (operations per second) E. g.: 848KBps/(178r/s) = 4.76KB/op or 1028KBps/(116r/s) = 8.86KB/op -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 14:30:15 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0398716A4A0 for ; Fri, 29 Dec 2006 14:30:15 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30306.mail.mud.yahoo.com (web30306.mail.mud.yahoo.com [209.191.69.68]) by mx1.freebsd.org (Postfix) with SMTP id 9D1DB13C46C for ; Fri, 29 Dec 2006 14:30:14 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 94561 invoked by uid 60001); 29 Dec 2006 14:30:14 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=xVgPTkwTE+b3sIZrjfB9WmFjMoMbwfj5chvbwVBckYrmwEc6BzupGQX8m0erM72+wLpxY3nq9I/88G5rLzuLMm2wbooVUBnj51H/9RnRCAQGPcIjSswHkew26719i6G4a0Q+KOpl1DgYxPTbUbVh0ArVBMVlcv6oCq0Rnw6nxlA= ; Message-ID: <20061229143014.94559.qmail@web30306.mail.mud.yahoo.com> X-YMail-OSG: u96t7qIVM1kHVcGORHSSzf.64c4voaj2R8K8_Pll82sGrpOw4k8XAUHXMV8fZl4zCEzYthW2YKnep3WIZsnLMMgrQhd2uGwE4nwwqtPEZY68qhtj3wUJAwixSEoxKTcUQCvTJd16DcqtmnI- Received: from [85.212.8.201] by web30306.mail.mud.yahoo.com via HTTP; Fri, 29 Dec 2006 06:30:14 PST Date: Fri, 29 Dec 2006 06:30:14 -0800 (PST) From: "R. B. Riddick" To: vd@FreeBSD.org In-Reply-To: <20061229142400.GA17217@qlovarnika.bg.datamax> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-geom@freebsd.org Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 14:30:15 -0000 --- Vasil Dimov wrote: > On Fri, Dec 29, 2006 at 04:55:44AM -0800, R. B. Riddick wrote: > In summary increasing the stripe size results in performance drop while > increasing the number of read processes results in performance raise. > (I also tested with stripe size 128*1024). My guess is that this is > because raidtest often generates read requests with size less than the > stripe size and then it reads from just one disk, or at most from > several ones. It rarely (never?) generates a request that has read size > 17*stripesize or greater so that all disks are used in parallel. > That is surprising for me... Small request size caused by raidtest shouldnt decrease the performance so much, because u used 20 concurrent processes... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 14:40:00 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BB19E16A416 for ; Fri, 29 Dec 2006 14:40:00 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 4081013C459 for ; Fri, 29 Dec 2006 14:40:00 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1H0Itk-0001kC-6o for freebsd-geom@freebsd.org; Fri, 29 Dec 2006 15:39:52 +0100 Received: from 89-172-40-174.adsl.net.t-com.hr ([89.172.40.174]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 29 Dec 2006 15:39:52 +0100 Received: from ivoras by 89-172-40-174.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 29 Dec 2006 15:39:52 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Fri, 29 Dec 2006 15:39:33 +0100 Lines: 48 Message-ID: References: <20061229142638.45844.qmail@web30315.mail.mud.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigB20C309C7C634A324D78B72B" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-40-174.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: <20061229142638.45844.qmail@web30315.mail.mud.yahoo.com> X-Enigmail-Version: 0.94.1.2 Sender: news Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 14:40:00 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB20C309C7C634A324D78B72B Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable R. B. Riddick wrote: > --- Ivan Voras wrote: >> Vasil Dimov wrote: >>> 17 drives: >>> dT: 0.563s w: 0.500s >>> L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name >>> 5 185 178 848 18.9 0 0 0.0 88.8| ggate1= 00 >> Ok, so ops/s is increasing but it's not conclusive. Does anyone know h= ow >> to get KB/t stats for synthetic geom devices? >> > I would say: > transaction size =3D (volume per second) / (operations per second) > E. g.: > 848KBps/(178r/s) =3D 4.76KB/op > or > 1028KBps/(116r/s) =3D 8.86KB/op Well "doh", nowadays I don't know how to calculate anymore :) Hmmm, it's strange - it's 2x lower than it should be. For 8 drives, each should do 16 KB/s but they do 8 KB/s. Maybe something's chopping the requests at 64 KB each? --------------enigB20C309C7C634A324D78B72B Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFlSirldnAQVacBcgRApn/AJ4wsi/Ot3cfgXAurLI9b+rx0D8XFACbBG12 O5D9ZDdEAq7nfGKsOIpnQAE= =4zvK -----END PGP SIGNATURE----- --------------enigB20C309C7C634A324D78B72B-- From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 14:47:19 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 48BDF16A403 for ; Fri, 29 Dec 2006 14:47:19 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30304.mail.mud.yahoo.com (web30304.mail.mud.yahoo.com [209.191.69.66]) by mx1.freebsd.org (Postfix) with SMTP id EE52913C44B for ; Fri, 29 Dec 2006 14:47:18 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 315 invoked by uid 60001); 29 Dec 2006 14:47:18 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=h8kHjQ5PNCdVS+xEUEViR7M7OLw57F6Xjzt6IJrXQsmWDwkcX5Q4pIyZjgpMDDQWvAem3eNu5JwquOhActkUzLDSra2sbax7YIzMZ9gwFg3S6hXTfTcpqtqUSSJmJzlVCpUpq3vnOMsnqd4TQ0xrcc7o/8iTAVn8LL12fsbWnjg=; X-YMail-OSG: d5nbDHcVM1lPA_REds_myCcJPbuZR1FfuExxeMCUYoHFOIdquJ63NWaglmgQdBFbH_SkzJGcewJQ5nPdhbfeYZgsPu1a0xE5hEFd7I_PayzyPNgIjt8FBWobWyQqszC34ejhFw_kl2rDQ_gHKxi2EeyQK2YSiSYL Received: from [85.212.8.201] by web30304.mail.mud.yahoo.com via HTTP; Fri, 29 Dec 2006 06:47:18 PST Date: Fri, 29 Dec 2006 06:47:18 -0800 (PST) From: "R. B. Riddick" To: Ivan Voras , freebsd-geom@freebsd.org In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <496105.98420.qm@web30304.mail.mud.yahoo.com> Cc: Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 14:47:19 -0000 --- Ivan Voras wrote: > Hmmm, it's strange - it's 2x lower than it should be. For 8 drives, each > should do 16 KB/s but they do 8 KB/s. Maybe something's chopping the > requests at 64 KB each? > Yup! It is the different test-tool: raidtest (from ports) It uses random access with various request sizes... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 14:56:59 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 496DB16A40F; Fri, 29 Dec 2006 14:56:59 +0000 (UTC) (envelope-from vd@datamax.bg) Received: from jengal.datamax.bg (jengal.datamax.bg [82.103.104.21]) by mx1.freebsd.org (Postfix) with ESMTP id 8BC8013C45B; Fri, 29 Dec 2006 14:56:58 +0000 (UTC) (envelope-from vd@datamax.bg) Received: from qlovarnika.bg.datamax (qlovarnika.bg.datamax [192.168.10.2]) by jengal.datamax.bg (Postfix) with SMTP id BB4CDB833; Fri, 29 Dec 2006 16:56:57 +0200 (EET) Received: (nullmailer pid 24742 invoked by uid 1002); Fri, 29 Dec 2006 14:56:57 -0000 Date: Fri, 29 Dec 2006 16:56:57 +0200 From: Vasil Dimov To: freebsd-geom@freebsd.org Message-ID: <20061229145657.GA24409@qlovarnika.bg.datamax> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LZvS9be/3tNcYl/X" Content-Disposition: inline Subject: graid3 device gives "Invalid argument" when requested to read >=128kb X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vd@FreeBSD.org List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 14:56:59 -0000 --LZvS9be/3tNcYl/X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I managed to create a graid3 array which returns "Invalid argument" when I attempt to read more than 128k with a single read(2) call. Here is the test program: --- graid3_test.c begins here --- #include #include #include #include #include #include #include #include int main(int argc, char **argv) { char buf[1048576]; int fd; ssize_t ret; assert((fd =3D open("/dev/raid3/r3test", O_RDONLY)) !=3D -1); if ((ret =3D read(fd, buf, 8192*16)) =3D=3D -1) printf("%s\n", strerror(errno)); else printf("ret: %zd\n", ret); close(fd); return 0; } /* EOF */ --- graid3_test.c ends here --- # ./graid3_test Invalid argument If I change 8192*16 to 8192*15 it works fine: # ./graid3_test ret: 122880 This is what /dev/raid3/r3test looks like (notice the sector size is 8192, that's why my test program is using multiples of 8192): # graid3 list Geom name: r3test State: COMPLETE Components: 17 Flags: NONE GenID: 0 SyncID: 1 ID: 4285731808 Zone64kFailed: 0 Zone64kRequested: 0 Zone16kFailed: 0 Zone16kRequested: 32 Zone4kFailed: 0 Zone4kRequested: 1280 Providers: 1. Name: raid3/r3test Mediasize: 536862720 (512M) Sectorsize: 8192 Mode: r0w0e0 Consumers: 1. Name: ggate100 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 0 Type: DATA 2. Name: ggate101 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 1 Type: DATA 3. Name: ggate102 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 2 Type: DATA 4. Name: ggate103 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 3 Type: DATA 5. Name: ggate104 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 4 Type: DATA 6. Name: ggate105 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 5 Type: DATA 7. Name: ggate106 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 6 Type: DATA 8. Name: ggate107 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 7 Type: DATA 9. Name: ggate108 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 8 Type: DATA 10. Name: ggate109 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 9 Type: DATA 11. Name: ggate110 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 10 Type: DATA 12. Name: ggate111 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 11 Type: DATA 13. Name: ggate112 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 12 Type: DATA 14. Name: ggate113 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 13 Type: DATA 15. Name: ggate114 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 14 Type: DATA 16. Name: ggate115 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 15 Type: DATA 17. Name: ggate116 Mediasize: 33554432 (32M) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Flags: NONE GenID: 0 SyncID: 1 Number: 16 Type: PARITY No such problems occur if the raid3 array is created with 9 (not 17) disks. Cheerz! --=20 Vasil Dimov gro.DSBeerF@dv % Never be afraid to tell the world who you are. -- Anonymous --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFFlSy5Fw6SP/bBpCARAvRRAJwMb+iQf8PnBTdc6ssw04bV1s/NpQCgteyn w/vWNIkADDp9H1ADRl9wyJk= =FNx5 -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X-- From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 15:58:44 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F1B9A16A415; Fri, 29 Dec 2006 15:58:44 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 94AD613C441; Fri, 29 Dec 2006 15:58:44 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 94B9848809; Fri, 29 Dec 2006 16:28:29 +0100 (CET) Received: from localhost (dkg70.neoplus.adsl.tpnet.pl [83.24.10.70]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id D08A848802; Fri, 29 Dec 2006 16:28:24 +0100 (CET) Date: Fri, 29 Dec 2006 16:27:59 +0100 From: Pawel Jakub Dawidek To: Vasil Dimov Message-ID: <20061229152759.GA2375@garage.freebsd.pl> References: <20061229145657.GA24409@qlovarnika.bg.datamax> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="/04w6evG8XlLl3ft" Content-Disposition: inline In-Reply-To: <20061229145657.GA24409@qlovarnika.bg.datamax> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: ** X-Spam-Status: No, score=2.6 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL,RCVD_IN_XBL autolearn=no version=3.0.4 Cc: freebsd-geom@freebsd.org Subject: Re: graid3 device gives "Invalid argument" when requested to read >=128kb X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 15:58:45 -0000 --/04w6evG8XlLl3ft Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 29, 2006 at 04:56:57PM +0200, Vasil Dimov wrote: > Hi, >=20 > I managed to create a graid3 array which returns "Invalid argument" when > I attempt to read more than 128k with a single read(2) call. >=20 > Here is the test program: [...] > This is what /dev/raid3/r3test looks like > (notice the sector size is 8192, that's why my test program is using > multiples of 8192): [...] Using sectorsize larger than page size won't work with UFS anyway. Not sure if this is graid3(8) issue or some general issue. Can you try create provider with 8kB sectorsize using gnop(8) and retry your test? # gnop create -S 8192 /dev/ad0 --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --/04w6evG8XlLl3ft Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFlTP/ForvXbEpPzQRArc5AJ9UbLqsTvysxAubWBW/u66VjAj7DwCfe1ZN 3/wWR93bdB7k8zyHW3jqMes= =1T5V -----END PGP SIGNATURE----- --/04w6evG8XlLl3ft-- From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 16:50:02 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E8DFA16A492 for ; Fri, 29 Dec 2006 16:50:02 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30302.mail.mud.yahoo.com (web30302.mail.mud.yahoo.com [209.191.69.64]) by mx1.freebsd.org (Postfix) with SMTP id 6AF8D13C442 for ; Fri, 29 Dec 2006 16:50:02 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 20218 invoked by uid 60001); 29 Dec 2006 16:50:01 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=a8WV3QAsoIIqA3tUVVYnDGwUtaI0DPyJ4AaV7UlaHj5RJaxLscB33gKxjoTlqGicXs2z+xvQNUnAghZCzNHHnwdb9Jmc15x7yBXhD1eyXZbLn0mbOAi/kPU60VOR28Hnwhbn3GMVB6wvwwbGrabD2cRbzmohcAzYIXqTjkomO+4=; X-YMail-OSG: fbfF58YVM1l62_Zft5_WTbNgHDHdetIbndKEOg4_hG2szWQdXs8rtjrb3E7iHz.KVhaMM8PaL8Vb_XnSt8zGLAT1ytKXro6SuCgIbPhMEvPGVyQrWuz5tS8QgNMqpV3fv4LRkT4zaw-- Received: from [85.212.8.201] by web30302.mail.mud.yahoo.com via HTTP; Fri, 29 Dec 2006 08:50:01 PST Date: Fri, 29 Dec 2006 08:50:01 -0800 (PST) From: "R. B. Riddick" To: Pawel Jakub Dawidek , Vasil Dimov In-Reply-To: <20061229152759.GA2375@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <332304.17512.qm@web30302.mail.mud.yahoo.com> Cc: freebsd-geom@freebsd.org Subject: Re: graid3 device gives "Invalid argument" when requested to read >=128kb X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 16:50:03 -0000 --- Pawel Jakub Dawidek wrote: > Using sectorsize larger than page size won't work with UFS anyway. > Not sure if this is graid3(8) issue or some general issue. Can you try > create provider with 8kB sectorsize using gnop(8) and retry your test? > He read from the raw device (/dev/raid3/...) IIRC... -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 17:08:56 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 91DF516A407; Fri, 29 Dec 2006 17:08:56 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 35AAD13C461; Fri, 29 Dec 2006 17:08:56 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id B4C3C487FF; Fri, 29 Dec 2006 18:08:54 +0100 (CET) Received: from localhost (dkg70.neoplus.adsl.tpnet.pl [83.24.10.70]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 72340487F0; Fri, 29 Dec 2006 18:08:49 +0100 (CET) Date: Fri, 29 Dec 2006 18:08:23 +0100 From: Pawel Jakub Dawidek To: "R. B. Riddick" Message-ID: <20061229170823.GB2375@garage.freebsd.pl> References: <20061229152759.GA2375@garage.freebsd.pl> <332304.17512.qm@web30302.mail.mud.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="1UWUbFP1cBYEclgG" Content-Disposition: inline In-Reply-To: <332304.17512.qm@web30302.mail.mud.yahoo.com> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: ** X-Spam-Status: No, score=2.6 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL,RCVD_IN_XBL autolearn=no version=3.0.4 Cc: Vasil Dimov , freebsd-geom@freebsd.org Subject: Re: graid3 device gives "Invalid argument" when requested to read >=128kb X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 17:08:56 -0000 --1UWUbFP1cBYEclgG Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 29, 2006 at 08:50:01AM -0800, R. B. Riddick wrote: > --- Pawel Jakub Dawidek wrote: > > Using sectorsize larger than page size won't work with UFS anyway. > > Not sure if this is graid3(8) issue or some general issue. Can you try > > create provider with 8kB sectorsize using gnop(8) and retry your test? > >=20 > He read from the raw device (/dev/raid3/...) IIRC... I know, I'm just saying that such large sectorsizes aren't too useful in practice. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --1UWUbFP1cBYEclgG Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFlUuHForvXbEpPzQRAke4AJ9JzVfv9mvYCxZgovpCg9qeO+YtVQCeIs4v 73sRYiKaQH4ZmtZm52dIiSo= =E6II -----END PGP SIGNATURE----- --1UWUbFP1cBYEclgG-- From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 23:05:45 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C123416A403 for ; Fri, 29 Dec 2006 23:05:45 +0000 (UTC) (envelope-from marius.nuennerich@gmx.net) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 1F5CE13C459 for ; Fri, 29 Dec 2006 23:05:44 +0000 (UTC) (envelope-from marius.nuennerich@gmx.net) Received: (qmail invoked by alias); 29 Dec 2006 22:39:04 -0000 Received: from dslb-084-061-204-067.pools.arcor-ip.net (EHLO sol.hackerzberg.local) [84.61.204.67] by mail.gmx.net (mp045) with SMTP; 29 Dec 2006 23:39:04 +0100 X-Authenticated: #5707313 Date: Fri, 29 Dec 2006 23:39:04 +0100 From: Marius Nuennerich To: freebsd-geom@freebsd.org Message-ID: <20061229233904.1b90ba62@sol.hackerzberg.local> X-Mailer: Sylpheed-Claws 2.6.0 (GTK+ 2.10.6; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Subject: GEOM_AES using malloc with M_WAITOK X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 23:05:45 -0000 Hi List, I'm new to geom and thought I'd take an easy class to learn. So I read geom_aes.c and noticed that it uses the M_WAITOK flag with g_malloc. In another place I read, that it is not OK to sleep in the g_up and g_down threads. Isn't this g_malloc called in one of these threads? Yes, I know that geom_aes is more for educational purpose. But then it should be right even more. regards Marius From owner-freebsd-geom@FreeBSD.ORG Sat Dec 30 01:29:15 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9485016A40F for ; Sat, 30 Dec 2006 01:29:15 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 4F2D313C448 for ; Sat, 30 Dec 2006 01:29:15 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1H0T20-0001K1-2Z for freebsd-geom@freebsd.org; Sat, 30 Dec 2006 02:29:04 +0100 Received: from 83-131-99-253.adsl.net.t-com.hr ([83.131.99.253]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 30 Dec 2006 02:29:04 +0100 Received: from ivoras by 83-131-99-253.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 30 Dec 2006 02:29:04 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Sat, 30 Dec 2006 02:28:30 +0100 Lines: 29 Message-ID: References: <20061229145657.GA24409@qlovarnika.bg.datamax> <20061229152759.GA2375@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig5F2D6405BE72AC29A025CD61" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 83-131-99-253.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: <20061229152759.GA2375@garage.freebsd.pl> X-Enigmail-Version: 0.94.1.2 Sender: news Subject: Re: graid3 device gives "Invalid argument" when requested to read >=128kb X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2006 01:29:15 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig5F2D6405BE72AC29A025CD61 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Pawel Jakub Dawidek wrote: > Using sectorsize larger than page size won't work with UFS anyway. It worked in 5.x for me (8 KB sectors), so if it doesn't now, it's a (not significant) regression. --------------enig5F2D6405BE72AC29A025CD61 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFlcDEldnAQVacBcgRAoobAKDvbCr4mkRP4ItwjcF2yIHvzbQbRwCcDP8z Q6o/qpSAPurde58BsyM8aaQ= =w1yk -----END PGP SIGNATURE----- --------------enig5F2D6405BE72AC29A025CD61-- From owner-freebsd-geom@FreeBSD.ORG Sat Dec 30 01:53:04 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C6B1F16A403 for ; Sat, 30 Dec 2006 01:53:04 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.freebsd.org (Postfix) with ESMTP id 6D11213C441 for ; Sat, 30 Dec 2006 01:53:04 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 823B4487F4; Sat, 30 Dec 2006 02:53:02 +0100 (CET) Received: from localhost (dkg70.neoplus.adsl.tpnet.pl [83.24.10.70]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id B5087456B1; Sat, 30 Dec 2006 02:52:57 +0100 (CET) Date: Sat, 30 Dec 2006 02:52:29 +0100 From: Pawel Jakub Dawidek To: Ivan Voras Message-ID: <20061230015229.GE2375@garage.freebsd.pl> References: <20061229145657.GA24409@qlovarnika.bg.datamax> <20061229152759.GA2375@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="bi5JUZtvcfApsciF" Content-Disposition: inline In-Reply-To: X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: ** X-Spam-Status: No, score=2.6 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL,RCVD_IN_XBL autolearn=no version=3.0.4 Cc: freebsd-geom@freebsd.org Subject: Re: graid3 device gives "Invalid argument" when requested to read >=128kb X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2006 01:53:04 -0000 --bi5JUZtvcfApsciF Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Dec 30, 2006 at 02:28:30AM +0100, Ivan Voras wrote: > Pawel Jakub Dawidek wrote: >=20 > > Using sectorsize larger than page size won't work with UFS anyway. >=20 > It worked in 5.x for me (8 KB sectors), so if it doesn't now, it's a > (not significant) regression. I don't remember where the problems were, but I'm quite sure there were somewhere. Have you tried some mmap(2) tests? --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --bi5JUZtvcfApsciF Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFlcZdForvXbEpPzQRAg7tAKDBYasEITvAFZTGkEy7GGNNX4jaWACfWCOM q9i4bsU0YZYXZHKcHRu60CI= =HrQ3 -----END PGP SIGNATURE----- --bi5JUZtvcfApsciF-- From owner-freebsd-geom@FreeBSD.ORG Sat Dec 30 06:45:00 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0CBBD16A403 for ; Sat, 30 Dec 2006 06:45:00 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [64.129.166.50]) by mx1.freebsd.org (Postfix) with ESMTP id AE57213C428 for ; Sat, 30 Dec 2006 06:44:59 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from [192.168.42.21] (andersonbox1.centtech.com [192.168.42.21]) by mh1.centtech.com (8.13.8/8.13.8) with ESMTP id kBU6UPeZ012983; Sat, 30 Dec 2006 00:30:26 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <45960781.3040304@centtech.com> Date: Sat, 30 Dec 2006 00:30:25 -0600 From: Eric Anderson User-Agent: Thunderbird 1.5.0.9 (X11/20061223) MIME-Version: 1.0 To: Ivan Voras References: <20061228171858.GA11296@qlovarnika.bg.datamax> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.88.4/2395/Fri Dec 29 17:16:50 2006 on mh1.centtech.com X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=8.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.6 X-Spam-Checker-Version: SpamAssassin 3.1.6 (2006-10-03) on mh1.centtech.com Cc: freebsd-geom@freebsd.org Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2006 06:45:00 -0000 On 12/28/06 12:27, Ivan Voras wrote: > Ivan Voras wrote: > >> I'll take a shot at this: Since maximum kernel reads are still limited >> to 128 KB/s, by adding more drives you're making individual requests >> shorter. I.e. with one drive, it gets 128 KB requests, with two, each >> gets 64 KB, with 16, each gets 8 KB. So network & kernel latency becomes >> visible. > > Btw. if I'm right, you should be able to observe this with gstat and/or > iostat. If you don't see it, then it's possible I'm wrong :) Maybe this should be tried with md* devices just to remove the network component from this mix. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology An undefined problem has an infinite number of solutions. ------------------------------------------------------------------------ From owner-freebsd-geom@FreeBSD.ORG Sat Dec 30 08:03:06 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 11B8516A47B for ; Sat, 30 Dec 2006 08:03:06 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30306.mail.mud.yahoo.com (web30306.mail.mud.yahoo.com [209.191.69.68]) by mx1.freebsd.org (Postfix) with SMTP id B20A713C471 for ; Sat, 30 Dec 2006 08:03:05 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 14319 invoked by uid 60001); 30 Dec 2006 08:03:05 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=QOcYQQ+ZpdLvB+XsTEZkH9Es9a1WbSStKzo872GLJoj5MkmTyEPnalv7gQL3kQen55kAvpXLoW5gj5dJJPny3x8gcTp6CHxnejn6npSyKT1KOEza+4ChOBtUnleR7joWNv1eArCE10iISaFuI/M9Ie63+Dx9uVT/WUudC7u1jq0= ; Message-ID: <20061230080305.14317.qmail@web30306.mail.mud.yahoo.com> X-YMail-OSG: F33IurUVM1mohvzk0Bwta8e7IX0OrbxTxDAL2tJRHabXwMPDpVpmZuFDT8TwDT4jr7JaMCG7eUQ9oXM3.IDSmPkaY7tg0eKC3sQfHsd7Iaso0wu1hjAXckTBR__P5vDlXhVysZkIm8hRXOM- Received: from [85.212.9.158] by web30306.mail.mud.yahoo.com via HTTP; Sat, 30 Dec 2006 00:03:05 PST Date: Sat, 30 Dec 2006 00:03:05 -0800 (PST) From: "R. B. Riddick" To: Marius Nuennerich , freebsd-geom@freebsd.org In-Reply-To: <20061229233904.1b90ba62@sol.hackerzberg.local> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: GEOM_AES using malloc with M_WAITOK X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2006 08:03:06 -0000 --- Marius Nuennerich wrote: > I'm new to geom and thought I'd take an easy class to learn. So I read > geom_aes.c and noticed that it uses the M_WAITOK flag with g_malloc. In > another place I read, that it is not OK to sleep in the g_up and g_down > threads. Isn't this g_malloc called in one of these threads? > In both threads geom_aes calls g_malloc with M_WAITOK. Both threads set the TDP_NOSLEEPING flag before they call the class' function. On the other hand side it is allowed to use mtx_lock() in those threads, although it can cause delays, too (maybe those delays are different from sleep?). In the ..._start() function it would be easy to push back the request for later processing with g_io_deliver(...,ENOMEM). In the ..._done() function it would be not so easy, so that this memory, that is needed for each and every read request, should be allocated somewhere else and should have a pointer stored in softc (e. g. in ..._create()). -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Sat Dec 30 12:05:06 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BD8B616A407 for ; Sat, 30 Dec 2006 12:05:06 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 77EDD13C441 for ; Sat, 30 Dec 2006 12:05:04 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1H0cxP-0005Zx-QW for freebsd-geom@freebsd.org; Sat, 30 Dec 2006 13:04:59 +0100 Received: from 89-172-50-199.adsl.net.t-com.hr ([89.172.50.199]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 30 Dec 2006 13:04:59 +0100 Received: from ivoras by 89-172-50-199.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 30 Dec 2006 13:04:59 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Sat, 30 Dec 2006 13:04:43 +0100 Lines: 34 Message-ID: References: <20061228171858.GA11296@qlovarnika.bg.datamax> <45960781.3040304@centtech.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigF7431A5591836E58C66B60C4" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-50-199.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: <45960781.3040304@centtech.com> X-Enigmail-Version: 0.94.1.2 Sender: news Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2006 12:05:06 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF7431A5591836E58C66B60C4 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Eric Anderson wrote: > Maybe this should be tried with md* devices just to remove the network > component from this mix. Network was introduced here to do bandwidth limiting so to get predictable results, but you're right - network could, at least in theory, be the issue here. I remember trying to do such a benchmark with USB sticks, which also have high latency, and getting similar scaling problems because the requests were split to small blocks. --------------enigF7431A5591836E58C66B60C4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFllXhldnAQVacBcgRArWTAJ9RJ1DOBgh+LGF9rdvozJbLDhIRjgCg46oD UxL4rl0mnEjATEmYnSmc7IQ= =8ENL -----END PGP SIGNATURE----- --------------enigF7431A5591836E58C66B60C4-- From owner-freebsd-geom@FreeBSD.ORG Sat Dec 30 12:28:24 2006 Return-Path: X-Original-To: freebsd-geom@FreeBSD.org Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0A88B16A412 for ; Sat, 30 Dec 2006 12:28:24 +0000 (UTC) (envelope-from ivoras@fer.hr) Received: from ls405.t-com.hr (ls405.t-com.hr [195.29.150.135]) by mx1.freebsd.org (Postfix) with ESMTP id B8E8D13C442 for ; Sat, 30 Dec 2006 12:28:23 +0000 (UTC) (envelope-from ivoras@fer.hr) Received: from ls422.t-com.hr (ls422.t-com.hr [195.29.150.237]) by ls405.t-com.hr (Postfix) with ESMTP id 932B61442E9 for ; Sat, 30 Dec 2006 12:57:54 +0100 (CET) Received: from ls422.t-com.hr (localhost.localdomain [127.0.0.1]) by ls422.t-com.hr (Qmlai) with ESMTP id 7DCA1C9005A for ; Sat, 30 Dec 2006 12:57:54 +0100 (CET) X-Envelope-Sender-Info: KDHLkYIFXCRHG7zIR7FVXsHrs7kBJq7A7WYxvuFw0uLBmXFtcrcp/AMZyQ1Lell9 X-Envelope-Sender: ivoras@fer.hr Received: from [10.0.0.100] (89-172-50-199.adsl.net.t-com.hr [89.172.50.199])by ls422.t-com.hr (Qmlai) with ESMTP id 33B891308047for ; Sat, 30 Dec 2006 12:57:54 +0100 (CET) Message-ID: <45965441.6010002@fer.hr> Date: Sat, 30 Dec 2006 12:57:53 +0100 From: Ivan Voras User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: freebsd-geom@FreeBSD.org References: <20061229145657.GA24409@qlovarnika.bg.datamax> <20061229152759.G A2375@garage.freebsd.pl> <20061230015229.GE2375@garage.freebsd.pl> In-Reply-To: <20061230015229.GE2375@garage.freebsd.pl> X-Enigmail-Version: 0.94.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-imss-version: 2.045 X-imss-result: Passed X-imss-scores: Clean:99.90000 C:2 M:3 S:5 R:5 X-imss-settings: Baseline:1 C:1 M:1 S:1 R:1 (0.0000 0.0000) Cc: Subject: Re: graid3 device gives "Invalid argument" when requested to read>=128kb X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2006 12:28:24 -0000 Pawel Jakub Dawidek wrote: > > On Sat, Dec 30, 2006 at 02:28:30AM +0100, Ivan Voras wrote: >> >> It worked in 5.x for me (8 KB sectors), so if it doesn't now, it's a >> >> (not significant) regression. > > > > I don't remember where the problems were, but I'm quite sure there were > > somewhere. Have you tried some mmap(2) tests? No, probably not. The whole thing was my very first attempt at doing anything with GEOM (ggcomp - it still exists somewhere), which wasn't very usable :) "Normal" file system operations worked, though I think there were problems even here with 16 KB sectors. From owner-freebsd-geom@FreeBSD.ORG Sat Dec 30 12:39:41 2006 Return-Path: X-Original-To: freebsd-geom@FreeBSD.org Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 153E116A407 for ; Sat, 30 Dec 2006 12:39:41 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30303.mail.mud.yahoo.com (web30303.mail.mud.yahoo.com [209.191.69.65]) by mx1.freebsd.org (Postfix) with SMTP id BBE9B13C45E for ; Sat, 30 Dec 2006 12:39:40 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 61873 invoked by uid 60001); 30 Dec 2006 12:39:40 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=FSCfDv2tAat63OkV2zyhsUtdeu9MTg+58U0Bql0eff0QTBjRRjPq+Fmkl7CX8sG2D9yPM3Rb2zlkyHfin4Qt8KSLA6LmOszYh1vyPTaJ2xb7wk+3uYFsZ8c1bh6FIcuUKuhJNrHMul9EbiwlBEqG/CL6uNLTowjBWkOckXV6xCw=; X-YMail-OSG: t598suIVM1nzZ219UuJVypXkkIZQem1GYOcyeCtmf2TqPGxBueZQmpcTZ1wkLaTDLZKmaSvbOK4ri7wx8nJoofETQWm4GqqKhVsMALrDXD.vO0YZ2gZeSQVoj.SHFgVWLb_jKnO.smckTaauh7lT0noDDolk2uerLQ-- Received: from [85.212.9.158] by web30303.mail.mud.yahoo.com via HTTP; Sat, 30 Dec 2006 04:39:39 PST Date: Sat, 30 Dec 2006 04:39:39 -0800 (PST) From: "R. B. Riddick" To: Ivan Voras , freebsd-geom@FreeBSD.org In-Reply-To: <45965441.6010002@fer.hr> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <36644.58831.qm@web30303.mail.mud.yahoo.com> Cc: Subject: Re: graid3 device gives "Invalid argument" when requested to read>=128kb X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2006 12:39:41 -0000 --- Ivan Voras wrote: > No, probably not. The whole thing was my very first attempt at doing > anything with GEOM (ggcomp - it still exists somewhere), which wasn't > very usable :) "Normal" file system operations worked, though I think > there were problems even here with 16 KB sectors. > Sector size seems to be not the reason (at least with gnop and dd on R6.1): neo# gnop create -S 16384 md0 neo# dd if=/dev/md0.nop of=/dev/null dd: /dev/md0.nop: Invalid argument 0+0 records in 0+0 records out 0 bytes transferred in 0.000155 secs (0 bytes/sec) neo# dd if=/dev/md0.nop of=/dev/null bs=16k 1024+0 records in 1024+0 records out 16777216 bytes transferred in 0.398660 secs (42084027 bytes/sec) Just UFS seems to have a problem: neo# newfs /dev/md0.nop /dev/md0.nop: 16.0MB (32768 sectors) block size 16384, fragment size 16384 using 4 cylinder groups of 4.02MB, 257 blks, 128 inodes. newfs: can't read old UFS1 superblock: read error from block device: Invalid argument -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-geom@FreeBSD.ORG Sat Dec 30 21:29:28 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A017D16A403 for ; Sat, 30 Dec 2006 21:29:28 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30308.mail.mud.yahoo.com (web30308.mail.mud.yahoo.com [209.191.69.70]) by mx1.freebsd.org (Postfix) with SMTP id 6A96713C44B for ; Sat, 30 Dec 2006 21:29:28 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: (qmail 34904 invoked by uid 60001); 30 Dec 2006 21:29:27 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-ID; b=teH+4rBWtjaTfb06MP7YGy1w5u/zYskXDqLcBOrXmH+g0af4/bf2zQVIKBNgDGgCGWBqHLB7RKzG3SPQq8nxPBi0LeJQydZQdVJgnczcWnRrOBkmJLM432MwtmIFMwIPzXlye2V0ZuFK44wE9WeMykTtcBB2/YFfgXTrRgQERvY=; X-YMail-OSG: 8osjnPIVM1k.UKGwuE0Ysq_eW1qcvkqlURcmhaTGIb0hKxwJ.D4Rmqlz84Gseuv7YmmHdUx6wRcaooTvWvbSoNiyAHFW8kteHStQxIQdYHpMfVMMuk6yRfat5GFupiDDTXdkkvYJ3.t0L0ow5PJyjmKcHeWmGqCWuozYUgUVZ2t2yFi6qfjIz6rSinbDwpHjZU1pxcvYc.pyq_k- Received: from [85.212.9.158] by web30308.mail.mud.yahoo.com via HTTP; Sat, 30 Dec 2006 13:29:27 PST Date: Sat, 30 Dec 2006 13:29:27 -0800 (PST) From: "R. B. Riddick" To: Marius Nuennerich , freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Message-ID: <876806.34709.qm@web30308.mail.mud.yahoo.com> Cc: Subject: Re: GEOM_AES using malloc with M_WAITOK X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2006 21:29:28 -0000 --- "R. B. Riddick" wrote: > --- Marius Nuennerich wrote: > > I'm new to geom and thought I'd take an easy class to learn. So I read > > geom_aes.c and noticed that it uses the M_WAITOK flag with g_malloc. In > > another place I read, that it is not OK to sleep in the g_up and g_down > > threads. Isn't this g_malloc called in one of these threads? > > > In both threads geom_aes calls g_malloc with M_WAITOK. > Both threads set the TDP_NOSLEEPING flag before they call the class' > function. > On the other hand side it is allowed to use mtx_lock() in those threads, > although it can cause delays, too (maybe those delays are different from > sleep?). > > In the ..._start() function it would be easy to push back the request for > later processing with g_io_deliver(...,ENOMEM). > > In the ..._done() function it would be not so easy, so that this memory, that > is needed for each and every read request, should be allocated somewhere else > and should have a pointer stored in softc (e. g. in ..._create()). > I looked a little bit at the kernel sources and found this: 1. mtx_lock uses turnstile_wait() in /sys/kern/kern_mutex.c, which (according to /sys/kern/subr_turnstile.c) causes a context switch (no sleep), which is obviously allowed with flag TDP_NOSLEEPING set... 2. malloc(9) uses uma_zalloc() or uma_large_malloc() which both use uma_zone_slab() in /sys/vm/uma_core.c which uses msleep, which might be not allowed with TDP_NOSLEEPING set (see subr_sleepqueue.c and kern_synch.c in /sys/kern)... -Arne --- from "Boston Legal": "Denny Denny Denny Denny Crane" __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com