From owner-freebsd-geom@FreeBSD.ORG Mon Nov 22 11:07:05 2010 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 628271065695 for ; Mon, 22 Nov 2010 11:07:05 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 455B48FC15 for ; Mon, 22 Nov 2010 11:07:05 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id oAMB75Pq051685 for ; Mon, 22 Nov 2010 11:07:05 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id oAMB74pl051681 for freebsd-geom@FreeBSD.org; Mon, 22 Nov 2010 11:07:04 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 22 Nov 2010 11:07:04 GMT Message-Id: <201011221107.oAMB74pl051681@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-geom@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-geom@FreeBSD.org X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Nov 2010 11:07:05 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o bin/151252 geom [geom] libgeom(3) manual page contains broken link in o kern/150858 geom [geom] [geom_label] [patch] glabel(8) is not compatibl o kern/150626 geom [geom] [gjournal] gjournal(8) destroys label o kern/150555 geom [geom] gjournal unusable on GPT partitions o kern/150334 geom [geom] [udf] [patch] geom label does not support UDF o kern/149762 geom volume labels with rogue characters o bin/149215 geom [panic] [geom_part] gpart(8): Delete linux's slice via o kern/147852 geom [geom] [panic] graid3 panic: wrong offset 16384 for se o kern/147851 geom [geom] [panic] graid3 panic: g_read_data: invalid leng o kern/147667 geom [gmirror] Booting with one component of a gmirror, the o kern/147664 geom [geom] [patch] Add the ability to create linux and fat o kern/145818 geom [geom] geom_stat_open showing cached information for n o kern/145042 geom [geom] System stops booting after printing message "GE o kern/144962 geom [geom] panic when accessing GPT disk with a large numb o kern/144905 geom [geom][geom_part] panic in gpart_ctlreq when unpluggin o kern/143455 geom gstripe(8) in RELENG_8 (31st Jan 2010) broken o kern/142563 geom [geom] [hang] ioctl freeze in zpool f kern/142365 geom [geom] FreeBSD RAID1 (gmirror) is much slower than Lin o kern/141740 geom [geom] gjournal(8): g_journal_destroy concurrent error s kern/141235 geom [geom_part] 8.0 no longer provides /dev entries for al o kern/140352 geom [geom] gjournal + glabel not working o kern/135898 geom [geom] Severe filesystem corruption - large files or l o kern/134922 geom [gmirror] [panic] kernel panic when use fdisk on disk o kern/134113 geom [geli] Problem setting secondary GELI key o kern/134044 geom [geom] gmirror(8) overwrites fs with stale data from r o kern/133931 geom [geli] [request] intentionally wrong password to destr o bin/132845 geom [geom] [patch] ggated(8) does not close files opened a o kern/132273 geom glabel(8): [patch] failing on journaled partition f kern/132242 geom [gmirror] gmirror.ko fails to fully initialize o kern/131353 geom [geom] gjournal(8) kernel lock p docs/130548 geom [patch] gjournal(8) man page is missing sysctls o kern/129674 geom [geom] gjournal root did not mount on boot o kern/129645 geom gjournal(8): GEOM_JOURNAL causes system to fail to boo o kern/129245 geom [geom] gcache is more suitable for suffix based provid f kern/128276 geom [gmirror] machine lock up when gmirror module is used o kern/127420 geom [geom] [gjournal] [panic] Journal overflow on gmirrore o kern/124973 geom [gjournal] [patch] boot order affects geom_journal con o kern/124969 geom gvinum(8): gvinum raid5 plex does not detect missing s o kern/123962 geom [panic] [gjournal] gjournal (455Gb data, 8Gb journal), o kern/123122 geom [geom] GEOM / gjournal kernel lock o kern/122738 geom [geom] gmirror list "losts consumers" after gmirror de f kern/122415 geom [geom] UFS labels are being constantly created and rem o kern/122067 geom [geom] [panic] Geom crashed during boot o kern/121364 geom [gmirror] Removing all providers create a "zombie" mir o bin/120990 geom [patch] support "BIOS Boot" partition type in gpt(8) o kern/120091 geom [geom] [geli] [gjournal] geli does not prompt for pass o kern/115856 geom [geli] ZFS thought it was degraded when it should have o kern/115547 geom [geom] [patch] [request] let GEOM Eli get password fro o kern/114532 geom [geom] GEOM_MIRROR shows up in kldstat even if compile f kern/113957 geom [gmirror] gmirror is intermittently reporting a degrad o kern/113837 geom [geom] unable to access 1024 sector size storage o kern/113419 geom [geom] geom fox multipathing not failing back o kern/107707 geom [geom] [patch] [request] add new class geom_xbox360 to o kern/94632 geom [geom] Kernel output resets input while GELI asks for o kern/90582 geom [geom] [panic] Restore cause panic string (ffs_blkfree o bin/90093 geom fdisk(8) incapable of altering in-core geometry o kern/88601 geom [geli] geli cause kernel panic under heavy disk usage o kern/87544 geom [gbde] mmaping large files on a gbde filesystem deadlo o kern/84556 geom [geom] [panic] GBDE-encrypted swap causes panic at shu o kern/79251 geom [2TB] newfs fails on 2.6TB gbde device o kern/79035 geom [vinum] gvinum unable to create a striped set of mirro o bin/78131 geom gbde(8) "destroy" not working. s kern/73177 geom kldload geom_* causes panic due to memory exhaustion 63 problems total. From owner-freebsd-geom@FreeBSD.ORG Tue Nov 23 17:40:27 2010 Return-Path: Delivered-To: freebsd-geom@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 670F910656CE; Tue, 23 Nov 2010 17:40:27 +0000 (UTC) (envelope-from lulf@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3C3488FC17; Tue, 23 Nov 2010 17:40:27 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id oANHeRDZ093825; Tue, 23 Nov 2010 17:40:27 GMT (envelope-from lulf@freefall.freebsd.org) Received: (from lulf@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id oANHeRWe093813; Tue, 23 Nov 2010 17:40:27 GMT (envelope-from lulf) Date: Tue, 23 Nov 2010 17:40:27 GMT Message-Id: <201011231740.oANHeRWe093813@freefall.freebsd.org> To: psteele@maxiscale.com, lulf@FreeBSD.org, freebsd-geom@FreeBSD.org From: lulf@FreeBSD.org Cc: Subject: Re: kern/134044: [geom] gmirror(8) overwrites fs with stale data from returning member X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2010 17:40:27 -0000 Synopsis: [geom] gmirror(8) overwrites fs with stale data from returning member State-Changed-From-To: open->closed State-Changed-By: lulf State-Changed-When: Tue Nov 23 17:36:52 UTC 2010 State-Changed-Why: - Closing, as reporter does not seem to still have the problem after 11 months. Re-open if the issue turns up again. http://www.freebsd.org/cgi/query-pr.cgi?pr=134044 From owner-freebsd-geom@FreeBSD.ORG Tue Nov 23 17:42:53 2010 Return-Path: Delivered-To: freebsd-geom@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C34B1065695; Tue, 23 Nov 2010 17:42:53 +0000 (UTC) (envelope-from lulf@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 21F0A8FC24; Tue, 23 Nov 2010 17:42:53 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id oANHgr1g002187; Tue, 23 Nov 2010 17:42:53 GMT (envelope-from lulf@freefall.freebsd.org) Received: (from lulf@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id oANHgqfS002183; Tue, 23 Nov 2010 17:42:52 GMT (envelope-from lulf) Date: Tue, 23 Nov 2010 17:42:52 GMT Message-Id: <201011231742.oANHgqfS002183@freefall.freebsd.org> To: arr@watson.org, lulf@FreeBSD.org, freebsd-geom@FreeBSD.org From: lulf@FreeBSD.org Cc: Subject: Re: kern/73177: kldload geom_* causes panic due to memory exhaustion X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2010 17:42:53 -0000 Synopsis: kldload geom_* causes panic due to memory exhaustion State-Changed-From-To: suspended->closed State-Changed-By: lulf State-Changed-When: Tue Nov 23 17:42:31 UTC 2010 State-Changed-Why: - Avoid having this floating around anymore. http://www.freebsd.org/cgi/query-pr.cgi?pr=73177 From owner-freebsd-geom@FreeBSD.ORG Tue Nov 23 20:38:02 2010 Return-Path: Delivered-To: freebsd-geom@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C8021065672; Tue, 23 Nov 2010 20:38:02 +0000 (UTC) (envelope-from lulf@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 121618FC16; Tue, 23 Nov 2010 20:38:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id oANKc1nr077546; Tue, 23 Nov 2010 20:38:01 GMT (envelope-from lulf@freefall.freebsd.org) Received: (from lulf@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id oANKc1W3077542; Tue, 23 Nov 2010 20:38:01 GMT (envelope-from lulf) Date: Tue, 23 Nov 2010 20:38:01 GMT Message-Id: <201011232038.oANKc1W3077542@freefall.freebsd.org> To: anatoly.borodin@gmail.com, lulf@FreeBSD.org, freebsd-geom@FreeBSD.org From: lulf@FreeBSD.org Cc: Subject: Re: kern/122415: [geom] UFS labels are being constantly created and removed. X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Nov 2010 20:38:02 -0000 Synopsis: [geom] UFS labels are being constantly created and removed. State-Changed-From-To: feedback->closed State-Changed-By: lulf State-Changed-When: Tue Nov 23 20:36:43 UTC 2010 State-Changed-Why: - Timeout. I think this issue is a non-issue. Moreover, we do not have this verbose information anymore either. Re-open if this is still a problem. http://www.freebsd.org/cgi/query-pr.cgi?pr=122415 From owner-freebsd-geom@FreeBSD.ORG Fri Nov 26 10:28:17 2010 Return-Path: Delivered-To: freebsd-geom@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B25601065672; Fri, 26 Nov 2010 10:28:17 +0000 (UTC) (envelope-from jh@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 885918FC17; Fri, 26 Nov 2010 10:28:17 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id oAQASHlZ004845; Fri, 26 Nov 2010 10:28:17 GMT (envelope-from jh@freefall.freebsd.org) Received: (from jh@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id oAQASHEx004841; Fri, 26 Nov 2010 10:28:17 GMT (envelope-from jh) Date: Fri, 26 Nov 2010 10:28:17 GMT Message-Id: <201011261028.oAQASHEx004841@freefall.freebsd.org> To: marko.krajnc@cursor.si, jh@FreeBSD.org, freebsd-geom@FreeBSD.org From: jh@FreeBSD.org Cc: Subject: Re: kern/142365: [geom] FreeBSD RAID1 (gmirror) is much slower than Linux RAID1! X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Nov 2010 10:28:17 -0000 Synopsis: [geom] FreeBSD RAID1 (gmirror) is much slower than Linux RAID1! State-Changed-From-To: feedback->closed State-Changed-By: jh State-Changed-When: Fri Nov 26 10:28:17 UTC 2010 State-Changed-Why: Feedback timeout. http://www.freebsd.org/cgi/query-pr.cgi?pr=142365 From owner-freebsd-geom@FreeBSD.ORG Fri Nov 26 22:44:50 2010 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5124106564A for ; Fri, 26 Nov 2010 22:44:50 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from ftp.translate.ru (ftp.translate.ru [80.249.188.42]) by mx1.freebsd.org (Postfix) with ESMTP id 2F12B8FC14 for ; Fri, 26 Nov 2010 22:44:49 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (89.112.15.178.pppoe.eltel.net [89.112.15.178]) (Authenticated sender: lev@serebryakov.spb.ru) by ftp.translate.ru (Postfix) with ESMTPA id 421AA13DF42 for ; Sat, 27 Nov 2010 01:29:10 +0300 (MSK) Date: Sat, 27 Nov 2010 01:29:03 +0300 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1299537757.20101127012903@serebryakov.spb.ru> To: freebsd-geom@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: Subject: GEOM profiling - how to? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Nov 2010 22:44:50 -0000 Hello, Freebsd-geom. I'm doing some simple benchmarking of geom_raid5 in preparation of putting it into ports. And I notice strange results. It is array of 5 disks, stripsize=3D128k. All disks are SATA2 disks on ICH10R, AHCI driver (8.1-STABLE). Reading from device itself (dd with bs=3D512K) gives speed of one HDD exactly. gstat shows 100% load of RAID geom and 1/5 of this speed (and 18-22% load) on all disk GEOMs. Reading big file from FS (dd with bs=3D512k, FS block size = 32K, vfs.read_max=3D32) gives about twice speed and every disk GEOM is loaded 38-42%. CPU time is about 8% system, 0.5% interrupt, so CPU is not a bottle neck. Bigger block for "dd" doesn't wotk better. I want to know, why this configuration can not load disks up their top (and gives 4*HDD speed, about 320MB/s, speed). I even can not understand, why reeading from FS is beter than from raw device. Of course, I don't think you can me give answers for my questions. Really, I'm asking how could I peek into GEOM infrastructure to find bottle necks, understand why these two scenarios gives such different speed, etc. How could I profile I/O and GEOM? --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-geom@FreeBSD.ORG Sat Nov 27 16:04:58 2010 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C63821065670 for ; Sat, 27 Nov 2010 16:04:58 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 3C6618FC2B for ; Sat, 27 Nov 2010 16:04:57 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PMNGi-0003Jv-Ba for freebsd-geom@freebsd.org; Sat, 27 Nov 2010 17:04:56 +0100 Received: from cpe-188-129-84-38.dynamic.amis.hr ([188.129.84.38]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 27 Nov 2010 17:04:56 +0100 Received: from ivoras by cpe-188-129-84-38.dynamic.amis.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Sat, 27 Nov 2010 17:04:56 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Sat, 27 Nov 2010 17:04:42 +0100 Lines: 35 Message-ID: References: <1299537757.20101127012903@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cpe-188-129-84-38.dynamic.amis.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6 In-Reply-To: <1299537757.20101127012903@serebryakov.spb.ru> Subject: Re: GEOM profiling - how to? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Nov 2010 16:04:58 -0000 On 11/26/10 23:29, Lev Serebryakov wrote: > Hello, Freebsd-geom. > > I'm doing some simple benchmarking of geom_raid5 in preparation of > putting it into ports. And I notice strange results. > > It is array of 5 disks, stripsize=128k. All disks are SATA2 disks on > ICH10R, AHCI driver (8.1-STABLE). > > Reading from device itself (dd with bs=512K) gives speed of one HDD > exactly. gstat shows 100% load of RAID geom and 1/5 of this speed > (and 18-22% load) on all disk GEOMs. This "100% load of RAID geom" is an approximation of disk load, not CPU load. I don't know how graid5 module works but if it's like most GEOM modules, you will probably need to use a very small stripe size, basically 128 / number_of_disks so that one request can span multiple drives. In your case, try 32 KiB stripe size or 16 KiB stripe size. > Reading big file from FS (dd with bs=512k, FS block size 32K, > vfs.read_max=32) gives about twice speed and every disk GEOM is > loaded 38-42%. CPU time is about 8% system, 0.5% interrupt, so CPU > is not a bottle neck. With big readahead (btw try larger read_max values, like 128) you get parallelism on the drive hardware level, not GEOM, this is why it works. > How could I profile I/O and GEOM? There is no specific answer to this question; basically you can use gstat to observe performance of every GEOM device individually, and use "top" and similar to observe CPU usage. If you turn on GEOM logging, your logs will be swamped by a huge number of messages which you can, in theory, create a tool to analyze them with. From owner-freebsd-geom@FreeBSD.ORG Sat Nov 27 21:29:16 2010 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36C08106564A for ; Sat, 27 Nov 2010 21:29:16 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from ftp.translate.ru (ftp.translate.ru [80.249.188.42]) by mx1.freebsd.org (Postfix) with ESMTP id E16C78FC13 for ; Sat, 27 Nov 2010 21:29:15 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (89.112.15.178.pppoe.eltel.net [89.112.15.178]) (Authenticated sender: lev@serebryakov.spb.ru) by ftp.translate.ru (Postfix) with ESMTPA id 6F43C13DF42; Sun, 28 Nov 2010 00:29:14 +0300 (MSK) Date: Sun, 28 Nov 2010 00:29:07 +0300 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1468561387.20101128002907@serebryakov.spb.ru> To: Ivan Voras In-Reply-To: References: <1299537757.20101127012903@serebryakov.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-geom@freebsd.org Subject: Re: GEOM profiling - how to? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Nov 2010 21:29:16 -0000 Hello, Ivan. You wrote 27 =D0=BD=D0=BE=D1=8F=D0=B1=D1=80=D1=8F 2010 =D0=B3., 19:04:42: >> I'm doing some simple benchmarking of geom_raid5 in preparation of >> putting it into ports. And I notice strange results. >> >> It is array of 5 disks, stripsize=3D128k. All disks are SATA2 disks on >> ICH10R, AHCI driver (8.1-STABLE). >> >> Reading from device itself (dd with bs=3D512K) gives speed of one HDD >> exactly. gstat shows 100% load of RAID geom and 1/5 of this speed >> (and 18-22% load) on all disk GEOMs. > This "100% load of RAID geom" is an approximation of disk load, not CPU I understand this. > load. I don't know how graid5 module works but if it's like most GEOM=20 > modules, you will probably need to use a very small stripe size,=20 > basically 128 / number_of_disks so that one request can span multiple=20 > drives. In your case, try 32 KiB stripe size or 16 KiB stripe size. It doesn't work, really. Now I can not reconfigure array (it contains a lot of valuable data), but I did very extensive benchmarking before (about 2 years ago), and speed was best with 128Kb stripe. Stripe of 32 or 16Kb gave about half of this speed. New round of benchamrking was insipired by upgrade to 8.x (and new AHCI driver) and ampount of spare time which I want to spend on maintaince and improving RAID5, which is abadoned by author. >> Reading big file from FS (dd with bs=3D512k, FS block s= ize 32K, >> vfs.read_max=3D32) gives about twice speed and every disk GEOM is >> loaded 38-42%. CPU time is about 8% system, 0.5% interrupt, so CPU >> is not a bottle neck. > With big readahead (btw try larger read_max values, like 128) you get > parallelism on the drive hardware level, not GEOM, this is why it works. Aagain, going above "32" (which means 32*32 =3D 1024Kb, twice array-wide stripe size) doesn't work better for one thread and work much worse in mult-thread loads accroding to my benchmarsk (it was multi-demensional multi-day run with varying stripe size, FS block size and pre-read setting). So, these numbers are local optimum with this RAID implementation. My interest is, how could I improve RAID implementation to get better speeds. For this I need to understand what is bottle neck in it. >> How could I profile I/O and GEOM? > There is no specific answer to this question; basically you can use > gstat to observe performance of every GEOM device individually, and use > "top" and similar to observe CPU usage. If you turn on GEOM logging, Yes, I know these tools, but what I see here, that in both cases "top" GEOM (raid array) loaded 100%, but in one case underlying disks loaded for 20% (with individual read speeds about 20Mb/s, 80-90Mb/s combined) and in other numbers are 100% and 40% (40Mb/s individual speed and 150-160Mb/s combined). It doesn't help a lot :( > your logs will be swamped by a huge number of messages which you can, in > theory, create a tool to analyze them with. Thnaks, I'll try it. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-geom@FreeBSD.ORG Sat Nov 27 21:33:49 2010 Return-Path: Delivered-To: freebsd-geom@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00EAB106566B; Sat, 27 Nov 2010 21:33:49 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id CB6DE8FC08; Sat, 27 Nov 2010 21:33:48 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id oARLXmFG008593; Sat, 27 Nov 2010 21:33:48 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id oARLXmEO008589; Sat, 27 Nov 2010 21:33:48 GMT (envelope-from linimon) Date: Sat, 27 Nov 2010 21:33:48 GMT Message-Id: <201011272133.oARLXmEO008589@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-geom@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/152609: [geli] geli onetime on gzero panics X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Nov 2010 21:33:49 -0000 Old Synopsis: geli onetime on gzero panics New Synopsis: [geli] geli onetime on gzero panics Responsible-Changed-From-To: freebsd-bugs->freebsd-geom Responsible-Changed-By: linimon Responsible-Changed-When: Sat Nov 27 21:33:00 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=152609