From owner-freebsd-geom@freebsd.org Wed Oct 5 18:28:03 2016 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31556AF6FC0 for ; Wed, 5 Oct 2016 18:28:03 +0000 (UTC) (envelope-from 1983-01-06@gmx.net) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97664EC5 for ; Wed, 5 Oct 2016 18:28:02 +0000 (UTC) (envelope-from 1983-01-06@gmx.net) Received: from [192.168.1.8] ([62.157.36.76]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0M1FAK-1b3KaQ0iCm-00t8PD for ; Wed, 05 Oct 2016 20:22:42 +0200 To: freebsd-geom@freebsd.org From: Michael Osipov <1983-01-06@gmx.net> Subject: Abysmally slow gvinum RAID5 on three drives Message-ID: Date: Wed, 5 Oct 2016 20:22:39 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:ITxlpJIAYfll2HQV4C+49kp/fVsyrDW0T/B9o9Dfv038/N5jZpJ 7rxTIp5WMRwKcn3o6irmZPZTSwYQI1bqtZkIrBs4ZktALmkXLPIx3BcYbVQLtU0OdrsG/a6 OfuUJ0ayLebrHasA6Y9+lDxwZDU4zkhjoMbYFoV6GJaVtZrlnZVXrttvA2Bn8LHYt1UjtDT R6A17HAmpQvCr4StZj/Qw== X-UI-Out-Filterresults: notjunk:1;V01:K0:n6R7ZxZkujE=:GUTj/hVuf0WMNIhawM1fDI qEYyQN0ALyzxjG0+x6TYm+L4IFNYWsWZFi/SkAjkum+lqjKtC1RgFB5KXXWwPR7l9tX8qTIGZ UrDFY9VEAcSk5ENhFdZdpOA31qz+usaqkHOTTw0GMqOHJsyUtEoHvS20tjjeQIqQ8Bh0ZVaof zE4L2ctZU36JtLIEnl0G2NmP8gqujc8UC1UvCIC65i14uocPMZkycebGqxIHVHG5u9owDLfJb mNddlswbT01f9y5hj4VyMBMNovNNzCtTE+2GDZ315lxLTBlkzZqFbyCwpAreH5q7zCdTxfU8N eSaYToGAniLidbzlF//la9Sl/klkFcZ7m1TQfL0MyI4r9P8G3dLvWYmNrdiUwbeYU4tQDaLKo P6A5Nx8rtQ+mK7b+YKnt0JmvB7UoYNj59uPfVxWPJjMJy+mYPRpuoWTGYttSd1Uspnt0sUTiu 3APL5JA3RCUNabWiTxTL0ZpDhN8AsnVoG1BlBXi5sjYjlYZIYpnembENwPLXvXVUfdx1lNX2D 2zIuRYjIupBf1qZp0EiBsmVNnLQzytOu9UCOJ6veV9vYKX2J8cI87his0tBbaQ/q70ww3e7rF YTGVxRaknkUQtbB1npAqCrNzfTEeJxlj54e+iIRyQnfGHSiXOZuzy3Gz59tfsbVRe4bXCEyb4 MUb1tYzx55p3UIREXLvpjppBhhYpSv5y9nflZn3k2AChXfQdvMH+IjpJuaTPPOEMkcM1cMtKY rKxJ1e6Lez6phEuo705vZePeox6nJbvg1vvj4FM7tGVOwygx1ts3caQVzVlruH1GILCUZl5Lq Kiw9DPT X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Oct 2016 18:28:03 -0000 Hi folks, I have created a RAID5 wie gvinum on three 200 GB IDE disks while the formatted disks do sequentially 20 to 50 MB/s, in RAID5 mode they to no more than 100 kB/s -- yet that is kilobytes! I am on: # uname -a FreeBSD bsd1home 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:37:29 UTC 2016 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386 The following CPU is used: # grep -i cpu /var/run/dmesg.boot CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2405.51-MHz 686-class CPU) cpu0: on acpi0 RAM: 2 GiB, swap: 2,5 GiB Chipset and controller: hostb0@pci0:0:0:0: class=0x060000 card=0x80b21043 chip=0x25608086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = '82845G/GL[Brookdale-G]/GE/PE DRAM Controller/Host-Hub Interface' class = bridge subclass = HOST-PCI atapci0@pci0:2:13:0: class=0x018000 card=0x00051103 chip=0x00041103 rev=0x03 hdr=0x00 vendor = 'HighPoint Technologies, Inc.' device = 'HPT366/368/370/370A/372/372N' class = mass storage My hard disks (system is on a separate disk and controller): # camcontrol devlist | grep WDC at scbus0 target 0 lun 0 (ada0,pass0) at scbus0 target 1 lun 0 (ada1,pass1) at scbus1 target 0 lun 0 (ada2,pass2) I have tested on the raw vinum device and on one with UFS: # gvinum raid5 -n data ada0 ada1 ada2 # gvinum start data # dd if=/dev/zero bs=1M count=10240 | pv -s 10G -treab --progress > /dev/gvinum/data 2260+0 records out 2369781760 bytes transferred in 252.553715 secs (9383278 bytes/sec) # newfs -j /dev/gvinum/data /dev/gvinum/data: 381564.0MB (781443072 sectors) block size 32768, fragment size 4096 using 610 cylinder groups of 626.22MB, 20039 blks, 80256 inodes. with soft updates ... Using inode 4 in cg 0 for 33554432 byte journal newfs: soft updates journaling set # mount /dev/gvinum/data /mnt # dd if=/dev/zero bs=1M count=10240 | pv -s 10G -treab --progress > /mnt/file.bin 4+0 records out 4194304 bytes transferred in 75.191040 secs (55782 bytes/sec) 4.75MiB 0:01:20 [48.7KiB/s] [60.2KiB/s] [> ] 0% ETA 1:23:53:0 Any idea what the problem is? How can I investigate on the penality in performance? Other geom classes like concat, stripe, raid3 perform very well. Thanks, Michael From owner-freebsd-geom@freebsd.org Thu Oct 6 18:34:30 2016 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1286ABECDC9 for ; Thu, 6 Oct 2016 18:34:30 +0000 (UTC) (envelope-from 1983-01-06@gmx.net) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 658EEB27 for ; Thu, 6 Oct 2016 18:34:28 +0000 (UTC) (envelope-from 1983-01-06@gmx.net) Received: from [192.168.1.8] ([62.157.48.193]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0Ld0E0-1bA7Iq12J6-00iFHJ for ; Thu, 06 Oct 2016 20:34:20 +0200 From: Michael Osipov <1983-01-06@gmx.net> Subject: Abysmally slow write to geom class volume over network To: freebsd-geom@freebsd.org Message-ID: <33da0f73-48f1-4727-fe76-41343dc4955b@gmx.net> Date: Thu, 6 Oct 2016 20:34:17 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:kwlKjueIc0d5o35BnwjtaApqAD/UIcRp/8hHXRM3vDzvdE5xtxp cIQlFP+qE/m4lB3E5qU7dwZzrGbQ+84HSXpVYvEDA7lU7+x1VFuLE0bRPtWjG9yXYOVPwhN /KO9LRco6GhTrh9gcQjq40ANnO2fRSH1viRNfO55HvrXAteLhAQbZ3VxIExdZ1Qej6Tah8G MlsKr0k1ZL3SK+rFzNebg== X-UI-Out-Filterresults: notjunk:1;V01:K0:inj7KHZBp8E=:9OpnutOSLxwjO2+e9p48dm gl8RJDKsWalH8nXW2924L0GyAIdWso5NLiCGPMFoH6AlOV5ghqkAWGjfiChLtNSOQ3mZ8rhvQ bPZ6Lmv24iV7qEFMqGq5SA8YcSevXW2P6l8uijPvustAN4S4p+aaWV+PyjcU9cBL167+zxben mv2EHrfFgw/yFZ25kNwjStmzNrrFKxUVClxBaTmG3y0+WSg1E47KF1lxdvUeuwuKdu+nSIVhT vLC4hvegLpJ7kKkNs38ZMUEju8+1Gs/vJW+qw+hHRFrZqd8mwvSo+abx3CpILBF2bKXJza+gB Xb5uNhMo0woOeiWDHyuF1IcKsS4iBQdeWyzR7cT9dTuArlz+lGYgy7wD8v2GNbC59qvt6I1+F KeVcOTB3wMPSGbSNX8w8GsA5ugasKx4Tue9MpEngV04i0mDtDwAH58Ojf/SDMkKyikk4nzFBN cc43Yym4lUC/JGKtCUZR2+EN2k5/3aOr7req36Vgp6lWKKIMzZV8EZS8zm+VCc/E16sAQ3trS 5+d7LdOSAVP+zyoT28QUz+OQj3XXhlVROSWcS+SZIuZK3hXA/3RlTU4mmc2WUlV/ALMq8oriO pFBZMPpfYUQMucu4yPfVfGZZQXDE8jZYaq/IJy68lDPlZrmg9e+T6XYZ4FERWgI2AH0AZFA5U 01NOzyvpkirb8sAe+KxNkNC7ungWSfnYR0jUOUkJFv06cCASrL9q69QnNNhu00VkIWRWaMnw0 FP5KvQg3j3zu1uXmyDcJrB3ClBfZdihXKqXbMJ42lqf6zR2XuimdnBMz6NHfXknOOxhNGGhed nDC624m X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Oct 2016 18:34:30 -0000 Hi folks, I am experiecing a performance breakdown when writing to some geom class volume over network. I have tried SCP/SFTP, SMB, nc, ggated/ggatec from Windows 10 and FreeBSD 10.3-RELEASE and I receive at most 500 kB/s. Reading is superfast as well as writing to a regular, non-geom volume, network is fully saturated. I am on: # uname -a FreeBSD bsd1home 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:37:29 UTC 2016 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386 Any idea what the problem is? How can I investigate on the penality in performance? Thanks, Michael