Date: Wed, 12 Oct 2016 20:54:45 +0200 From: Michael Osipov <1983-01-06@gmx.net> To: freebsd-geom@freebsd.org Subject: Re: Abysmally slow write to geom class volume over network Message-ID: <15e9cb94-7ad8-e547-b06a-699ce2250624@gmx.net> In-Reply-To: <346c7da7-02b2-ba34-1463-f3f0a5a3cd9a@rlwinm.de> References: <33da0f73-48f1-4727-fe76-41343dc4955b@gmx.net> <18314f27-849b-31df-d88d-af64e89c133f@gmx.net> <346c7da7-02b2-ba34-1463-f3f0a5a3cd9a@rlwinm.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Am 2016-10-12 um 19:08 schrieb Jan Bramkamp:
>
>
> On 12/10/2016 17:25, Michael Osipov wrote:
>> Am 2016-10-06 um 20:34 schrieb Michael Osipov:
>>> Hi folks,
>>>
>>> I am experiecing a performance breakdown when writing to some geom
class
>>> volume over network. I have tried SCP/SFTP, SMB, nc, ggated/ggatec from
>>> Windows 10 and FreeBSD 10.3-RELEASE and I receive at most 500 kB/s.
>>> Reading is superfast as well as writing to a regular, non-geom volume,
>>> network is fully saturated.
>>>
>>> I am on:
>>> # uname -a
>>> FreeBSD bsd1home 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11
>>> 18:37:29 UTC 2016
>>> root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386
>>>
>>> Any idea what the problem is? How can I investigate on the penality in
>>> performance?
>>
>> Seriously no one able to help?
>
> Not with the description you've given.
Probably, yes though I wasn't even sure where to start.
> Please describe the network setup, how you access remote block devices,
> what you mean by "geom volume".
Simply put, I have this:
$ uname -a
FreeBSD bsd1home 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11
18:37:29 UTC 2016
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386
atapci0@pci0:2:13:0: class=0x018000 card=0x00051103 chip=0x00041103
rev=0x03 hdr=0x00
vendor = 'HighPoint Technologies, Inc.'
device = 'HPT366/368/370/370A/372/372N'
class = mass storage
bfe0@pci0:2:5:0: class=0x020000 card=0x80a81043 chip=0x440114e4
rev=0x01 hdr=0x00
vendor = 'Broadcom Corporation'
device = 'BCM4401 100Base-T'
class = network
subclass = ethernet
# camcontrol devlist | grep WDC
<WDC WD2000JB-00GVA0 08.02D08> at scbus0 target 0 lun 0 (ada0,pass0)
<WDC WD2000JB-00FUA0 15.05R15> at scbus0 target 1 lun 0 (ada1,pass1)
<WDC WD2000JB-00EVA0 15.05R15> at scbus1 target 0 lun 0 (ada2,pass2)
# graid3 label -rv data ada0 ada1 ada2
Metadata value stored on ada0.
Metadata value stored on ada1.
Metadata value stored on ada2.
Done.
# newfs -j /dev/raid3/data
/dev/raid3/data: 381564.4MB (781443928 sectors) block size 32768,
fragment size 4096
using 610 cylinder groups of 626.22MB, 20039 blks, 80256 inodes.
with soft updates
...
Using inode 4 in cg 0 for 33554432 byte journal
newfs: soft updates journaling set
# mount /dev/raid3/data /mnt
After the filesystem has been mounted, I have copied huge files,
hundreds of megabytes each, via SFTP and nc(1) to that /mnt from another
machine:
$ stat Image.bin
File: 'Image.bin'
Size: 654302880 Blocks: 638968 IO Block: 65536 regular file
Device: 3041c3c6h/809616326d Inode: 562949953455390 Links: 1
Access: (0644/-rw-r--r--) Uid: (197609/ mosipov) Gid: (197121/ UNKNOWN)
Access: 2016-01-23 09:30:50.807257500 +0100
Modify: 2010-10-10 12:35:43.609375000 +0200
Change: 2010-10-10 12:35:43.609375000 +0200
Birth: 2016-01-23 09:30:50.807257500 +0100
$ scp Image.bin 192.168.1.7:/mnt
Password for mosipov@bsd1home:
Image.bin 0% 5888KB 52.8KB/s
3:19:57 ETA
Now let's copy to non-geom filesystem:
$ scp Image.bin 192.168.1.7:/usr/home/mosipov
Password for mosipov@bsd1home:
Password for mosipov@bsd1home:
Image.bin 100% 624MB 10.6MB/s 00:59
It it neither the network card nor the controller as you can see,
non-geom is fast and local writes too:
# dd if=/dev/zero bs=1M count=10240 | pv -s 10G -treab --progress >
/mnt/file.bin
10240+0 records out
10737418240 bytes transferred in 216.084333 secs (49690869 bytes/sec)
10GiB 0:03:36 [47.4MiB/s] [47.4MiB/s] [==========================>] 100%
As if there is a bottleneck between socket read and geom write to FS.
Is that better?
Best regards,
Michael
PS: iSCSI won't be an option because the machine will be access either
from Windows via SMB or SFTP, alternatively via DNLA (HTTP) from phones
and TV.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?15e9cb94-7ad8-e547-b06a-699ce2250624>