Date: Wed, 12 Oct 2016 20:54:45 +0200 From: Michael Osipov <1983-01-06@gmx.net> To: freebsd-geom@freebsd.org Subject: Re: Abysmally slow write to geom class volume over network Message-ID: <15e9cb94-7ad8-e547-b06a-699ce2250624@gmx.net> In-Reply-To: <346c7da7-02b2-ba34-1463-f3f0a5a3cd9a@rlwinm.de> References: <33da0f73-48f1-4727-fe76-41343dc4955b@gmx.net> <18314f27-849b-31df-d88d-af64e89c133f@gmx.net> <346c7da7-02b2-ba34-1463-f3f0a5a3cd9a@rlwinm.de>
index | next in thread | previous in thread | raw e-mail
Am 2016-10-12 um 19:08 schrieb Jan Bramkamp:
>
>
> On 12/10/2016 17:25, Michael Osipov wrote:
>> Am 2016-10-06 um 20:34 schrieb Michael Osipov:
>>> Hi folks,
>>>
>>> I am experiecing a performance breakdown when writing to some geom
class
>>> volume over network. I have tried SCP/SFTP, SMB, nc, ggated/ggatec from
>>> Windows 10 and FreeBSD 10.3-RELEASE and I receive at most 500 kB/s.
>>> Reading is superfast as well as writing to a regular, non-geom volume,
>>> network is fully saturated.
>>>
>>> I am on:
>>> # uname -a
>>> FreeBSD bsd1home 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11
>>> 18:37:29 UTC 2016
>>> root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386
>>>
>>> Any idea what the problem is? How can I investigate on the penality in
>>> performance?
>>
>> Seriously no one able to help?
>
> Not with the description you've given.
Probably, yes though I wasn't even sure where to start.
> Please describe the network setup, how you access remote block devices,
> what you mean by "geom volume".
Simply put, I have this:
$ uname -a
FreeBSD bsd1home 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11
18:37:29 UTC 2016
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386
atapci0@pci0:2:13:0: class=0x018000 card=0x00051103 chip=0x00041103
rev=0x03 hdr=0x00
vendor = 'HighPoint Technologies, Inc.'
device = 'HPT366/368/370/370A/372/372N'
class = mass storage
bfe0@pci0:2:5:0: class=0x020000 card=0x80a81043 chip=0x440114e4
rev=0x01 hdr=0x00
vendor = 'Broadcom Corporation'
device = 'BCM4401 100Base-T'
class = network
subclass = ethernet
# camcontrol devlist | grep WDC
<WDC WD2000JB-00GVA0 08.02D08> at scbus0 target 0 lun 0 (ada0,pass0)
<WDC WD2000JB-00FUA0 15.05R15> at scbus0 target 1 lun 0 (ada1,pass1)
<WDC WD2000JB-00EVA0 15.05R15> at scbus1 target 0 lun 0 (ada2,pass2)
# graid3 label -rv data ada0 ada1 ada2
Metadata value stored on ada0.
Metadata value stored on ada1.
Metadata value stored on ada2.
Done.
# newfs -j /dev/raid3/data
/dev/raid3/data: 381564.4MB (781443928 sectors) block size 32768,
fragment size 4096
using 610 cylinder groups of 626.22MB, 20039 blks, 80256 inodes.
with soft updates
...
Using inode 4 in cg 0 for 33554432 byte journal
newfs: soft updates journaling set
# mount /dev/raid3/data /mnt
After the filesystem has been mounted, I have copied huge files,
hundreds of megabytes each, via SFTP and nc(1) to that /mnt from another
machine:
$ stat Image.bin
File: 'Image.bin'
Size: 654302880 Blocks: 638968 IO Block: 65536 regular file
Device: 3041c3c6h/809616326d Inode: 562949953455390 Links: 1
Access: (0644/-rw-r--r--) Uid: (197609/ mosipov) Gid: (197121/ UNKNOWN)
Access: 2016-01-23 09:30:50.807257500 +0100
Modify: 2010-10-10 12:35:43.609375000 +0200
Change: 2010-10-10 12:35:43.609375000 +0200
Birth: 2016-01-23 09:30:50.807257500 +0100
$ scp Image.bin 192.168.1.7:/mnt
Password for mosipov@bsd1home:
Image.bin 0% 5888KB 52.8KB/s
3:19:57 ETA
Now let's copy to non-geom filesystem:
$ scp Image.bin 192.168.1.7:/usr/home/mosipov
Password for mosipov@bsd1home:
Password for mosipov@bsd1home:
Image.bin 100% 624MB 10.6MB/s 00:59
It it neither the network card nor the controller as you can see,
non-geom is fast and local writes too:
# dd if=/dev/zero bs=1M count=10240 | pv -s 10G -treab --progress >
/mnt/file.bin
10240+0 records out
10737418240 bytes transferred in 216.084333 secs (49690869 bytes/sec)
10GiB 0:03:36 [47.4MiB/s] [47.4MiB/s] [==========================>] 100%
As if there is a bottleneck between socket read and geom write to FS.
Is that better?
Best regards,
Michael
PS: iSCSI won't be an option because the machine will be access either
from Windows via SMB or SFTP, alternatively via DNLA (HTTP) from phones
and TV.
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?15e9cb94-7ad8-e547-b06a-699ce2250624>