From owner-freebsd-geom@freebsd.org Fri Oct 14 16:12:55 2016 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70FFAC10ACE for ; Fri, 14 Oct 2016 16:12:55 +0000 (UTC) (envelope-from 1983-01-06@gmx.net) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D8B95132 for ; Fri, 14 Oct 2016 16:12:54 +0000 (UTC) (envelope-from 1983-01-06@gmx.net) Received: from [192.168.1.8] ([62.157.52.151]) by mail.gmx.com (mrgmx101) with ESMTPSA (Nemesis) id 0LuKHz-1cwID62bgt-011i9e; Fri, 14 Oct 2016 18:12:51 +0200 Subject: Re: Abysmally slow write to geom class volume over network To: Fabian Keil References: <33da0f73-48f1-4727-fe76-41343dc4955b@gmx.net> <18314f27-849b-31df-d88d-af64e89c133f@gmx.net> <346c7da7-02b2-ba34-1463-f3f0a5a3cd9a@rlwinm.de> <15e9cb94-7ad8-e547-b06a-699ce2250624@gmx.net> <20161013014143.GA1669@funkthat.com> <636763f0-a732-18ba-262b-c3fc01f4342c@gmx.net> <20161014114330.396fe534@fabiankeil.de> Cc: freebsd-geom@freebsd.org From: Michael Osipov <1983-01-06@gmx.net> Message-ID: <5a12f70d-3488-e799-b875-9b358ef7aff9@gmx.net> Date: Fri, 14 Oct 2016 18:12:49 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161014114330.396fe534@fabiankeil.de> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:XcSPIKUuXXLjQxsDfLLyiH81xpoaxV6tPBklRejca2Zii5FUTOZ hD8YhmCFtxoXIE+h7LyPQDtSrSYFqxy/Xdj/sOvZRBjFEaJ4eGBv4P9ZLwxvODRdCZzfjdN yAUL9j/1kQN24jnyzYatDacWOBoiFtYJni/YliXzKbkTEN8u3YzXD2/oA0LB/gMxVdmYmDf hhEfuQFkKoYnTd3vvCrqg== X-UI-Out-Filterresults: notjunk:1;V01:K0:8kk/7Bf3u+I=:cBf2KXV2i8XsBQuHP20kSS YV6y7hTQByu5NIXF/wc8RXI65jqCJXDy/QOzLZgdlJxRcPbW3APgYLV9e0/InRXSnZiC9UZKy 171km9xnFmm+Bya9TnPjO/kwD/YoIz6dN/T61n8RQ7cDADfYsOsecWCA/TnBADo95I1hH8iqD +dZtdwnvdGbVc2wqf/uOZ74+jPtaREO2k488bXHUreRfrDb8f223ryAwYB5TuHkDUtHV8R2QE 8zrO6qDqK/gF87Av7q+v0rvhM2HAcUhqg0WzVkPorBL/tA4KBH0mN48UA4BGuSrg1iFlODtSs j+DLzSlzyO9OOWQleGUcOQ51SL4rM++1llgutwTuqTuao2hkQaEyKJtYRdHMLI7USbaXLUSE9 fqyUCVUuAydhwGXdqLwkFiDAO0X2Q9asu5pfJp+YLMWCya1leNM4ZdmxNSpc0p0kwUeqqh0Th 0H6LX26bTFBPMdnl64zU9otwKtuyRvG3wizTFl+hdUJwVZvF6em4fTQ+U5hxkCTcUj+cVSIVv jfBUymLNL3JKVtNfhc4AX9Hxmu5xtmpRhvchVIem5guRchpj4+s4XfBs9Qa6uh8wx/hzAlRhL myHWVR9zvmJ+laQsshnhca7WlHqB8u0wTyXvZpcZPcmAPHkDB5lOoGubo9m7VN1mMz7eqfgGh NSNauZxYELIzVA5rwh/gxvtof3SSjWBEaX0adlQwxAkagYURpPkspPliuN6eb1a6GBU7t3Y18 o+Osnb+xuynj2/9BZVViwd3ATX0TzBWh98lDNia15RdSqRq6sv7entiS0kXkIkIGjBfTndas1 SdudXhw X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2016 16:12:55 -0000 Am 2016-10-14 um 11:43 schrieb Fabian Keil: > Michael Osipov <1983-01-06@gmx.net> wrote: > >> Am 2016-10-13 um 03:41 schrieb John-Mark Gurney: >>> Michael Osipov wrote this message on Wed, Oct 12, 2016 at 20:54 >>> +0200: >>>> As if there is a bottleneck between socket read and geom write to FS. >>>> >>>> Is that better? >>> >>> Have you run gstat on the system to see if there is an IO bottle >>> neck? Since you are using graid3, you want to look to see if >>> it's %busy is ~100, while the underlying components are not. >> >> This is hardly impossible because as soon as I start some SFTP transfer, >> all of my SSH sessions free or receive connetion timeout/abort. Doing a >> SFTP from FreeBSD to FreeBSD gives me on both physical disks and RAID3 >> volume a busy of zero to one perfect. In other terms, the drives are >> bored. > > Try checking the FAIL and SLEEP columns in the "vmstat -z" output. I assume that you expect a rise on those numbers. I have made several runs. Rebooted the machine and then started SFTP transfer. After seconds my SSH sessions locked up. The transfer was aborted manually after 10 minutes which should have saturated the entire connection. After that, I reran vmstat -z, no or minimal rise in FAIL and SLEEP. Interesting to say that this happens if is is a UFS volume on gconcat/graid3/gvinum/gstripe configuration. Regular gpart with GPT has no performance penalty. Additionally, it is not limited to SSH but virtually everything with sockets: nc, ggate, smb. > This could be related to: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209680#c2 It pretty much sounds like it, though I do not use ipfw, pf or any NAT stuff. I will try your first patch and let you know. Do you want me to add my usecase to the issue? Michael