From owner-freebsd-geom@freebsd.org Thu Oct 13 18:41:43 2016 Return-Path: Delivered-To: freebsd-geom@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6377FC10D1A for ; Thu, 13 Oct 2016 18:41:43 +0000 (UTC) (envelope-from 1983-01-06@gmx.net) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B759813A2 for ; Thu, 13 Oct 2016 18:41:42 +0000 (UTC) (envelope-from 1983-01-06@gmx.net) Received: from [192.168.1.8] ([62.157.47.35]) by mail.gmx.com (mrgmx101) with ESMTPSA (Nemesis) id 0M2ojS-1b26MF0uOM-00sbfe for ; Thu, 13 Oct 2016 20:41:33 +0200 Subject: Re: Abysmally slow write to geom class volume over network To: freebsd-geom@freebsd.org References: <33da0f73-48f1-4727-fe76-41343dc4955b@gmx.net> <18314f27-849b-31df-d88d-af64e89c133f@gmx.net> <346c7da7-02b2-ba34-1463-f3f0a5a3cd9a@rlwinm.de> <15e9cb94-7ad8-e547-b06a-699ce2250624@gmx.net> <20161013014143.GA1669@funkthat.com> From: Michael Osipov <1983-01-06@gmx.net> Message-ID: <636763f0-a732-18ba-262b-c3fc01f4342c@gmx.net> Date: Thu, 13 Oct 2016 20:41:32 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161013014143.GA1669@funkthat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:F+hNP6Ra/8HSu0ON+9QARBqqCkvoKDBmv+zr21G2QtRBKV2iAID fTAMzVl1ZP6CAuWetLH88sH2MRxKwIR7ijmINZsFp7gXZVfYF/nGpEF/khwZRhKpx/kCENg y46K4h4EdiICFMeTKgvEkF+wTCUPk4cU4pZVsfXyoHqkLrL1xSva8SVIKyiBed4RTLaxmUu jqewe4gUmzbfqtSW1dwBQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:3Ij86d39uYs=:02eZ/w1sbTixB3eqo8T+xG +p8bYSEPSnxxPgoXqOwiyAxkqn8FRq1esZbtctNbXJc6pEyAvpv1aV99c1O0vQg9U4ltRVCoW JhyPnkeCbwKYWQ4jbmmFlwB9z3HBNW9m8hp6Y2Lm1psmaZ22hwO2IV/+cxSMa9D+r1S0tOJIs SHUVv6ae9hl8xc52uF/vLNNCxwE2C6r5nr1qyiFyIld4hxLcVvtxL//qiozsQ+RZIP3Q9M7Xr d2Tdj6HTtHwyqqwbJYlc8Y2Wsdde2i13J1DwLblac1p3whJu/hZhE1vZ3/3hzipsZ9hAr28At KL5QDHvwqk8bDmVcs8mTywkzk3vnQ/dltsQZH5vX8zpe8nJegxyx0zRQ+waGY0JgzM33ukpJv Ac46IqGMtDlWiou+QFkFyUwsrnYW0/eHmBBxAp/SWbw7pTyTze/+vbkxAh4RSmluVPlyZrRin 78jrEMWK0+oWCiolw3VrLltqtk+vqvSoLMKvStrwggbXv2w9pMzl0CKOwZgY0UJlThviumOpx d5C2/CWKBHD9qdA4tI1Z1ijzTjMPRh+pi02f8jOLnOo/BFUe/ZBdU2kI4xuPgtLqGEdzDnbLi 9+lfcr8GDvNuXLcQinFoaz8MxyQIK9wZZKtjD65U8ay4MDXFWU5E48RUaGbym0WbBDaRay0VS MT0dN7LgrIIhaUBCCDS4hWZUrbOBzju2kZJxSdEiuwt2Iu6X2k4Zr7M1Fxnsm4IANFTmGPrSA hUD93RrEVd8tCcEgBtFrKAlK0KL0q2tlEG3KKmi9XPxUUKwKOaflRFCYxFMfvsDXWB0dyBtcj OSkZ8Qy X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Oct 2016 18:41:43 -0000 Am 2016-10-13 um 03:41 schrieb John-Mark Gurney: > Michael Osipov wrote this message on Wed, Oct 12, 2016 at 20:54 +0200: >> As if there is a bottleneck between socket read and geom write to FS. >> >> Is that better? > > Have you run gstat on the system to see if there is an IO bottle > neck? Since you are using graid3, you want to look to see if > it's %busy is ~100, while the underlying components are not. This is hardly impossible because as soon as I start some SFTP transfer, all of my SSH sessions free or receive connetion timeout/abort. Doing a SFTP from FreeBSD to FreeBSD gives me on both physical disks and RAID3 volume a busy of zero to one perfect. In other terms, the drives are bored. > Also, if you do dd if=/dev/zero of=/mnt/somefile bs=1m count=1024, > from the local system, are things still slow? I have done this in the first place, this is fast: # dd if=/dev/zero bs=1M count=10240 | pv -s 10G -treab --progress > /mnt/file.bin 10240+0 records out 10737418240 bytes transferred in 216.084333 secs (49690869 bytes/sec) 10GiB 0:03:36 [47.4MiB/s] [47.4MiB/s] [==========================>] 100% > P.S. This is basic debugging, have to identify which part of the > system is misbehaving. I am aware of that. Thank you. > P.P.S. Your "non-geom" test is still using geom, just not w/ the graid3 > module. Oh ok, I wasn't aware of that. Actually, I have tried all of them: concat, stripe, mirror, gvinum. Local operations are fast, though remote still crawls slowly. Michael