From owner-freebsd-stable@freebsd.org Mon Oct 9 16:38:11 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 19105E35B46 for ; Mon, 9 Oct 2017 16:38:11 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3CFE669283 for ; Mon, 9 Oct 2017 16:38:08 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [192.168.243.2] ([192.168.243.2]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id v99Gc1bm093349 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 9 Oct 2017 21:38:02 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1507567082; bh=jmtwV0O+c/u0AsjN1033iLbHjG3ZqekLUOJ53/YCmNU=; h=Subject:Cc:References:From:Date:In-Reply-To; b=RgpJugrT5dOSZJ5Ok7PpKbhl8OqL3WdI1oROqrg1tA/fkXwnC/upTjdhZzO9NIzFW LEaXvM/pAV1E41RXqVEJIEhyfVXLMnse1PipNkWsdHWB31ic61IsRZjKTrV4ZBxnqu Tf5gMMXJdIIVrs2lTrOsIy7OR/Zf33jmOUwKyD2w= Subject: Re: zfs, iSCSI and volmode=dev Cc: FreeBSD Stable References: From: "Eugene M. Zheganin" Message-ID: Date: Mon, 9 Oct 2017 21:38:03 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Oct 2017 16:38:11 -0000 Hi, On 27.09.2017 16:07, Edward Napierala wrote: > 2017-08-30 11:45 GMT+02:00 Eugene M. Zheganin >: > > Hi, > > > I have an iSCSI production system that exports a large number of > zvols as the iSCSI targets. System is running FreeBSD > 11.0-RELEASE-p7 and initially all of the zvols were confugured > with default volmode. I've read that it's recommended to use them > in dev mode, so the system isn't bothered with all of these geom > structures, so I've switched all of the zvols to dev mode, then I > exported/imported the pools back. Surprisingly, the performance > has fallen down like 10 times (200-300 Mbits/sec against 3-4 > Gbits/sec previously). After observing for 5 minutes the ESXes > trying to boot up, and doing this extremely slowly, I switched the > volmode back to default, then again exported/imported the pools. > The performance went back to normal. > > > So... why did this happen ? The result seems to be > counter-intuitive. At least not obvious to me. > > > I don't really have an answer - mav@ would be the best person to ask.  > Based > on his description, "ZVOLs in GEOM mode don't support DPO/FUA cache > control > bits, had to chunk large I/Os into MAXPHYS-sized pieces and go through > GEOM." > There also used to be so that TRIM was only supported in the "dev" > mode, but > that changed a while ago. > Yeah, but you mean dev is faster by design. So was my first thought too, but it seems like the opposite. Default volmode is geom, and it's much faster than dev. Eugene.