From owner-freebsd-stable@freebsd.org Wed Sep 27 11:07:39 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0831E30FDA for ; Wed, 27 Sep 2017 11:07:39 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-vk0-x230.google.com (mail-vk0-x230.google.com [IPv6:2607:f8b0:400c:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 498CE6DAEA for ; Wed, 27 Sep 2017 11:07:39 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-vk0-x230.google.com with SMTP id d12so6850398vkf.1 for ; Wed, 27 Sep 2017 04:07:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=pn1Cr93OOGoUliWajVrgAnnrCxe3/Clv5d+UctasPhw=; b=cmgvRxuypNp+MYE/G/MtTPkoTXMcRPGBHXoXdw6gjnzarxw1utme/Bkgqg5dRE5cAl RGQTrAbsSvX+R/LXX1/InTx/Im32s0DZHmAKOCDg15V/8K3kIDpWk1akWF/5wDwKDe9s FHNGOlM1r9Ubm2Pf/WvZ3TbKA1jQ8sT8Un22F4bImMBRQM4XgxoiDuJoeo5W1Hk+qK/o IxhZelVLSJ3ubTJQGkhOBanvaY1evka/5YhMUZt8pc6LfQZVV3Jr0Bih1ITQTMRJJjML UA1AUEqeEGrgzdmKyqcHVHr7TSewLb9AqC852okxn/LP19rotcvNQZiMjOgFO5hWiwRj ht9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=pn1Cr93OOGoUliWajVrgAnnrCxe3/Clv5d+UctasPhw=; b=UXeuqhE8+QfvHmkwdmmuqZyuvBSl9t2bWufNfAdicG+l7A2uuewPnfALTcf61MvGR5 snTtFLj50IzQtlGUssiu55wUI5uuj6XtLzljpzi4tXRXRXOCK66BdMTtq46j6KaG3EgF XppLBXmXvW9IjAtIDr/mp9vkArZMOnRNudmFsoOafzZEcgz+sHB3sXX76AgAHp/ffslt /RMVIfhPh+AXnFWlUV6yRxQ64tzO8zWWY6WdeDffZWD5ZhY2dBidGSin+CJXBMs+2SQo d+kw2ZL11d6gBzkSazePYwsTjtmJf33DZFdVU8eChZgVI9MIbYgJ6FPKQt8ejokvFauP Sbdg== X-Gm-Message-State: AHPjjUjSD2Ma9n3pJf75MGk65alf1ja8Rfj4DRSf/kGBvPhmSNVz424V omAUN0SHiYsnWcG5mtGNIfVlwY+FnIziiBrrkhufHw== X-Google-Smtp-Source: AOwi7QCArz3ciEqy9VvQw5UG1+jqWutj5kmpM/2cAQ1PmONBqXnk6pcElna5ntF0uav3S4hRyw0tgsQwOWISeeGeTgg= X-Received: by 10.31.178.200 with SMTP id b191mr557809vkf.46.1506510458419; Wed, 27 Sep 2017 04:07:38 -0700 (PDT) MIME-Version: 1.0 Sender: etnapierala@gmail.com Received: by 10.159.51.232 with HTTP; Wed, 27 Sep 2017 04:07:38 -0700 (PDT) In-Reply-To: References: From: Edward Napierala Date: Wed, 27 Sep 2017 13:07:38 +0200 X-Google-Sender-Auth: bxVQMui_bU9omalkt3lTG5Swr24 Message-ID: Subject: Re: zfs, iSCSI and volmode=dev To: "Eugene M. Zheganin" Cc: FreeBSD Stable Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 11:07:39 -0000 2017-08-30 11:45 GMT+02:00 Eugene M. Zheganin : > Hi, > > > I have an iSCSI production system that exports a large number of zvols as > the iSCSI targets. System is running FreeBSD 11.0-RELEASE-p7 and initially > all of the zvols were confugured with default volmode. I've read that it's > recommended to use them in dev mode, so the system isn't bothered with all > of these geom structures, so I've switched all of the zvols to dev mode, > then I exported/imported the pools back. Surprisingly, the performance has > fallen down like 10 times (200-300 Mbits/sec against 3-4 Gbits/sec > previously). After observing for 5 minutes the ESXes trying to boot up, and > doing this extremely slowly, I switched the volmode back to default, then > again exported/imported the pools. The performance went back to normal. > > > So... why did this happen ? The result seems to be counter-intuitive. At > least not obvious to me. I don't really have an answer - mav@ would be the best person to ask. Based on his description, "ZVOLs in GEOM mode don't support DPO/FUA cache control bits, had to chunk large I/Os into MAXPHYS-sized pieces and go through GEOM." There also used to be so that TRIM was only supported in the "dev" mode, but that changed a while ago.