From owner-freebsd-stable@freebsd.org Thu Sep 22 11:54:57 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C7AFBE5FB1 for ; Thu, 22 Sep 2016 11:54:57 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B48F81240 for ; Thu, 22 Sep 2016 11:54:56 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from bsdrookie.norma.com. ([IPv6:fd00::7fe]) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPS id u8MBsqmr036632 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 22 Sep 2016 16:54:52 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1474545292; bh=G5TbbUvZhIL3hLyO9kY1RvHA9CIQc6CB1ojt4D5kQ+E=; h=To:From:Subject:Date; b=iGoFEAQQgK58G++jBa45xLC0CREVsBd+Nd6k2CqxqR0rtNREDWAwUYXUhJuM4+/S+ aREPZjRqCUM1efrXP6PYqQKyzYL0lth4w31JDaUF57DuNqMylXz32U312BrdCpJbXx V3Vle6BGxtCWYjRIUL+JXIPMmKVxJ1XBztAlqkFI= To: freebsd-stable@FreeBSD.org From: "Eugene M. Zheganin" Subject: zfs/raidz and creation pause/blocking Message-ID: <57E3C68C.8060200@norma.perm.ru> Date: Thu, 22 Sep 2016 16:54:52 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Sep 2016 11:54:57 -0000 Hi. Recently I spent a lot of time setting up various zfs installations, and I got a question. Often when creating a raidz on disks considerably big (>~ 1T) I'm seeing a weird stuff: "zpool create" blocks, and waits for several minutes. In the same time system is fully responsive and I can see in gstat that the kernel starts to tamper all the pool candidates sequentially at 100% busy with iops around zero (in the example below, taken from a live system, it's doing something with da11): (zpool create gamestop raidz da5 da7 da8 da9 da10 da11) dT: 1.064s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| da0 0 0 0 0 0.0 0 0 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0.0| da2 0 0 0 0 0.0 0 0 0.0 0.0| da3 0 0 0 0 0.0 0 0 0.0 0.0| da4 0 0 0 0 0.0 0 0 0.0 0.0| da5 0 0 0 0 0.0 0 0 0.0 0.0| da6 0 0 0 0 0.0 0 0 0.0 0.0| da7 0 0 0 0 0.0 0 0 0.0 0.0| da8 0 0 0 0 0.0 0 0 0.0 0.0| da9 0 0 0 0 0.0 0 0 0.0 0.0| da10 150 3 0 0 0.0 0 0 0.0 112.6| da11 0 0 0 0 0.0 0 0 0.0 0.0| da0p1 0 0 0 0 0.0 0 0 0.0 0.0| da0p2 0 0 0 0 0.0 0 0 0.0 0.0| da0p3 0 0 0 0 0.0 0 0 0.0 0.0| da1p1 0 0 0 0 0.0 0 0 0.0 0.0| da1p2 0 0 0 0 0.0 0 0 0.0 0.0| da1p3 0 0 0 0 0.0 0 0 0.0 0.0| da0p4 0 0 0 0 0.0 0 0 0.0 0.0| gpt/boot0 0 0 0 0 0.0 0 0 0.0 0.0| gptid/22659641-7ee6-11e6-9b56-0cc47aa41194 0 0 0 0 0.0 0 0 0.0 0.0| gpt/zroot0 0 0 0 0 0.0 0 0 0.0 0.0| gpt/esx0 0 0 0 0 0.0 0 0 0.0 0.0| gpt/boot1 0 0 0 0 0.0 0 0 0.0 0.0| gptid/23c1fbec-7ee6-11e6-9b56-0cc47aa41194 0 0 0 0 0.0 0 0 0.0 0.0| gpt/zroot1 0 0 0 0 0.0 0 0 0.0 0.0| mirror/mirror 0 0 0 0 0.0 0 0 0.0 0.0| da1p4 0 0 0 0 0.0 0 0 0.0 0.0| gpt/esx1 The most funny thing is that da5,7-11 are SSD, with a capability of like 30K iops at their least. So I wonder what is happening during this and why does it take that long. Because usually pools are creating very fast. Thanks.