From owner-freebsd-fs@freebsd.org Thu Jan 28 23:53:51 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 45F2E4F1259; Thu, 28 Jan 2021 23:53:51 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qt1-x830.google.com (mail-qt1-x830.google.com [IPv6:2607:f8b0:4864:20::830]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DRckQ2HYTz4sds; Thu, 28 Jan 2021 23:53:50 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-qt1-x830.google.com with SMTP id l23so5477929qtq.13; Thu, 28 Jan 2021 15:53:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=WJFNSvMBdVpfyLl0M2CBNJgNvupOByuSqUd2ep2DIuY=; b=AicjVQWQX5EZReAYxtbRTmS2f8IGhvYvX4Yzkw3v8pSuvrwEXK/uVSRerxegPnnK9/ G/Zu3Oj4fCLtHzjl7WPND+FBHUR54hnafCkca8hd2Ey+VvvCF5yiesmboEKL6w0w6qc+ jiD8xhCFNWsBCdiQsWCZ2one0bJ9bdrkLEZTg9FwXkQB0GrGuv4AyxI8Kh3Y5lOZ+r6j Et7CeULflhyaohJdLjKoKIKGjZ2VYYtuvvRY0arV9ats0Lultx56vgtKWUZHpTYtR52D jfJofOcFB7jml1JnE5mPjv4hYMUmo6WldpNgy/edmIljIUxuPjSCzj6WoDhHJ72F5wUt 8e2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=WJFNSvMBdVpfyLl0M2CBNJgNvupOByuSqUd2ep2DIuY=; b=jNNUogTalH0dJTFxnBqMRsIJU+GveZ/5rztXSwT4MUIcBcPTgL+fSh/UzSsPcpwa3N /fkOnD4FQYu59Jm3PRKrkdHI18ObxKJybTrphMyAb8I4nOMFzqs7C/eqcB8EdNIs9Gde fH3fku2ytC1kYY5xD0iaKmdtDYnjsvU2W4yvjXXA58laXIwwCXQ1yZCXMOk170FYQpRW y1bv2CQbO7zwHmM9cIKj56vuT4qkd7GHZ2gMrwoOLbQTbxepeuKqB9sAYTG2Q1ebrf8Z GXr3jxdRUQpJcimWdPbTAq8WeLIynIUgO+t5ifCLA2fSbZkKrDLbkZdUk3ohubz8RG1S u9Qw== X-Gm-Message-State: AOAM531f5oCz0f0i61DnFx5uuXhaaTWaWPHsIF6Ko3MAEzVv7ZNF+u0Y GckeRfa4lUeGTDtpWVwkoHypR0cnLpo5qhsPlhVXSIvT X-Google-Smtp-Source: ABdhPJyeTnxPI1rTfytbUuLgDNMR/j4Kgv7MrErWVw2fpQ/sBWagCmYO1clh/ZwzPb8bCtydX+HO3wTsjBUUR0ntxwo= X-Received: by 2002:ac8:6b44:: with SMTP id x4mr2060459qts.224.1611878028336; Thu, 28 Jan 2021 15:53:48 -0800 (PST) MIME-Version: 1.0 From: Freddie Cash Date: Thu, 28 Jan 2021 15:53:37 -0800 Message-ID: Subject: OpenZFS dRAID questions To: FreeBSD Filesystems , FreeBSD Stable X-Rspamd-Queue-Id: 4DRckQ2HYTz4sds X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=AicjVQWQ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of fjwcash@gmail.com designates 2607:f8b0:4864:20::830 as permitted sender) smtp.mailfrom=fjwcash@gmail.com X-Spamd-Result: default: False [-3.73 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::830:from]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::830:from:127.0.2.255]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::830:from]; NEURAL_HAM_SHORT(-0.73)[-0.725]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-fs,freebsd-stable]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2021 23:53:51 -0000 [Not sure which list is more applicable for this question, so sending to -fs and -stable. If it should be only one or the other, let me know.] Trying to get an understanding of how the dRAID vdev support works in ZFS, and what a good setup would be for a storage server using multiple JBODs full of SATA drives. Right now, my storage pools are made up of 6-disk raidz2 vdevs. So my 24-bay systems have 4x raidz2 vdevs, my 45-bay systems have 7x raidz2 vdevs with 3 (cold) spares, and my 90-bay systems have 15x raidz2 vdevs (1 vdev uses the 3 extra drives from each 45-drive JBOD). If I'm reading the dRAID docs correctly, instead of having multiple raidz2 vdevs in the pool, I'd have a single draid vdev, configured with children that are configured with similar data/parity devices to "mimic" raidz2? If that's correct, would it make sense to have a single draid vdev per pool (splitting the draid vdev across JBODs)? Or a single draid vdev per JBOD chassis (so 2x draid vdevs)? What's the practical limit for the number of drives in a single draid vdev? I have a brand new storage server sitting in a box with a 44-bay JBOD that will be going into the server room next week, and I'm tempted to try draid on it instead of the multiple-raidz2 setup. This would use 4+2 (data+parity) children with 2 spares, I believe. This server will be replacing a 90-bay system (2x JBODs), which will then be used as another storage server once all the dying drives are replaced. Will be interesting to see how draid works on that one as well, but not sure how to configure it. -- Freddie Cash fjwcash@gmail.com