Date: Tue, 12 Jan 2021 10:40:28 -0800 From: joe mcguckin <joe@via.net> To: Freddie Cash <fjwcash@gmail.com> Cc: FreeBSD-Current <freebsd-current@freebsd.org> Subject: Re: Preparing ZFS drives Message-ID: <A7304655-D620-4D1A-AA05-FE0D5CD95B29@via.net> In-Reply-To: <CAOjFWZ5=ctMwujLxr-p76GTJAqPVdhVsWwH3Ly_SHkJAXFNirQ@mail.gmail.com> References: <CF6CB87D-D1A4-4328-976D-31764C0BD1F2@via.net> <CAOjFWZ5=ctMwujLxr-p76GTJAqPVdhVsWwH3Ly_SHkJAXFNirQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Freddie, Thanks! I have to say that this is the most cogent explanation on how to = prepare ZFS drives that=20 anyone has ever given me. Thanks again! Joe Joe McGuckin ViaNet Communications joe@via.net 650-207-0372 cell 650-213-1302 office 650-969-2124 fax > On Jan 12, 2021, at 10:32 AM, Freddie Cash <fjwcash@gmail.com> wrote: >=20 > On Tue, Jan 12, 2021 at 10:10 AM joe mcguckin <joe@via.net = <mailto:joe@via.net>> wrote: > Folks, >=20 > I want to buy some 16TB drives and raid them up >=20 > How should I label and prepare the drives for ZFS? Someone ought to = write a =E2=80=98cookbook=E2=80=99 on that! >=20 > If these drives will be strictly data drives (not booting from them), = partitioning them is fairly easy. You will want to determine a = labelling system for them. Personally, I like to label the drives using = a grid system (columns are letters, rows are numbers). For systems with = multiple JBODs attached, I include which JBOD chassis their in as well. = For example, a 24-bay chassis would use disk-a1, disk-a2, disk-a3 ... = disk-d4, disk-d5, disk-d6. A system with 2 24-bay JBODs would use = jbod1-a1, jbod1-a2, jbod1-a3 ... jbod2-d4, jbod2-d5, jbod2-d6. So you = label the GPT partition on each disk, and build the pool using the GPT = partition labels. >=20 > gpart create -s gpt da0 > gpart add -t freebsd-zfs -a 1M -l disk-a1 da0 >=20 > gpart create -s gpt da1 > gpart add -t freebsd-zfs -a 1M -l disk-a2 da1 >=20 > And so on. Add 1 disk, partition/label it based on its location. = Then add the next disk. And so on. >=20 > Then use the GPT labels to create the pool (they show up as devices = under the /dev/gpt/ directory): >=20 > zpool create mypool mirror gpt/disk-a1 gpt/disk-a2 mirror gpt/disk-a3 = gpt/disk-a4 mirror gpt/disk-a5 gpt/disk-a6 >=20 >=20 > If you need to boot from these drives (make a root pool), then things = get more complicated. Personally, I'd recommend using the 16 TB drives = strictly for a data pool, and then use some smaller SSDs for a root = pool, in a simple mirror vdev setup. Separate the OS from the data. :) >=20 > Do I need to start the volume on a particular sector boundary? >=20 > The "-a 1M" argument for gpart does it for you. It aligns the = partition at the 1 MB boundary, and figures out which sector of the disk = that corresponds to based on the sector size of the disk (512B or 4K). > =20 > Are the 4096 byte sector drives usable? >=20 > Yeah, they work without issues. Try not to mix 512B and 4K drives = within a single vdev (it'll work, but may cause performance issues). = Mixing them in a pool (a vdev using 512B drives, another vdev using 4K = drives) is okay, so long as you set the vfs.zfs.min_auto_ashift sysctl = to 12 (force the minimum block size used by ZFS to be 4K). That way, in = the future, you can replace the 512B drives with 4K drives without any = performance issues. >=20 > You can check what the ashift value is for each of the vdevs using: = zdb | grep -B5 ashift >=20 > If it shows ashift=3D9 anywhere, then destroy the pool, change the = sysctl value, and recreate the pool. Check to make sure it shows = ashift=3D12 in zdb output. >=20 > --=20 > Freddie Cash > fjwcash@gmail.com <mailto:fjwcash@gmail.com>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A7304655-D620-4D1A-AA05-FE0D5CD95B29>