Date: Wed, 11 Dec 2019 17:39:03 +0000 From: Norman Gray <Norman.Gray@glasgow.ac.uk> To: FreeBSD questions list <freebsd-questions@freebsd.org> Subject: Adding to a zpool -- different redundancies and risks Message-ID: <6104097C-009B-4E9C-A1D8-A2D0E5FECADF@glasgow.ac.uk>
next in thread | raw e-mail | index | archive | help
Greetings. I'd like to add a new VDEV to a pool, and I'm being warned (slightly to my surprise) that there's a 'mismatched replication level'. I'm trying to get a sense of how much of a risk I'd be running by forcing this with -f. Context: * I currently have two raidz2 VDEVs composed of nine 5.5TB disks (thus 2 x ~40TB available) * I'd like to add another raidz2 VDEV composed of six 12TB disks (thus adding a VDEV of ~48TB, roughly the same size as the other two) -- this is what prompts the warning about replication level * The storage is a local mirror which it would be very annoying to lose, but it's not holding unique copies of anything * I don't like using -f options unless I'm pretty damn confident I know what's happening (I wouldn't set this up in quite this way from scratch, but this is an old-ish server, and a small budget windfall has allowed me to max out the remaining available slots with new disks). I can appreciate that the ideal planned setup would, in principle, be to have all the VDEVs be symmetrical, in terms of size and number of disks. Is a VDEV mix merely 'not ideal', or 'not great but you'll be fine', or Bad? My mental model of what's going on suggests that, since the pool simply stripes across the VDEVs, it doesn't have to care how the VDEVs themselves are structured, so that a 9x5.5 raidz2 and a 6x12 raidz2 would be roughly equally used, and I can't see why there would be a performance or a utilisation difference between the two (but I still count myself as a ZFS tyro). I can see that there would be a reliability issue if the various VDEVs were different sizes of mirrored ones -- this would create different amounts of resilience, and so the warning makes sense in a 'are you sure?' way. If the VDEVs were different sizes and the pool was mirroring over them, then there would obviously be a utilisation issue. Though both of [1] and [2] illustrate only mixing VDEVs of the same type, [2] says merely that 'When using RAIDZ vdevs, it is also a good idea to keep them at the same width and of the same type.' and illustrates a 4-wide plus 8-wide raidz2 as 'not horrible'. The forum post at [3] asks essentially the same question as this email, but receives a rather oblique answer. The question at [4] gets a confident answer which I don't _think_ makes complete sense. Since [1] and [2] are both more authoritative and match my own understanding, I'm inclined to believe that adding this new VDEV would be less than perfect, but reasonable. Am I deceiving myself? Thanks for any advice you can offer, Norman [1] https://forums.freenas.org/index.php?threads/slideshow-explaining-vdev-zpool-zil-and-l2arc-for-noobs.7775/ [2] https://www.ixsystems.com/community/resources/introduction-to-zfs.111/ [3] https://forums.freebsd.org/threads/zfs-mismatched-repli-levels.28226/ [4] https://serverfault.com/questions/522782/zfs-with-unsymmetric-vdevs -- Norman Gray : https://nxg.me.uk SUPA School of Physics and Astronomy, University of Glasgow, UK
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6104097C-009B-4E9C-A1D8-A2D0E5FECADF>