Date: Wed, 11 Dec 2019 17:39:03 +0000 From: Norman Gray <Norman.Gray@glasgow.ac.uk> To: FreeBSD questions list <freebsd-questions@freebsd.org> Subject: Adding to a zpool -- different redundancies and risks Message-ID: <6104097C-009B-4E9C-A1D8-A2D0E5FECADF@glasgow.ac.uk>
next in thread | raw e-mail | index | archive | help
Greetings. I'd like to add a new VDEV to a pool, and I'm being warned (slightly to=20 my surprise) that there's a 'mismatched replication level'. I'm trying=20 to get a sense of how much of a risk I'd be running by forcing this with=20 -f. Context: * I currently have two raidz2 VDEVs composed of nine 5.5TB disks=20 (thus 2 x ~40TB available) * I'd like to add another raidz2 VDEV composed of six 12TB disks=20 (thus adding a VDEV of ~48TB, roughly the same size as the other two) --=20 this is what prompts the warning about replication level * The storage is a local mirror which it would be very annoying to=20 lose, but it's not holding unique copies of anything * I don't like using -f options unless I'm pretty damn confident I=20 know what's happening (I wouldn't set this up in quite this way from scratch, but this is an=20 old-ish server, and a small budget windfall has allowed me to max out=20 the remaining available slots with new disks). I can appreciate that the ideal planned setup would, in principle, be to=20 have all the VDEVs be symmetrical, in terms of size and number of disks.=20 Is a VDEV mix merely 'not ideal', or 'not great but you'll be fine',=20 or Bad? My mental model of what's going on suggests that, since the pool simply=20 stripes across the VDEVs, it doesn't have to care how the VDEVs=20 themselves are structured, so that a 9x5.5 raidz2 and a 6x12 raidz2=20 would be roughly equally used, and I can't see why there would be a=20 performance or a utilisation difference between the two (but I still=20 count myself as a ZFS tyro). I can see that there would be a reliability issue if the various VDEVs=20 were different sizes of mirrored ones -- this would create different=20 amounts of resilience, and so the warning makes sense in a 'are you=20 sure?' way. If the VDEVs were different sizes and the pool was=20 mirroring over them, then there would obviously be a utilisation issue. Though both of [1] and [2] illustrate only mixing VDEVs of the same=20 type, [2] says merely that 'When using RAIDZ vdevs, it is also a good=20 idea to keep them at the same width and of the same type.' and=20 illustrates a 4-wide plus 8-wide raidz2 as 'not horrible'. The forum=20 post at [3] asks essentially the same question as this email, but=20 receives a rather oblique answer. The question at [4] gets a confident=20 answer which I don't _think_ makes complete sense. Since [1] and [2] are both more authoritative and match my own=20 understanding, I'm inclined to believe that adding this new VDEV would=20 be less than perfect, but reasonable. Am I deceiving myself? Thanks for any advice you can offer, Norman [1]=20 https://forums.freenas.org/index.php?threads/slideshow-explaining-vdev-zpoo= l-zil-and-l2arc-for-noobs.7775/ [2]=20 https://www.ixsystems.com/community/resources/introduction-to-zfs.111/ [3]=20 https://forums.freebsd.org/threads/zfs-mismatched-repli-levels.28226/ [4] https://serverfault.com/questions/522782/zfs-with-unsymmetric-vdevs --=20 Norman Gray : https://nxg.me.uk SUPA School of Physics and Astronomy, University of Glasgow, UK
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6104097C-009B-4E9C-A1D8-A2D0E5FECADF>