Date: Tue, 22 Jan 2019 13:04:45 +0100 From: Borja Marcos <borjam@sarenet.es> To: andy thomas <andy@time-domain.co.uk> Cc: Ireneusz Pluta <ipluta@wp.pl>, freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: ZFS on Hardware RAID Message-ID: <B4991FA9-7E94-4994-BDDB-EC59AF6DB960@sarenet.es> In-Reply-To: <alpine.BSF.2.21.1901211548570.24493@mail0.time-domain.co.uk> References: <1180280695.63420.1547910313494.JavaMail.zimbra@gausus.net> <92646202.63422.1547910433715.JavaMail.zimbra@gausus.net> <CAOeNLurgn-ep1e=Lq9kgxXK%2By5xqq4ULnudKZAbye59Ys7q96Q@mail.gmail.com> <alpine.BSF.2.21.1901200834470.12592@mail0.time-domain.co.uk> <ee6353dc-161f-407e-d976-71ca652970a0@wp.pl> <alpine.BSF.2.21.1901211548570.24493@mail0.time-domain.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 22 Jan 2019, at 12:15, andy thomas <andy@time-domain.co.uk> wrote: > > > Yesterday I set up a spare Dell 2950 with Perc 5/i Integrated HBA and six 73 GB SAS disks, with the first two disks configured as a RAID 1 system disk (/dev/mfid0) and the remaining 4 disks as RAID 0 (mfid1- mfid4). After adding a freebsd-zfs GPT partition to each of these 4 disks I then created a RAIDz1 pool using mfid1p1, mfid2p1 and mfid3p1 with mfid4p1 as a spare, followed by creating a simple ZFS filesystem. > > After copying a few hundred MB of files to the ZFS filesystem, I yanked /dev/mfid3 out to simulate a disk failure. I was then able to manually detach the failed disk and replace it with the spare. Later, after pushing /dev/mfid3 back in followed by a reboot and scrubbing the pool, mfid4 automatically replaced the former mfid3 that was pulled out and mfid3 became the new spare. You shouldn’t require a reboot. If the actual targets are exposed to the CAM layer your disks will appear as “da” (SAS backplane) and you can offline a device, hot plug a new one, at most do a “camcontrol rescan” to detect it and run a zfs replace (or whatever) without stopping the system. If your drives are “mfid” devices you may need either a reboot or some magic rituals using “sas2ircu” or “sas3ircu” to have the controller recognize the new drive and accept it as a valid volume. > So a spare disk replacing a failed disk seems to be semi-automatic in FreeBSD (this was version 10.3) although I have seen fully automatic replacement on a Solaris parc platform. There are several stages at play here: 1- Starting up and recognizing a SAS or SATA drive. 2- Having it recognized as a volume by a RAID card. With LSI cards and single disk RAID0 volumes it may require a reboot or using the sas2ircu/sas3ircu utility. 3- ZFS replacement, which on Solaris can be automatic and on FreeBSD is done manually (I haven’t tried zfsd yet). Borja.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B4991FA9-7E94-4994-BDDB-EC59AF6DB960>