Date: Mon, 28 Feb 2011 22:35:24 -0500 From: Bill Desjardins <bill@ethernext.com> To: freebsd-fs@freebsd.org Subject: HAST, zvols, istgt and carp working... Message-ID: <AANLkTim5FRJkf_S0aSV74S=JY%2Bg1DBZLhjYW7X9C0MkP@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hello All,
as an experiment today I setup a couple 8-stable guests on vmware ESXI to test
hast with zfs, carp and istgt for a redundant nas system I am putting
together.
I havent seen any mention that anyone has used hast to mirror a zfs zvol so I
figured I would try it and atleast my proof of concept seems to work just fine.
is anyone doing this and using it in a production environment?
heres how I setup the testing environment...
- created (2) 8-stable hosts on esxi 4.1: hast-1 & hast-2 (os on da0)
on both hast-1 and hast-2
- added 4 x 8GB disk's to each (da1 - da4)
- glabel'd disks disk1 - disk4
- zpool create tank mirror disk1 disk2 mirror disk3 disk4
- zfs create -p -s -b 64k -V 4G tank/hzvol.1
hast.conf on each
resource tank_hzvol.1 {
local /dev/zvol/tank/hzvol.1
on hast-1 {
remote x.x.x.9
}
on hast-2 {
remote x.x.x.8
}
}
on hast-1 and hast-2
> hastd_enable="YES" in rc.conf
> hastctl create tank_hzvol.1
> /etc/rc.d/hastd start
on hast-2
> hastctl role secondary tank_hzvol.1
on hast-1
> hastctl role primary tank_hzvol.1
hastctl status reports all is well so far...
next I configured istgt identically on hast-1 and hast-2 for the hast device
>> LUN0 Storage /dev/hast/tank_hzvol.1 3G
istgt was started (istgt onestart) and the zvol target was setup on
another vmware esxi server
which then was formatted as a vmfs volume. I created a 2GB disk on this volume
and added it to another 8-stable host as a ufs disk mounted on /mnt. so far
going good, everything working as expected.
to test hast replication, I created a few 200MB files on the host with the
ufs vmdk volume and seen traffic over the hast network from hast-1 to hast-2. on
hast-1, the zvol size reflected correct sparse disk space usage, but
hast-2 showed
the full 4GB zvol allocated which I suspect is due to hast.
to test failover of the isci zvol target from hast-1 to hast-2:
on hast-1
> istgt stop
> hastctl role secondary tank_hzvol.1
on hast-2
> hastctl role primary tank_hzvol.1
> istgt onestart
NOTE: carp does not seem to work on esxi for me so between hast-1 and hast-2 I
manually moved the IP for istgt to hast-2.
the result was that the istgt hast zvol successfully failed over to hast-2
with only a brief stall while I manually performed the failover process. I only
performed the ideal manual failover scenario for proof of concept. I will be
testing this on 2 real development servers later this week for a more
complete understanding.
I see some real advantages for zvols only hast:
++++++++++++++++++++++++++++++++++++++++++++++++
+ no need to hast each individual disk in the zpool so you can access all
available storage on either storage unit
+ maintaining storage units remains functionally consistent between them
+ once setup, zvols are easily migrated to new storage environments in real-time
since there is only a single zvol hast resource to replicate. (no need
to have all
matching zpool hast members, just reconfigure the primary zvol hast
resource to point to
a new secondary server and swap roles/failover when ready)
+ can have active hast zvols on each unit to distribute IO
+ no need for zpool export/import on failover
+ hast easily added to current zvols
+ retains performance of entire zpool
+ zpool can be expanded without changing hast config
+ minimizes hast replication traffic between storage units
+ hast split-brain localized to specific zvol's
+ can use ufs on hast zvol resource for things like samba and nfs
cons
-------------------------------------------
- performace impact (???)
- each hast zvol requires distinct application configurations (more
confgurations to deal with/screw up)
- zfs sparse volumes seem not to be working correctly via hast (???)
- expanding zvol requires hastctl create, init, startup plus may need
application specific changes/restart.
- other methods needed to replicate data in rest of pool
- possible long rebuild time on large zvols?
- snapshots / rollbacks (???)
- many more???
my main question is if using hast to replicate a zvol is a supported
configuration and what are the possible drawbacks? Its more than
likely I am overlooking some very basic requirement/restrictions and
am blatantly wrong in all this, but if it can perform, I think its a big+
for freebsd and zfs useability as a nas server.
thoughts? comments? criticisms? :)
Best,
Bill
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTim5FRJkf_S0aSV74S=JY%2Bg1DBZLhjYW7X9C0MkP>