Date: Thu, 31 Aug 2017 13:33:30 +0200 From: Julien Cigar <julien@perdition.city> To: Frank Leonhardt <freebsd-doc@fjl.co.uk> Cc: freebsd-questions@freebsd.org Subject: Re: Fun with HAST and inter-host connections Message-ID: <20170831113330.GA27165@mordor.lan> In-Reply-To: <6cc75798-b7f2-b794-faec-8807616fd7f4@fjl.co.uk> References: <6cc75798-b7f2-b794-faec-8807616fd7f4@fjl.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
--T4sUOijqQbZv57TR Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 30, 2017 at 10:40:03PM +0100, Frank Leonhardt wrote: > Please note - I'm pushing what can be done with commodity hardware for=20 > amusement here, and trying to do interesting things with FreeBSD. I *do*= =20 > want to do it this way. >=20 > Okay, so the game is this: I've got some identical hosts with disks and= =20 > I've been working on ways of clustering them for a long time. It's what= =20 > I do for fun, right? >=20 > I'm currently playing with failover storage. Not necessarily with HAST;= =20 > iSCSI is fun and I'm messing with geom stuff in general. But let's stick= =20 > with HAST as it illustrates the dilemma. >=20 > When I started this game, 1Gb Ethernet was blistering. Now it's not so=20 > hot. How do I "network" the hosts with as much throughput as possible=20 > (with IP sockets, preferably)? Options: >=20 > 1) 10Gb Ethernet is expensive. 10Gb switches even more so, but I can do= =20 > point-to-point. >=20 > 2) LAG is more about failover than speed. And anyway, you end up needing= =20 > a lot of Ethernet ports on each host and it soon gets crazy. >=20 > 3) Fibrechannel - if I bought a few old (cheap) fiberchannel cards, I'm= =20 > not sure how I could use them point-to-point. Is this possible and has=20 > anyone done it? >=20 > 4) USB 3.1. 10Gb. PCIe cards cost about $30. Now this sounds fun. IP=20 > over USB anyone? >=20 > Now please don't advise me to get a pair of fabric switches and do the=20 > job properly. That's not what this is about. I want to see if it's=20 > possible to make a fast(er) storage solution using cheap components. A=20 > sort of Redundant Array of Inexpensive NAS. >=20 > Thoughts anyone? In particular, is the USB 3.1 idea crazy? And is anyone= =20 > else crazy enough to be trying the same thing? I've set up a low-cost redundant storage here at work with ZFS + iSCSI, basically a zpool mirror over 2 local disks and 2 iSCSI disks. Failover is done through CARP and devd. It works well (except when Jumbo Frames are turned on) for almost a year now. Just for extra security, and also because some people (smarter than me in this area) advised me against=20 this approach, I've setup a third node with ZFS replication (zrep). Basically if you can afford to lose "some" data, I would suggest to go with ZFS replication. >=20 > Thanks, Frank. >=20 >=20 > _______________________________________________ > freebsd-questions@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.o= rg" --=20 Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. --T4sUOijqQbZv57TR Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEE7vn2l0to0nV7EWolsrs3EKIEI8AFAlmn9AcACgkQsrs3EKIE I8B76RAAhcTD9BTjYcBOnk4nTjSveSTs6dL5bvMwtqPiaAodU/XDmVYnNaAIMBe+ FyGZOJgpIE9pAPPhtft/rpv/9gR6VDw4tzxQHWG+m5ShVtlyNbMt+a6D8uCaWZGq o7fImWNi0ir4sthtDoUE40zvCAhGJrcEOjCc9JjI9fk1pSoHH6Gc4VTMI8tcZ7Q2 JqLC0axEp+Nd3aI42Po6xApdg89DX8u8QIgvWDgNZoewtQEyi9QMvvgH0SFu5dbK D2MI7FnksxtPcD5hK1u9dcB3nQjWi9NUdEZIFUpKEgTO3OchhJ39MzzsqkEo6x7o Enu8JlA15BbtvWYYPJQ5QLabzArK/dwfxSTrDIb5NMbLoKK0BYGM7auys1S2bYuu LSLqHVSP32j3KSedYVwqrFienOki1uF9vLwJobamZ7kXQ8hTkZiRySdudAr8Dciv WBVBVTl8ohpC90sce/pSZ5fTbGfNV+msPEK1WnulxNOQBYmZCvvzBYdXJQBle8Rs IAWxXQumlcOXeTD5qaGEx4rttik+PHf6VJZgRLnvoKYLFiG9MoEpsJGVzpniUPh5 ldKaktVHa56g79MSAY1rM8ORw326bHJNWbRA71uBUqiyzIhsFVqllYhsqIzkVIQj xGFjooWhZ1P89mpvO6zkVZ8p2NjffWQmNv0WVZZwFNCokcOIG0E= =O5Hm -----END PGP SIGNATURE----- --T4sUOijqQbZv57TR--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170831113330.GA27165>