Date: Tue, 21 Jan 2014 15:46:09 +0100 From: Joar Jegleim <joar.jegleim@gmail.com> To: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: hast and zfs Message-ID: <CAFfb-hq4R-f7yNCbAGS9X8wW4FcYY8%2B_jyxsqRsPxcnkYtEA7g@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
--f46d04428fd0a7cdf204f07c1106 Content-Type: text/plain; charset=UTF-8 Hi list ! I've setup hast with zfs on 9.1-RELEASE-p10 I've got an lsi hba connected to an external sas hp msa that has 20 disks in it, same setup on both nodes. I've created a hast disk dev such as /dev/hast/disk1a, /dev/hast/disk1b etc... ( see attached hast.conf ) And built a zpool mirror accross the 20 disks. While setting up this, and testing, all looked pretty good, I've tried numerous failovers and so on. When I just switched over to this setup in production, it worked for about half an hour until the primary node got a zfs hang where zpool list hung and hastctl status hung completely . There was no output in dmesg nor /var/log/messages that said anything related to this (?) . The server serves some 3 000 000 jpegs for a site, I see anything from 50 to just above 1000 iops, though the average iops is bellow 100. It's completely random what pictures are being fetched at any time. Could there be some sysctls to tune related to this setup mabye ? Anybody using hast and zfs in production got any tip ? -- ---------------------- Joar Jegleim Homepage: http://cosmicb.no Linkedin: http://no.linkedin.com/in/joarjegleim fb: http://www.facebook.com/joar.jegleim AKA: CosmicB @Freenode ---------------------- --f46d04428fd0a7cdf204f07c1106 Content-Type: application/octet-stream; name="hast.conf" Content-Disposition: attachment; filename="hast.conf" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hqp9zo6g0 Y2hlY2tzdW0gc2hhMjU2CnJlcGxpY2F0aW9uIGZ1bGxzeW5jCmxpc3RlbiAxOTIuMTY4Ljc3LjEx CiN0aW1lb3V0IDUKCnJlc291cmNlIGRpc2sxYSB7CiAgICAgICAgb24gc3RvcmUwMSB7CiAgICAg ICAgICAgICAgICBsb2NhbCAvZGV2L2RhMgogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIK ICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rl di9kYTIKICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAxCiAgICAgICAgfQp9CgpyZXNvdXJj ZSBkaXNrMWIgewogICAgICAgIG9uIHN0b3JlMDEgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rl di9kYTMKICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAyCiAgICAgICAgfQogICAgICAgIG9u ICBzdG9yZTAyIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGEzCiAgICAgICAgICAgICAg ICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazJhIHsKICAgICAgICBv biBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGE0CiAgICAgICAgICAgICAg ICByZW1vdGUgc3RvcmUwMgogICAgICAgIH0KICAgICAgICBvbiAgc3RvcmUwMiB7CiAgICAgICAg ICAgICAgICBsb2NhbCAvZGV2L2RhNAogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDEKICAg ICAgICB9Cn0KCnJlc291cmNlIGRpc2syYiB7CiAgICAgICAgb24gc3RvcmUwMSB7CiAgICAgICAg ICAgICAgICBsb2NhbCAvZGV2L2RhNQogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAg ICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9k YTUKICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAxCiAgICAgICAgfQp9CgpyZXNvdXJjZSBk aXNrM2EgewogICAgICAgIG9uIHN0b3JlMDEgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9k YTYKICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAyCiAgICAgICAgfQogICAgICAgIG9uICBz dG9yZTAyIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGE2CiAgICAgICAgICAgICAgICBy ZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazNiIHsKICAgICAgICBvbiBz dG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGE3CiAgICAgICAgICAgICAgICBy ZW1vdGUgc3RvcmUwMgogICAgICAgIH0KICAgICAgICBvbiAgc3RvcmUwMiB7CiAgICAgICAgICAg ICAgICBsb2NhbCAvZGV2L2RhNwogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDEKICAgICAg ICB9Cn0KCnJlc291cmNlIGRpc2s0YSB7CiAgICAgICAgb24gc3RvcmUwMSB7CiAgICAgICAgICAg ICAgICBsb2NhbCAvZGV2L2RhOAogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAg ICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTgK ICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAxCiAgICAgICAgfQp9CgpyZXNvdXJjZSBkaXNr NGIgewogICAgICAgIG9uIHN0b3JlMDEgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTkK ICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAyCiAgICAgICAgfQogICAgICAgIG9uICBzdG9y ZTAyIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGE5CiAgICAgICAgICAgICAgICByZW1v dGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazVhIHsKICAgICAgICBvbiBzdG9y ZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGExMAogICAgICAgICAgICAgICAgcmVt b3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAg ICAgbG9jYWwgL2Rldi9kYTEwCiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQogICAgICAg IH0KfQoKcmVzb3VyY2UgZGlzazViIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAgICAgICAg ICAgIGxvY2FsIC9kZXYvZGExMQogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAg ICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTEx CiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlz azZhIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGEx MgogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24gIHN0 b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTEyCiAgICAgICAgICAgICAgICBy ZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazZiIHsKICAgICAgICBvbiBz dG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGExMwogICAgICAgICAgICAgICAg cmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAg ICAgICAgbG9jYWwgL2Rldi9kYTEzCiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQogICAg ICAgIH0KfQoKcmVzb3VyY2UgZGlzazdhIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAgICAg ICAgICAgIGxvY2FsIC9kZXYvZGExNAogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAg ICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9k YTE0CiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2Ug ZGlzazdiIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYv ZGExNQogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24g IHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTE1CiAgICAgICAgICAgICAg ICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazhhIHsKICAgICAgICBv biBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGExNgogICAgICAgICAgICAg ICAgcmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAg ICAgICAgICAgbG9jYWwgL2Rldi9kYTE2CiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQog ICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazhiIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAg ICAgICAgICAgIGxvY2FsIC9kZXYvZGExNwogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIK ICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rl di9kYTE3CiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3Vy Y2UgZGlzazlhIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9k ZXYvZGExOAogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAg b24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTE4CiAgICAgICAgICAg ICAgICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazliIHsKICAgICAg ICBvbiBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGExOQogICAgICAgICAg ICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAg ICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTE5CiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUw MQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazEwYSB7CiAgICAgICAgb24gc3RvcmUwMSB7CiAg ICAgICAgICAgICAgICBsb2NhbCAvZGV2L2RhMjAKICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9y ZTAyCiAgICAgICAgfQogICAgICAgIG9uICBzdG9yZTAyIHsKICAgICAgICAgICAgICAgIGxvY2Fs IC9kZXYvZGEyMAogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDEKICAgICAgICB9Cn0KCnJl c291cmNlIGRpc2sxMGIgewogICAgICAgIG9uIHN0b3JlMDEgewogICAgICAgICAgICAgICAgbG9j YWwgL2Rldi9kYTIxCiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMgogICAgICAgIH0KICAg ICAgICBvbiAgc3RvcmUwMiB7CiAgICAgICAgICAgICAgICBsb2NhbCAvZGV2L2RhMjEKICAgICAg ICAgICAgICAgIHJlbW90ZSBzdG9yZTAxCiAgICAgICAgfQp9CgoK --f46d04428fd0a7cdf204f07c1106--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFfb-hq4R-f7yNCbAGS9X8wW4FcYY8%2B_jyxsqRsPxcnkYtEA7g>