From owner-freebsd-geom@FreeBSD.ORG Thu Apr 24 12:46:56 2014 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B06145A9 for ; Thu, 24 Apr 2014 12:46:56 +0000 (UTC) Received: from mail.tdx.com (mail.tdx.com [62.13.128.18]) by mx1.freebsd.org (Postfix) with ESMTP id 58A0D147D for ; Thu, 24 Apr 2014 12:46:55 +0000 (UTC) Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251]) (authenticated bits=0) by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s3OCkn9B090989 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 24 Apr 2014 13:46:49 +0100 (BST) Date: Thu, 24 Apr 2014 13:46:48 +0100 From: Karl Pielorz To: freebsd-geom@freebsd.org Subject: Anyone using HAST in production / performance? Message-ID: <2D7624DF74301FC17FEA865E@Mail-PC.tdx.co.uk> X-Mailer: Mulberry/4.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 12:46:56 -0000 Hi, I've been looking at HAST for a while (I posted here a while ago about performance). Is anyone using it in production? - How do you find the performance? On a 10 stable box (Xeon 1230v3/16Gb/LSI2308/amd64) - even if I setup HAST for async, no compression, no crc - and 'none' for the other node on a disk, simply changing '/dev/gpt/partition' for '/dev/hast/partition' loses around 50% performance. Running bonnie++ on a 'raw' UFS disk gives 133962K/sec. Same drive via HAST (with 'remote' set to none) drops to 72019K/sec. This was for a Crucial SSD (1Mbyte partition aligned). A SanDisk SSD shows a bigger drop from 230Mbyte/sec writes to 90Mbyte/sec. The same thing happens with regular spindle SATA disks as well (it makes their performance 'dismal'). Using the onboard SATA ports vs. the LSI doesn't make any difference either. Interestingly, reconfiguring HAST with an actual secondary node doesn't show any more noticeable performance loss (other node is via a 10Gbit connection - and I have verified replication is happening). It's just the initial change to HAST (even for a local, with no remote) that causes the huge hit. No combination of async, memsync etc. makes any difference. By comparison - putting GELI atop of the raw disk (with hardware crypto) shows a performance fall of around 1-2%! Any suggestions for what I can look at to check/change? -Karl