From owner-freebsd-geom@FreeBSD.ORG Thu Mar 5 14:37:06 2015 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43BFABD4 for ; Thu, 5 Mar 2015 14:37:06 +0000 (UTC) Received: from mail-wg0-f52.google.com (mail-wg0-f52.google.com [74.125.82.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CB1EBFA4 for ; Thu, 5 Mar 2015 14:37:04 +0000 (UTC) Received: by wggx12 with SMTP id x12so53841589wgg.6 for ; Thu, 05 Mar 2015 06:36:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=JSBMsSnVAved2HaaS2bDJpMi1x0U9xzs2hnfsqeVfRc=; b=FRszq2ND/21s65HdgOHRKqOf/01nOKH386gkYqVAh1eqQ8VQ3RXzMz5aTvUsxsjE0P afO8uh/eNOkBGYCU4XrxmyF+IckOGTseqzWXtd3N/JsQxogKVCPd8jwMWzwaK4dZopiB M7s0nkxAmOo18odHQD8TCx6poBnfSFabs2ucgJtNOp1Y7KdDRotfQokUXmtTZZzARRTs 2JaBvjcb3ZiTbSQhkv4ZW6mnA++S7DkrdJFcrYgVd9vnVk+0nLVnBL1vbck5klxtCs0a mRpXKV2+QHXceTkdXgeX2Jgp3RLopNnmFsPdr/pR4pruz7GI0tWbyMxVbxs3DNqHN1/G brvg== X-Gm-Message-State: ALoCoQkMSS/FjT9faWSCwiq7M7Fp79MhMW5vPWPt96QbvKnW/wxtq1qyk+PjuGtF24dh3IgylNuX X-Received: by 10.180.105.40 with SMTP id gj8mr67714342wib.67.1425566216970; Thu, 05 Mar 2015 06:36:56 -0800 (PST) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id hv5sm10792300wjb.16.2015.03.05.06.36.55 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 05 Mar 2015 06:36:56 -0800 (PST) Message-ID: <54F86A06.4090301@multiplay.co.uk> Date: Thu, 05 Mar 2015 14:36:54 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-geom@freebsd.org Subject: Re: Trim on gmirrored SSDs is slow and results in inresponsive system References: <54F6FE2E.60303@bytecamp.net> <54F707CC.6070105@multiplay.co.uk> <54F7147F.8070206@bytecamp.net> <54F718D9.1080201@multiplay.co.uk> <54F721BC.5070204@bytecamp.net> <54F72E39.2070105@multiplay.co.uk> <54F863C9.3070002@bytecamp.net> In-Reply-To: <54F863C9.3070002@bytecamp.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 14:37:06 -0000 On 05/03/2015 14:10, Robert Schulze wrote: > Hi, > > Am 04.03.2015 um 17:09 schrieb Steven Hartland: >> Transaction sizes look like they are the same but the tps from gmirror >> is significantly lower. >> >> As iostat doesn't properly understand deletes it would still be nice to >> see that gstat -d -p output as that will also give us queue depth. > Since gstat doesnt just output row by row, I had to screenshot a running > gstat and transcribe the values. Running gstat -b in a loop doesn't help > here, since the system is inresponsive and won't start any program in > time during the DELETE ops. > > So after deleting a 2GB file on a gmirror with two active SSDs I get: > > # gstat -dpfada -I3s > > dT: 3.189s w: 3.000s filter: ada > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > 0 3342 0 0 0.0 0 0 0.0 3342 106939 10.2 80.5| ada0 > 0 3342 0 0 0.0 0 0 0.0 3342 106939 10.0 82.0| ada1 > > 9 2606 0 0 0.0 0 0 0.0 2606 83400 60.8 98.3| ada0 > 9 2606 0 0 0.0 0 0 0.0 2606 83400 56.9 98.7| ada1 > > 46 1705 0 0 0.0 0 0 0.0 1705 54565 62.5 98.4| ada0 > 51 1704 0 0 0.0 0 0 0.0 1704 54512 62.9 97.6| ada1 > > 22 1540 0 0 0.0 0 0 0.0 1540 49285 20.9 96.6| ada0 > 24 1541 0 0 0.0 0 0 0.0 1541 49317 21.8 96.0| ada1 > > 2 1496 0 0 0.0 0 0 0.0 1496 47887 13.3 98.6| ada0 > 7 1495 0 0 0.0 0 0 0.0 1495 47855 14.1 98.9| ada1 > > 11 1032 0 0 0.0 0 0 0.0 1032 33010 18.2 98.8| ada0 > 10 1033 0 0 0.0 0 0 0.0 1033 33070 15.4 98.1| ada1 > > 12 775 0 0 0.0 0 0 0.0 775 24798 26.2 97.5| ada0 > 16 773 0 0 0.0 0 0 0.0 773 24746 29.6 98.5| ada1 > > 65 743 0 0 0.0 0 0 0.0 743 23784 25.4 98.8| ada0 > 59 746 0 0 0.0 0 0 0.0 746 23887 24.4 99.0| ada1 These values look totally reasonable in terms a d/s. How does this compare to the none gmirror case? >> Try backing out r268816 and see if that has any impact. > could you please explain what you mean by that? > Compile a version of the kernel with this change reverted. Details of the change can be seen here. https://svnweb.freebsd.org/base?view=revision&revision=268816 You can download the changes (only ata is relevant in your case) and apply the patch with reverse option. Once you've done that recompile and re-install your kernel. This feels like it might be an request ordering issue. Regards Steve