From owner-freebsd-geom@FreeBSD.ORG Wed Mar 4 14:38:29 2015 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2068E11E for ; Wed, 4 Mar 2015 14:38:29 +0000 (UTC) Received: from mail-we0-f175.google.com (mail-we0-f175.google.com [74.125.82.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A97DCC6E for ; Wed, 4 Mar 2015 14:38:28 +0000 (UTC) Received: by wevl61 with SMTP id l61so46807850wev.2 for ; Wed, 04 Mar 2015 06:38:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=SPTnpfx7hJzgoe1hfH6WS9kNxmienLNU3eQ3WyoUGMM=; b=ZGaABhWmUbQLY2Sj4DpbZQnA6DnRTOtczLH2GAYxZcOId/LMqMfS6lguOKND13Dhtz KJPEcsef0dexyOMy1sw0ER8/vvRKidceyeKbXnzAwpZTBXtsT/JLu2T7kEzIG1GZfcI5 IHMXpigx/ZBrY/JifEPYD/Z1PoTfoOmos6KUIui3LfGsyPNoIYHT2GETfVQHf99JWywt IeuJwDlOHdRJMpCiLi8AzmkmSt8XvHYzle3A8cczIDWyNCnvbaDpdq+Fx9L3Z9+KCi4Y QebTaaTBybb1HSledE8sxQ9TBs/s9DGEWw5fVjiHOcfKSvkAc1xrRoqFxY+LwJMRjWVG 7vTw== X-Gm-Message-State: ALoCoQl8G2Y+H1D4vuIfpeR4cHWuGO5AE2i2ZVf4AAVN8vXz5kSLq8wGCABhCnWG6L/EG1GfV+2y X-Received: by 10.194.193.9 with SMTP id hk9mr9090474wjc.42.1425479901152; Wed, 04 Mar 2015 06:38:21 -0800 (PST) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id m9sm25451960wiz.24.2015.03.04.06.38.20 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Mar 2015 06:38:20 -0800 (PST) Message-ID: <54F718D9.1080201@multiplay.co.uk> Date: Wed, 04 Mar 2015 14:38:17 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-geom@freebsd.org Subject: Re: Trim on gmirrored SSDs is slow and results in inresponsive system References: <54F6FE2E.60303@bytecamp.net> <54F707CC.6070105@multiplay.co.uk> <54F7147F.8070206@bytecamp.net> In-Reply-To: <54F7147F.8070206@bytecamp.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Mar 2015 14:38:29 -0000 On 04/03/2015 14:19, Robert Schulze wrote: > Hi, > > Am 04.03.2015 um 14:25 schrieb Steven Hartland: >> Do you have one disk which has really slow TRIM? >> > please note, it does not depend on the disk, it is related to gmirror. Yes but gmirror will wait for all disks to complete the delete before the top level delete is marked as complete, so if you have an issue with one of the underlying devices, be that cabling issue or disk on the way out, this could be your issue. >> Also how does gstat -d -p compare between gmirror and none gmirror >> installs on the same machine? > Deleting on a non-mirrored UFS does not influence the system, because > the BIO_DELETE calls are processed extremely fast (~ 1 sec with 1GB > file), with a high number of d/s (~ 100k). Can you get snapshot of both in action please > >> Do you see high kernel CPU when the deletes are happening? > no. Here are the top system processes after deleting a 5GB file: > > 0:40 144.87% cam > 0:22 77.59% g_mirror var > 0:14 62.79% bufdaemon > 0:14 0.88% intr > 0:13 0.59% geom > 0:01 0.00% rand_harvestq > 0:00 0.00% kernel 144% in cam is very high and g_mirror is high too. What OS build is this? http://svnweb.freebsd.org/changeset/base/267118 could well be the cause of high CAM CPU, this was one of main reasons why the original bypass or sorting was added. Regards Steve