From owner-freebsd-geom@FreeBSD.ORG Thu Mar 5 15:45:36 2015 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7D134995 for ; Thu, 5 Mar 2015 15:45:36 +0000 (UTC) Received: from mail-wi0-f179.google.com (mail-wi0-f179.google.com [209.85.212.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0E518AAF for ; Thu, 5 Mar 2015 15:45:35 +0000 (UTC) Received: by wivr20 with SMTP id r20so7755607wiv.3 for ; Thu, 05 Mar 2015 07:45:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=ixU5L/WnVjXtvhYlh27erMqrbScRiaSJh9IK/hCUD/Q=; b=d4W29K6W/kGP1l1jVRFASL1c3qIHVB4m8aybxE1CqA9f9mPGSWYzg4Gjt/ZmrT44Sm J3BrzCmEifJodxf1FwXrM0QD1PwNo3yEUSOi+IhMvGGvJ+lXLSdldaxFXiYxDu2Gtr3O qIZDi6H4YbV/6ye4KRgGMhBjnbBWz3sz6rUyYpXVO6Q0rfy0LyT1pg85vCuHgFvWGYHF 0jHDPFE57yWe4miBzwW7g/T+Ejjv50hiW5bZuhxszcAH2MCaXRKes1sJ8tidIqY3GQE/ /qgze30Ud7dOjYIJyNw+WE/xrv3d5eN2Jz3d4yWom6HQLPWDe9o67oK5rFfs6D/ZyyUW 1Pgw== X-Gm-Message-State: ALoCoQnjDBeJ/gNCV53GYensSYcBjhkH2Lt5cG64A+em4KUFVz8qL4SHgu5GpqZ7D797ml+STmGM X-Received: by 10.194.120.40 with SMTP id kz8mr20045693wjb.21.1425570328231; Thu, 05 Mar 2015 07:45:28 -0800 (PST) Received: from [10.10.1.68] (82-69-141-170.dsl.in-addr.zen.co.uk. [82.69.141.170]) by mx.google.com with ESMTPSA id v5sm8979955wiw.24.2015.03.05.07.45.26 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 05 Mar 2015 07:45:27 -0800 (PST) Message-ID: <54F87A14.8070504@multiplay.co.uk> Date: Thu, 05 Mar 2015 15:45:24 +0000 From: Steven Hartland User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-geom@freebsd.org Subject: Re: Trim on gmirrored SSDs is slow and results in inresponsive system References: <54F6FE2E.60303@bytecamp.net> <54F707CC.6070105@multiplay.co.uk> <54F7147F.8070206@bytecamp.net> <54F718D9.1080201@multiplay.co.uk> <54F721BC.5070204@bytecamp.net> <54F72E39.2070105@multiplay.co.uk> <54F863C9.3070002@bytecamp.net> <54F86A06.4090301@multiplay.co.uk> <54F8734A.40409@bytecamp.net> In-Reply-To: <54F8734A.40409@bytecamp.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Mar 2015 15:45:36 -0000 On 05/03/2015 15:16, Robert Schulze wrote: > Hi, > >> These values look totally reasonable in terms a d/s. >> >> How does this compare to the none gmirror case? > Its barely measurable, but in this case I can start a loop directly > after unlink with very small interval. The only measurement with > non-null values is: > > dT: 0.205s w: 0.200s filter: ada > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0 ada0 > 0 51537 5 156 2.4 0 0 0.0 51532 1649018 3.8 18.2 ada1 > > thats all. It takes not even half a second to push the DELETE ops to the > SSD in case of non-gmirrored UFS. Again, with a 2 GB file. That is a high rate, but not inconceivable. >> Compile a version of the kernel with this change reverted. >> >> Details of the change can be seen here. >> https://svnweb.freebsd.org/base?view=revision&revision=268816 > done, but it did not change behaviour. > Thanks, I think we need to do some dtrace instrumentation, to get a better picture of what's happening. If you pick a smaller file, say 10MB what does the following dtrace script report for the single disk no gmirror and for a single disk gmirror (if thats possible) #!/usr/sbin/dtrace -s #pragma D option quiet enum bio_type { BIO_DELETE = 0x04 }; fbt::adastrategy:entry /((struct bio *)arg0)->bio_cmd == BIO_DELETE/ { bp = (struct bio *)arg0; periph = (struct cam_periph *)bp->bio_disk->d_drv1; softc = (struct ada_softc *)periph->softc; printf("%Y ada%d bcount: %d, offset: %d\n", walltimestamp, periph->unit_number, bp->bio_bcount, bp->bio_pblkno); }