From owner-freebsd-stable@FreeBSD.ORG Fri Jan 16 09:33:08 2015 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C565D9F8 for ; Fri, 16 Jan 2015 09:33:08 +0000 (UTC) Received: from mail-wi0-x235.google.com (mail-wi0-x235.google.com [IPv6:2a00:1450:400c:c05::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 559355EE for ; Fri, 16 Jan 2015 09:33:08 +0000 (UTC) Received: by mail-wi0-f181.google.com with SMTP id hi2so2580165wib.2 for ; Fri, 16 Jan 2015 01:33:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=j9ee4bCW6G+6gMojTywdJnjHHsEme2muTzLKkly/1g0=; b=SIZxtILY2JDpz0/aLUdSklBXWwXaIijs/MVaW6iiDBQQs+dGLSz814aCEEDy2NHMvK flzO+3TbsK27QrJs0gHobYZnDWLNZyCfLzJTaXN3KkHYH9PJNBPCOsDon89VUkw3AHak piwr8lN537r3QrTgh2/T1DCNAoakT1x+XeoqtMLFteVGpIFNoIFTZ0bTGayPl6QTS6Xh qBjQ6twQGPFDedEysEVIPGgNEPmANsLhZkF6zcOQU71MLCNG1hOdg74o8mrzlVw5rNYK Tm417iSycFFpwdP0tr4PxkCWFI+w+gYom48g56mWGzi2VISQg6SRR6WFBCn4zoTT5tTo R33Q== X-Received: by 10.180.93.1 with SMTP id cq1mr4368150wib.2.1421400786025; Fri, 16 Jan 2015 01:33:06 -0800 (PST) Received: from [192.168.3.105] ([193.148.0.35]) by mx.google.com with ESMTPSA id e4sm5255741wjw.48.2015.01.16.01.33.05 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Jan 2015 01:33:05 -0800 (PST) Message-ID: <54B8DAD0.5040606@gmail.com> Date: Fri, 16 Jan 2015 11:33:04 +0200 From: Mihai Vintila User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: Re: Poor performance on Intel P3600 NVME driver References: <54B7F769.40605@gmail.com> <20150115175927.GA19071@zxy.spb.ru> <54B8C7E9.3030602@gmail.com> <54B8CEF4.9040206@gmail.com> <20150116091317.GA83755@zxy.spb.ru> In-Reply-To: <20150116091317.GA83755@zxy.spb.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 09:33:08 -0000 I've tried it and are similar. Performance on this should be in the 450/70 read/write on 4k and i'm getting 120/40. Best regards, Vintila Mihai Alexandru On 1/16/2015 11:13 AM, Slawa Olhovchenkov wrote: > On Fri, Jan 16, 2015 at 10:42:28AM +0200, Mihai Vintila wrote: > >> Actually seems that i've was wrong vfs.zfs.top_maxinflight does not have >> any influence and this is confirmed by the nvmecontrol perftests. >> nvmecontrol perftest -n 32 -o read -s 512 -t30 nvme0ns1 >> Threads: 32 Size: 512 READ Time: 30 IO/s: 270212 MB/s: 131 >> nvmecontrol perftest -n 32 -o write -s 512 -t30 nvme0ns1 >> Threads: 32 Size: 512 WRITE Time: 30 IO/s: 13658 MB/s: 6 > I see datasheet specified performance metrics for 4K (not 512bytes) random r/w. > Can you try this? This is may be impact write IO/s at least. > >> I was able to recover from the errors from previous message. They were >> cause by the fact that i've commented hw.nvme.per_cpu_io_queues=0 in >> loader.conf . After setting it back things got back to "slow normal" >> Performance is half of what it should be and issue seems to be the nvme >> driver. I'll try it on another OS to confirm it's not a hardware setting >> issue.