From owner-svn-src-head@freebsd.org Tue Dec 8 22:01:24 2015 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37A909A0DE8 for ; Tue, 8 Dec 2015 22:01:24 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CAC7F18EA for ; Tue, 8 Dec 2015 22:01:23 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: by wmww144 with SMTP id w144so47836786wmw.0 for ; Tue, 08 Dec 2015 14:01:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=from:subject:to:references:cc:message-id:date:user-agent :mime-version:in-reply-to:content-type; bh=vvmQt7olSgepxMcjybXza0JE0E+58s+UJ0yiT9SP+Gc=; b=U4NrjV/RnOZhVzxS1Ovr24D2XeVE1COAGHOJQYEV4BUvvGn/DK3J+kJr1h1bcyCAwL Kd7CmmVYvxr0XeXZccZIpiUs3rA/zmnfcxze5T+D6/ULsYysaXsdpC/B3n/NMr41aZyN DGLNm4FAUXZtRXDMLExa2ITm/iwxFEGNDDp6Wg3mumVlq8SZ+0HY/bXuXIPtjfjy3mK+ wJgh5FgqWbDxnfRtH/GZWcfV4YmiSdD4tLP0J4LMdNsAZJhbBp5XQkX9fcoFSRzvaQAW z6S6acyCJY4k5/6N/aPqEtDnFJAGiue4MPtZkTpvr5KRB5QUAvM/ZS3mhzZBvi14N0V/ NjfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:subject:to:references:cc:message-id:date :user-agent:mime-version:in-reply-to:content-type; bh=vvmQt7olSgepxMcjybXza0JE0E+58s+UJ0yiT9SP+Gc=; b=U0QAsimtjojRDqVz/RITpqVqWSQqzpZIehT+pF3en6ceaos4nC3zuM+v0r2EEXTH6J FLkaJS7VeVjYbLCXW20OPvAe/TJoo2LbyPLnZuQyq0gBjC20x/6MWcb2cxSdcRXjAczu 0zslt0jUfJ+Asi6LT32AAz6ab4l6We9Bc8pxzlGDYllHowS6nm8VFACr0xVnGM3GzOxi HteZezqUIwjuJ84X1ZzwtOuVKQuxJ7WclsA7pl75eZXvVURYCoDewZ3v/Q7EH8J70AXa wIkJmFoHs2QjRWOK9nhTgd6FvNFdAe/JwQfYxTwnRmGeEQmbfmBn092mwOKTQJUccel0 veww== X-Gm-Message-State: ALoCoQkgzf6QrstE9qn/WPLgaTF/u4wDe1LOeWChfBIyha2uTW30hf/TUs72nsY5AXCeMxJHNy9tMah458TPiPAb8QaxMuf6dg== X-Received: by 10.28.97.197 with SMTP id v188mr7480441wmb.63.1449612081500; Tue, 08 Dec 2015 14:01:21 -0800 (PST) Received: from [10.10.1.58] (liv3d.labs.multiplay.co.uk. [82.69.141.171]) by smtp.gmail.com with ESMTPSA id i84sm5208099wmc.20.2015.12.08.14.01.19 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 08 Dec 2015 14:01:20 -0800 (PST) From: Steven Hartland X-Google-Original-From: Steven Hartland Subject: Re: svn commit: r290199 - in head/sys/dev: nvd nvme To: Jim Harris References: <201510301635.t9UGZI0F085365@repo.freebsd.org> <5667422C.1050806@freebsd.org> Cc: svn-src-head@freebsd.org Message-ID: <56675336.8080104@freebsd.org> Date: Tue, 8 Dec 2015 22:01:26 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Dec 2015 22:01:24 -0000 On 08/12/2015 21:17, Jim Harris wrote: > > > On Tue, Dec 8, 2015 at 1:48 PM, Steven Hartland > > wrote: > > Hi Jim could you let me know the use case for exposing the > controller stripe size as the disk stripe size done by this commit? > > I ask as it actually causes problems for ZFS which has checks to > ensure zpools perform optimally by correctly configuring ashift to > match the stripesize if reported. > > This is usually fine as stripe size typically reports the physical > block size of device, where sectorsize is the logical block size, > unfortunately this is currently limited to ashift of 13 (8KB) so > when nvme reports 128KB it limits it 8KB and hence every > subsequent zpool status reports a warning about optimal performance. > > Before I look to fix one or the other, I wanted to fully > understand the reasoning behind how nvme behaves here. > > > Some Intel NVMe controllers have a slow path for I/Os that span a > 128KB stripe boundary. The FreeBSD NVMe driver checks for this > condition, and will split the I/O inside of the NVMe driver in these > cases, to ensure we do not hit this slow path. > > The idea behind reporting the stripe size up through GEOM was to > provide a hint to upper layers, especially for file system layout - in > hopes of reducing the number of I/Os that need to be split. > > Based on your findings, limiting the stripe size reported up through > GEOM to 4KB would be OK. This may result in some small number of > additional I/Os to require splitting, but the NVMe I/O path is very > efficient so these additional I/Os would cause very minimal (if any) > difference in performance or CPU utilization. > Thanks for the fast reply Jim most appreciated. I've created a review for the change here: https://reviews.freebsd.org/D4446 If you're happy I'll get that committed. Regards Steve