Date: Thu, 10 Mar 2016 21:58:45 -0700 From: Alan Somers <asomers@freebsd.org> To: Warner Losh <imp@bsdimp.com> Cc: Steven Hartland <smh@freebsd.org>, "src-committers@freebsd.org" <src-committers@freebsd.org>, "svn-src-all@freebsd.org" <svn-src-all@freebsd.org>, "svn-src-head@freebsd.org" <svn-src-head@freebsd.org> Subject: Re: svn commit: r292074 - in head/sys/dev: nvd nvme Message-ID: <CAOtMX2h16eb=W9VC-hMtuHLknv8pEzA6OxP9=5uFtrftYsBTvw@mail.gmail.com> In-Reply-To: <CANCZdfp3aq4Ysb%2Bwbew-KjUvg7yqbzoqLSS82hKQQut=QRJQbQ@mail.gmail.com> References: <201512110206.tBB264Ad039486@repo.freebsd.org> <CAOtMX2gAmt_--_vs6M=be9nShkCpKbwzK-K_N4t1MahMijyoog@mail.gmail.com> <CANCZdfp3aq4Ysb%2Bwbew-KjUvg7yqbzoqLSS82hKQQut=QRJQbQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Do they behave badly for writes that cross a 128KB boundary, but are nonetheless aligned to 128KB boundaries? Then I don't understand how this change (or mav's replacement) is supposed to help. The stripesize is supposed to be the minimum write that the device can accept without requiring a read-modify-write. ZFS guarantees that it will never issue a write smaller than the stripesize, nor will it ever issue a write that is not aligned to a stripesize-boundary. But even if ZFS worked with 128KB stripesizes, it would still happily issue writes a multiple of 128KB in size, and these would cross those boundaries. Am I not understanding something here? -Alan On Thu, Mar 10, 2016 at 9:34 PM, Warner Losh <imp@bsdimp.com> wrote: > Some Intel NVMe drives behave badly when the LBA range crosses a 128k > boundary. Their > performance is worse for those transactions than for ones that don't cross > the 128k boundary. > > Warner > > On Thu, Mar 10, 2016 at 11:01 AM, Alan Somers <asomers@freebsd.org> wrote: > >> Are you saying that Intel NVMe controllers perform poorly for all I/Os >> that are less than 128KB, or just for I/Os of any size that cross a 128KB >> boundary? >> >> On Thu, Dec 10, 2015 at 7:06 PM, Steven Hartland <smh@freebsd.org> wrote: >> >>> Author: smh >>> Date: Fri Dec 11 02:06:03 2015 >>> New Revision: 292074 >>> URL: https://svnweb.freebsd.org/changeset/base/292074 >>> >>> Log: >>> Limit stripesize reported from nvd(4) to 4K >>> >>> Intel NVMe controllers have a slow path for I/Os that span a 128KB >>> stripe boundary but ZFS limits ashift, which is derived from d_stripesize, >>> to 13 (8KB) so we limit the stripesize reported to geom(8) to 4KB. >>> >>> This may result in a small number of additional I/Os to require >>> splitting in nvme(4), however the NVMe I/O path is very efficient so these >>> additional I/Os will cause very minimal (if any) difference in performance >>> or CPU utilisation. >>> >>> This can be controller by the new sysctl >>> kern.nvme.max_optimal_sectorsize. >>> >>> MFC after: 1 week >>> Sponsored by: Multiplay >>> Differential Revision: https://reviews.freebsd.org/D4446 >>> >>> Modified: >>> head/sys/dev/nvd/nvd.c >>> head/sys/dev/nvme/nvme.h >>> head/sys/dev/nvme/nvme_ns.c >>> head/sys/dev/nvme/nvme_sysctl.c >>> >>> >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2h16eb=W9VC-hMtuHLknv8pEzA6OxP9=5uFtrftYsBTvw>