From owner-svn-src-all@freebsd.org Sat Aug 4 16:00:30 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F01C106F9F7; Sat, 4 Aug 2018 16:00:30 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03CC97DABF; Sat, 4 Aug 2018 16:00:30 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: by mail-it0-x235.google.com with SMTP id 139-v6so5800465itf.0; Sat, 04 Aug 2018 09:00:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ClC+/gYGwRGTK/aDcdxBv6De/2B6HBhKgPZl8a7pW4s=; b=Eu/Nx4JLCyl4OTA0tIDavNz1Jwyz/h2V8BfdBsh5HiX7CIgjy9j7WwdB6Jtslk7KGf kVycqggWI5FyYOxJ0x8ogVNIVzoSKQrkrohafYMQmFRSz/sbbH3zcF2Qaa03BhxmvOPS 38LKZC2FeOthoDOxAKlfB/s21HX5eUB8fkBpYEjRLdRFXBGvpmSRDfz9evHUUj/h3rO2 6Jn0rkjOaOwftLpyrmk3jXszSA67fn3TbDHLekLOSqzIjVeKO3gGtCSXrsm0aoMi3ni1 lbneUURcDIK1YwDR4+6LhiSbMm8bbVbNxAQBQn5/cTE7pw6vSxvowS3qLfKFpZXihR6R oYrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ClC+/gYGwRGTK/aDcdxBv6De/2B6HBhKgPZl8a7pW4s=; b=I0FHR/U4U2F4zPtxuZ3xh3qcPU5fU6LMkgX5HFVWe02VHRmmprFv8qQNeIp1fjdLvt NvwkeRArIXTQzpXdUzx1YI/9Uy2zj43wtqadsSu0OyZdoJEVqvr3xMQ5Uc18hjDAZJG9 i4xnRXYvUDSVpn5DS3QopqP2EUSFH78FvbJ4FHG7T9d95kmSxkbhX68i0jskNMb3uzm/ LkxbdncsrvC1izzFP5hk/K6skhKAFy4pfTtfctT4ge5Rtyg2QqkVZZauNWfMy1SZewSJ d/MZi6mq2FKucphdFdsbBbxl7wqecTcAwSi0LOEelcNGoOaLaJKqTgJEE0W/D+XTriLl Ny8Q== X-Gm-Message-State: AOUpUlEsEvwGMFpjR923U0KYipNnvOB5+Rf5rJw36CBJ5PdbemV9EkWO hcJo1zVfbCOfL+HNOaG90mYDjUYXw5vaKKsFfkQ= X-Google-Smtp-Source: AAOMgpeXxX2lkaPkW1r5kJDbY26N1Wyh188EVm3t34eOm9cchiZwZUAUfg3vOO5HlnZpsZPGEFRTbMdgzpqEm/+lrYk= X-Received: by 2002:a24:d442:: with SMTP id x63-v6mr10606516itg.152.1533398429478; Sat, 04 Aug 2018 09:00:29 -0700 (PDT) MIME-Version: 1.0 References: <201808032004.w73K46XJ053249@repo.freebsd.org> <20180804080840.GI6049@kib.kiev.ua> <20180804130315.GK6049@kib.kiev.ua> <20180804144733.GO6049@kib.kiev.ua> In-Reply-To: <20180804144733.GO6049@kib.kiev.ua> From: Jim Harris Date: Sat, 4 Aug 2018 09:00:18 -0700 Message-ID: Subject: Re: svn commit: r337273 - head/sys/dev/nvme To: Konstantin Belousov Cc: jhibbits@freebsd.org, John Baldwin , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Aug 2018 16:00:30 -0000 On Sat, Aug 4, 2018 at 7:47 AM Konstantin Belousov wrote: > > Yes, if something special is needed, it should happen in > platform-specific > > > busdma code. > > > > > > Also, if wmb() is needed, then it is not a supposed semantic or > > > wmb(), but a specific side-effects of one of the instruction in the > > > implementation of wmb(). > > > > > > As I noted, on x86 it is not needed and detrimental to the performance. > > > > > > > According to Jim Harris it is needed for x86. I tried to remove it > thinking > > it was unnecessary with the sync in place now. The wmb() was added way > > back in r240616 by him, when only x86 was supported by the driver. > > bus_dmamap_sync() for all archs except powerpc lack a barrier, from what > I > > can see. See the review for more discussion that took place. If it isn't > > needed I will gladly remove it. > > I am very curious why x86 would need MFENCE there. I do not see an > explanation in the review. > On x86, store buffers are not coherent. Parts of the 64-byte submission queue entry could be in store buffers and not globally visible at this point. The wmb() ensures that all of the data is globally visible before the doorbell write. This ensures the NVMe device will DMA the correct data from the submission queue. (Note: the Linux nvme driver also does an explicit wmb() before writing the submission queue doorbell.)