From nobody Fri Jan 21 02:26:56 2022 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E8FC81968E81; Fri, 21 Jan 2022 02:27:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jg3DK4SRsz4nyd; Fri, 21 Jan 2022 02:26:57 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642732019; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0EiJni5qiV01WkJIX45Wb7ulIUwGPzeKp7BWAYHoeh4=; b=UC/B2Kut/slL9IPR2dio4Z9bug2DiAzVjVnW8l1dBjeWemmpQ6xt2az32UiEjY67HWRPkG ddCy/xjXPlqPgsU+eiPx0EhYR1GPpZI/28cv/7vA5ZTSXoO/FAsn7zimWuOQG5s4npp5yT R3lWRLQA0qJ+7aQu39CnPBrHZe6+STgC35WexaGMLiEnGNAZPYEF2n97d1OZnvH4lAwp6b MrUAd2RIVzAyZ/D7Cr4f8RW0Ht2uOyUGug8MdgSApTcxjfjxslbBQyTe58wFOF5Z0DK7LT Hyv3amTxsvQ6AHdL0755Bgl3aMpStyYoU2TMdKQqV6oUPIOqCxlv/ovZNfIT4A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2AA9213A68; Fri, 21 Jan 2022 02:26:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20L2Qutx001306; Fri, 21 Jan 2022 02:26:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20L2Quc2001305; Fri, 21 Jan 2022 02:26:56 GMT (envelope-from git) Date: Fri, 21 Jan 2022 02:26:56 GMT Message-Id: <202201210226.20L2Quc2001305@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: a8ce65544595 - stable/13 - nvme_ctrlr_enable: Remove unnecessary 5ms delays List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: a8ce65544595144b81130b7524ac684084d5220e Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1642732019; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0EiJni5qiV01WkJIX45Wb7ulIUwGPzeKp7BWAYHoeh4=; b=JPxgMrbUkW28Zihu5sk6tj2VLuLf+OdlMS3tVG6TTZ5zhbHBMgLkxpDxJvZ3GwIBQULIIC F16EEJ/3GClLsf7+ItHRZ5QBja4VOW7/yrLu0lAo1IhkAREIDI4VXjHLDPPfpdwv7uwIe6 X0GhEAeS3/1xYWQFRRop+30rR4yHJ2FdrMnG7uqlnRxbj3dspLHsy1cgorpb5BYl3FqbwO 92toK+iPiJ2Pf306qpV/gE1cYQL/UTbJHj1Zin3O5pY0EC9WSbweu0lkfQ4oP4YGXpWjoH NKEk90mLfEv514IgCy77ojzLwDWlajqg4bR8w1Hoz8h5KfWiolpz4wfpcmpk9Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1642732019; a=rsa-sha256; cv=none; b=YDABvksIbR/eU9F73r8p3SJbbqV3VjGlIuKjxTYll3LFhTw5UIbwcivKw9571GhcwpN1hm CZLtKUufSgIgqPx4ME48lhJPDlH0lhaL4ZwNsoSCmovrjyxlmI8HyXQavPwc/pSp1A77EL S7RthsiqGlsbgwjt72neZqZVUrAOnb7U2SJYk77eg2BxPbGxOYWlOGbho/PXpreb4KJrOA Idbsbm11aTBef8kNgXbMN3KBGh0jME91HjAOgnq5Jbr/5YJ68K+5p4JLCgA0Ao6EhFgBDl /5VlLKqyO4Vvl25JLPdVN+sw8JAghl8fOsDG2w/tzXmGsgDdDBIeTC1664laMQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=a8ce65544595144b81130b7524ac684084d5220e commit a8ce65544595144b81130b7524ac684084d5220e Author: Warner Losh AuthorDate: 2021-10-01 16:47:27 +0000 Commit: Alexander Motin CommitDate: 2022-01-21 02:07:30 +0000 nvme_ctrlr_enable: Remove unnecessary 5ms delays Remove the 5ms delays after writing the administrative queue registers. These delays are from the very earliest days of the driver (they are in the first commit) and were most likely vestiges of the Chatham NVMe prototype card that was used to create this driver. Many of the workarounds necessary for it aren't necessary for standards compliant cards. The original driver had other areas marked for Chatham, but these were not. They are unneeded. There's three lines of supporting evidence. First, the NVMe standards make no mention of a delay time after these registers are written. Second, the Linux driver doesn't have them, even as an option. Third, all my nvme cards work w/o them. To be safe, add a write barrier between setting up the admin queue and enabling the controller. Sponsored by: Netflix Reviewed by: mav Differential Revision: https://reviews.freebsd.org/D32247 (cherry picked from commit d5fca1dc1d7de15695b65374d6457abd29a747ee) --- sys/dev/nvme/nvme_ctrlr.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/sys/dev/nvme/nvme_ctrlr.c b/sys/dev/nvme/nvme_ctrlr.c index 39cde3f065dd..897d832047ce 100644 --- a/sys/dev/nvme/nvme_ctrlr.c +++ b/sys/dev/nvme/nvme_ctrlr.c @@ -52,6 +52,12 @@ __FBSDID("$FreeBSD$"); static void nvme_ctrlr_construct_and_submit_aer(struct nvme_controller *ctrlr, struct nvme_async_event_request *aer); +static void +nvme_ctrlr_barrier(struct nvme_controller *ctrlr, int flags) +{ + bus_barrier(ctrlr->resource, 0, rman_get_size(ctrlr->resource), flags); +} + static void nvme_ctrlr_devctl_log(struct nvme_controller *ctrlr, const char *type, const char *msg, ...) { @@ -356,9 +362,7 @@ nvme_ctrlr_enable(struct nvme_controller *ctrlr) } nvme_mmio_write_8(ctrlr, asq, ctrlr->adminq.cmd_bus_addr); - DELAY(5000); nvme_mmio_write_8(ctrlr, acq, ctrlr->adminq.cpl_bus_addr); - DELAY(5000); /* acqs and asqs are 0-based. */ qsize = ctrlr->adminq.num_entries - 1; @@ -367,7 +371,6 @@ nvme_ctrlr_enable(struct nvme_controller *ctrlr) aqa = (qsize & NVME_AQA_REG_ACQS_MASK) << NVME_AQA_REG_ACQS_SHIFT; aqa |= (qsize & NVME_AQA_REG_ASQS_MASK) << NVME_AQA_REG_ASQS_SHIFT; nvme_mmio_write_4(ctrlr, aqa, aqa); - DELAY(5000); /* Initialization values for CC */ cc = 0; @@ -381,6 +384,7 @@ nvme_ctrlr_enable(struct nvme_controller *ctrlr) /* This evaluates to 0, which is according to spec. */ cc |= (PAGE_SIZE >> 13) << NVME_CC_REG_MPS_SHIFT; + nvme_ctrlr_barrier(ctrlr, BUS_SPACE_BARRIER_WRITE); nvme_mmio_write_4(ctrlr, cc, cc); return (nvme_ctrlr_wait_for_ready(ctrlr, 1));