From owner-freebsd-current@freebsd.org Fri May 1 01:24:56 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 595892CD40D for ; Fri, 1 May 2020 01:24:56 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 49CvgX0CDrz4Hsm for ; Fri, 1 May 2020 01:24:56 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 04E822CD40C; Fri, 1 May 2020 01:24:56 +0000 (UTC) Delivered-To: current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 038F32CD40B for ; Fri, 1 May 2020 01:24:56 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49CvgV2Zldz4Hsl for ; Fri, 1 May 2020 01:24:53 +0000 (UTC) (envelope-from ypankov@fastmail.com) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id A576E68A for ; Thu, 30 Apr 2020 21:24:52 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Thu, 30 Apr 2020 21:24:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= subject:from:to:references:message-id:date:mime-version :in-reply-to:content-type; s=fm2; bh=RASgt9f5v71FezaQQt5qPORdNwE HueOM9LH9TGsWK5o=; b=WVnmPlAcebDKqAYQ/wBr9f9zrbxcl4oZUngBrsOaULh 5QYZCXQLnX5rXfrxayWzT61bZ/s2PvDiDdo/lThML9wa8TGC/ZL1jPdRvmqrLrw3 AeY66DL1QnZcMXEuDIhlJhQLkfHzJWvu6mGiA6g7/4oFjnqG80Np2mSDC9a1ieg3 OdrMO5WN7LTAznIIzyOkBUw1t5ngNiyhghssIgekmgEu6IPqAsA1s+oZoDaS24MB MfeHY8kwEouYdLg22seXxchmVCD66XPEl1ZCcLdFIw3Qp1mZL0wmMRxVyiAoaAqy QzleGhHjUAtITYTGGYEgCbEA6okfVitRHbSRzBXxoiA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=RASgt9 f5v71FezaQQt5qPORdNwEHueOM9LH9TGsWK5o=; b=oj2/Xp2rQ6hkZmAz1OGRM1 cIKlCYN/tBHYxDJbZsD5BhtJ7FZjQ+rSgda0HWK2JL/bhKw0n24auXQsetniAIwl SNHtdor7pyqazaHO7CmISAxmAXKa4keB7YXQozPhp58HvxlGeMvFFQpOpCOK89qo Wf04LiFwe/rB4eiIXPGJa9HjmmnhDa8salwKBBWxn9UASlVmTgXlt46XDqlXUIBZ Nph5X5LFZyiiqMr5JLywDItop10RLwwXQWcdn9csx83o07lu1EAWqZHRBZwfFcmt j3ulVZNFgZInpAa8wtDoBrMIIk2Vh1xL6qDUy2XbFJF72ibPdPsr6HXsIVCN5nBQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrieeigdeghecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepuffhvfhfkffffgggjggtsehmtderre dtfeejnecuhfhrohhmpegjuhhrihcurfgrnhhkohhvuceohihprghnkhhovhesfhgrshht mhgrihhlrdgtohhmqeenucggtffrrghtthgvrhhnpeegkedvgefhffefgfeigffgtdevtd eghfehhefhlefhkefffedttddtffeuieeihfenucffohhmrghinhepghhithhhuhgsrdgt ohhmnecukfhppeekhedrudejgedrvddtjedrvddujeenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeihphgrnhhkohhvsehfrghsthhmrghilhdr tghomh X-ME-Proxy: Received: from [192.168.1.6] (unknown [85.174.207.217]) by mail.messagingengine.com (Postfix) with ESMTPA id 02C493065F65 for ; Thu, 30 Apr 2020 21:24:50 -0400 (EDT) Subject: Re: nvme on 2019 macbooks From: Yuri Pankov To: current@freebsd.org References: Message-ID: <79fec490-d3b1-b91b-652c-e9bf0d121d7c@fastmail.com> Date: Fri, 1 May 2020 04:24:49 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------A6596766C38AEAEA6873BEF1" Content-Language: en-US X-Rspamd-Queue-Id: 49CvgV2Zldz4Hsl X-Spamd-Bar: +++++++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fastmail.com header.s=fm2 header.b=WVnmPlAc; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=oj2/Xp2r; dmarc=pass (policy=none) header.from=fastmail.com; spf=pass (mx1.freebsd.org: domain of ypankov@fastmail.com designates 64.147.123.20 as permitted sender) smtp.mailfrom=ypankov@fastmail.com X-Spamd-Result: default: False [7.22 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:64.147.123.20]; FREEMAIL_FROM(0.00)[fastmail.com]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[fastmail.com:+,messagingengine.com:+]; CTYPE_MIXED_BOGUS(1.00)[]; MIME_BASE64_TEXT(0.10)[]; DMARC_POLICY_ALLOW(0.00)[fastmail.com,none]; RECEIVED_SPAMHAUS_PBL(0.00)[217.207.174.85.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; MIME_TRACE(0.00)[0:+,1:+,2:+]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[20.123.147.64.list.dnswl.org : 127.0.5.1]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_XBL(5.00)[217.207.174.85.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.4]; R_DKIM_ALLOW(0.00)[fastmail.com:s=fm2,messagingengine.com:s=fm2]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[fastmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; NEURAL_SPAM_MEDIUM(0.22)[0.222,0]; RCPT_COUNT_ONE(0.00)[1]; BAD_REP_POLICIES(0.10)[]; IP_SCORE_FREEMAIL(0.00)[]; IP_SCORE(0.00)[ip: (-9.85), ipnet: 64.147.123.0/24(-4.92), asn: 11403(-2.69), country: US(-0.05)]; NEURAL_SPAM_LONG(1.00)[0.996,0] X-Spam: Yes X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2020 01:24:56 -0000 This is a multi-part message in MIME format. --------------A6596766C38AEAEA6873BEF1 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Yuri Pankov wrote: > I have tried booting latest -current snapshot on 2019 macbook pro 16, > and noticed that nvme driver didn't attach, as class reported is > 0x018002 and not 0x010802 that we expect in nvme_pci.c. > > The following linux changes seem to be related: > https://github.com/torvalds/linux/commit/66341331ba0d2de4ff421cdc401a1e34de50502a > > https://github.com/torvalds/linux/commit/d38e9f04ebf667d9cb8185b45bff747485f1d3e9 > > > I have tried adding the exact PCI IDs, but that also fails as number of > msix vectors seems to be 0 in nvme_ctrlr_setup_interrupts() and > nvme_ctrlr_configure_intx() fails with "unable to allocate shared IRQ". > > Any hints on how to proceed here? So it looks like we need to fallback to MSI if we failed to enable MSI-X. With the attached patch we still fail to attach the target device (as below) most likely due to the quirks needed as seen in linux driver, but it's definitely a start: nvme0: CREATE IO CQ (05) sqid:0 cid:15 nsid:0 cdw10:00800001 cdw11:00010003 nvme0: INVALID_FIELD (00/02) sqid:0 cid:15 cdw0:0 nvme0: nvme_create_io_cq failed! --------------A6596766C38AEAEA6873BEF1 Content-Type: text/plain; charset=UTF-8; name="nvme.diff.txt" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="nvme.diff.txt" ZGlmZiAtLWdpdCBhL3N5cy9kZXYvbnZtZS9udm1lX3BjaS5jIGIvc3lzL2Rldi9udm1lL252 bWVfcGNpLmMKaW5kZXggNDQ4YmZkYTZhNzE4Li5lNjA5OTY3YjUzZmUgMTAwNjQ0Ci0tLSBh L3N5cy9kZXYvbnZtZS9udm1lX3BjaS5jCisrKyBiL3N5cy9kZXYvbnZtZS9udm1lX3BjaS5j CkBAIC05MCw2ICs5MCw3IEBAIHN0YXRpYyBzdHJ1Y3QgX3Bjc2lkCiAJeyAweDA1NDAxYzVm LAkJMCwgMCwgIk1lbWJsYXplIFBibGF6ZTQiLCBRVUlSS19ERUxBWV9CNF9DSEtfUkRZIH0s CiAJeyAweGE4MjExNDRkLAkJMCwgMCwgIlNhbXN1bmcgUE0xNzI1IiwgUVVJUktfREVMQVlf QjRfQ0hLX1JEWSB9LAogCXsgMHhhODIyMTQ0ZCwJCTAsIDAsICJTYW1zdW5nIFBNMTcyNWEi LCBRVUlSS19ERUxBWV9CNF9DSEtfUkRZIH0sCisJeyAweDIwMDUxMDZiLAkJMCwgMCwgIkFO UzIgTlZNZSBDb250cm9sbGVyIiB9LAogCXsgMHgwMDAwMDAwMCwJCTAsIDAsIE5VTEwgIH0K IH07CiAKQEAgLTI2Nyw3ICsyNjgsNyBAQCBudm1lX2N0cmxyX3NldHVwX2ludGVycnVwdHMo c3RydWN0IG52bWVfY29udHJvbGxlciAqY3RybHIpCiAKIAlmb3JjZV9pbnR4ID0gMDsKIAlU VU5BQkxFX0lOVF9GRVRDSCgiaHcubnZtZS5mb3JjZV9pbnR4IiwgJmZvcmNlX2ludHgpOwot CWlmIChmb3JjZV9pbnR4IHx8IHBjaV9tc2l4X2NvdW50KGRldikgPCAyKSB7CisJaWYgKGZv cmNlX2ludHgpIHsKIAkJbnZtZV9jdHJscl9jb25maWd1cmVfaW50eChjdHJscik7CiAJCXJl dHVybjsKIAl9CkBAIC0yOTcsOSArMjk4LDE0IEBAIG52bWVfY3RybHJfc2V0dXBfaW50ZXJy dXB0cyhzdHJ1Y3QgbnZtZV9jb250cm9sbGVyICpjdHJscikKIAkvKiBPbmUgdmVjdG9yIGZv ciBwZXIgY29yZSBJL08gcXVldWUsIHBsdXMgb25lIHZlY3RvciBmb3IgYWRtaW4gcXVldWUu ICovCiAJbnVtX3ZlY3RvcnNfcmVxdWVzdGVkID0gbnVtX2lvX3F1ZXVlcyArIDE7CiAJbnVt X3ZlY3RvcnNfYWxsb2NhdGVkID0gbnVtX3ZlY3RvcnNfcmVxdWVzdGVkOworCisJLyogVHJ5 IE1TSS1YICovCiAJaWYgKHBjaV9hbGxvY19tc2l4KGRldiwgJm51bV92ZWN0b3JzX2FsbG9j YXRlZCkgIT0gMCkgewotCQludm1lX2N0cmxyX2NvbmZpZ3VyZV9pbnR4KGN0cmxyKTsKLQkJ cmV0dXJuOworCQkvKiBNU0ktWCBmYWlsZWQsIHRyeSBNU0kgKi8KKwkJaWYgKHBjaV9hbGxv Y19tc2koZGV2LCAmbnVtX3ZlY3RvcnNfYWxsb2NhdGVkKSAhPSAwKSB7CisJCQludm1lX2N0 cmxyX2NvbmZpZ3VyZV9pbnR4KGN0cmxyKTsKKwkJCXJldHVybjsKKwkJfQogCX0KIAlpZiAo bnVtX3ZlY3RvcnNfYWxsb2NhdGVkIDwgMikgewogCQlwY2lfcmVsZWFzZV9tc2koZGV2KTsK --------------A6596766C38AEAEA6873BEF1--