From owner-freebsd-current@freebsd.org Fri Apr 17 19:18:24 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B6B5F2C5B97 for ; Fri, 17 Apr 2020 19:18:24 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 493m8b5Jz7z4W0T for ; Fri, 17 Apr 2020 19:18:23 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id E2A5A456; Fri, 17 Apr 2020 15:18:21 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 17 Apr 2020 15:18:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsco.org; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm3; bh=4 Rk9dRwCcn4dnx52iPzntreiSeKmEcZ2Pq9HsPQbVKo=; b=Qk2XyCG8ewFg1uTKR Y1usF9VYn9kaR+czGtMjVqGKA+ScxtHoIkjTHr/lHU5elsI0h5bRq4CCdQQiY+6N 9FERjTFAFj3KG6RRdXIAiiQF4rUOL90o73kLKjHd6Gg5BrZbl236kcScjz1qbF0w JgyStL1i2wBkK3LXx2+l7hWrZi3h325rOc25cTcEsStV8IPjSAeeX2L1aRKK64Md Gxt6aikjCWL4w7y1s2hoyMS5xVni2xqu59dpNQYdGEEErHgwsNvE8xvBqdnwlNuv W1wXg33SvXW3GIlw5key6qZTQgf+bF3sogJQT//n0XjPqPV1dhOdyjXoujE94D1H gElhw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=4Rk9dRwCcn4dnx52iPzntreiSeKmEcZ2Pq9HsPQbV Ko=; b=B8loiBkJnKtTG2uvOS1KfnY2cJMB4aPvz4Ugj8ER1q4V+YG3BTYbkxBwI V8biirYOZKhmvXKdN0U7K0XAswHRuC02vk4bZQdkKHP9NBBkzOnUhmpBuNmNCblc gHdn5MmPlMIDzhLjdIdbdtdsG3aqssJaY4QJaVfjI8pjt75qEJ41UT3ifm7jUK2L TRMxlR0dadf/0G7NUJYlSjM4oigvqYaDjJpdAOFMlcZQw+vrY6nBCuLbHk35FwDR anqilXU2FUOHhuTU0Rw/Lhtgv529MpfjgC2VpTd89o998Q4/p7oWwq08n/jVoXdi x5czSf9wyX9wDSTSXnw0j4Zzy4p4g== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrfeejgddufeehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpegtggfuhfgjfffgkfhfvffosehtqh hmtdhhtdejnecuhfhrohhmpefutghothhtucfnohhnghcuoehstghothhtlhesshgrmhhs tghordhorhhgqeenucfkphepkedrgeeirdekledrvddufeenucevlhhushhtvghrufhiii gvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehstghothhtlhesshgrmhhstghordho rhhg X-ME-Proxy: Received: from [192.168.0.114] (unknown [8.46.89.213]) by mail.messagingengine.com (Postfix) with ESMTPA id B2170306005C; Fri, 17 Apr 2020 15:18:20 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\)) Subject: Re: PCIe NVME drives not detected on Dell R6515 From: Scott Long In-Reply-To: <573f5fab-1ef6-151f-18ca-58d3a4a89c72@quip.cz> Date: Fri, 17 Apr 2020 13:18:20 -0600 Cc: Warner Losh , FreeBSD-Current Content-Transfer-Encoding: quoted-printable Message-Id: <07B6763F-C23B-4B7C-B76A-26267AC35453@samsco.org> References: <0F8BCB8C-DE60-4A34-A4D8-F1BB4B9F906A@samsco.org> <9EF043C1-FF8F-4997-B59A-EC3BF7D1CEEE@samsco.org> <31E8B2BE-BED2-4084-868D-32C48CB3CD6E@samsco.org> <573f5fab-1ef6-151f-18ca-58d3a4a89c72@quip.cz> To: Miroslav Lachman <000.fbsd@quip.cz> X-Mailer: Apple Mail (2.3601.0.10) X-Rspamd-Queue-Id: 493m8b5Jz7z4W0T X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=samsco.org header.s=fm3 header.b=Qk2XyCG8; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=B8loiBkJ; dmarc=none; spf=pass (mx1.freebsd.org: domain of scottl@samsco.org designates 64.147.123.21 as permitted sender) smtp.mailfrom=scottl@samsco.org X-Spamd-Result: default: False [-5.60 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[samsco.org:s=fm3,messagingengine.com:s=fm2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.21]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[samsco.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[samsco.org:+,messagingengine.com:+]; IP_SCORE(-3.50)[ip: (-9.87), ipnet: 64.147.123.0/24(-4.92), asn: 11403(-2.69), country: US(-0.05)]; RCVD_IN_DNSWL_LOW(-0.10)[21.123.147.64.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Apr 2020 19:18:24 -0000 > On Apr 17, 2020, at 11:46 AM, Miroslav Lachman <000.fbsd@quip.cz> = wrote: >=20 > Scott Long wrote on 04/17/2020 18:17: >> You are correct about Intel vs AMD. Comparing the full output of = pciconf from FreeBSD with the fragment of lspci from Linux suggests that = there=E2=80=99s at least one set of a PCIe switch and child devices that = is not being enumerated by FreeBSD. Can you send the full output of = `lspci -tvv` from linux? >=20 > Sorry for my late reply. Booting the Linux SystemRescueCd is too slow. >=20 > lspci -tvv output is attached >=20 > I tried to connect to SOL by SSH but it shows black screen only. >=20 > lspci shows drives: >=20 > Intel Corporation NVMe Datacenter SSD [3DNAND, Beta Rock Controller] >=20 > Kind regards > Miroslav Lachman >=20 It looks like pcib12 and pcib13 in FreeBSD should be the bridges that = have the NVMe devices behind them, but those devices aren=E2=80=99t = showing up. I don=E2=80=99t see any obvious code in linux to handle = those bridges, so there must be something subtle that we=E2=80=99re = missing in FreeBSD. Maybe we=E2=80=99re having trouble enumerating = above PCI bus 128? I think that was a problem a few years ago but I = also thought it was fixed. Can you do the following in FreeBSD: pciconf -lBc pcib12 pciconf -lBc pcib13 Thanks, Scott