From owner-freebsd-current@freebsd.org Fri Apr 17 19:42:00 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 232632C6B37 for ; Fri, 17 Apr 2020 19:42:00 +0000 (UTC) (envelope-from SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 493mgq0Jvnz4YS1 for ; Fri, 17 Apr 2020 19:41:58 +0000 (UTC) (envelope-from SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 7660828433; Fri, 17 Apr 2020 21:41:56 +0200 (CEST) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 7510B28434; Fri, 17 Apr 2020 21:41:55 +0200 (CEST) Subject: Re: PCIe NVME drives not detected on Dell R6515 To: Scott Long Cc: Warner Losh , FreeBSD-Current References: <0F8BCB8C-DE60-4A34-A4D8-F1BB4B9F906A@samsco.org> <9EF043C1-FF8F-4997-B59A-EC3BF7D1CEEE@samsco.org> <31E8B2BE-BED2-4084-868D-32C48CB3CD6E@samsco.org> <573f5fab-1ef6-151f-18ca-58d3a4a89c72@quip.cz> <07B6763F-C23B-4B7C-B76A-26267AC35453@samsco.org> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: Date: Fri, 17 Apr 2020 21:41:54 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: <07B6763F-C23B-4B7C-B76A-26267AC35453@samsco.org> X-Rspamd-Queue-Id: 493mgq0Jvnz4YS1 X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz has no SPF policy when checking 94.124.105.4) smtp.mailfrom=SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz X-Spamd-Result: default: False [4.01 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; DMARC_NA(0.00)[quip.cz]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(1.00)[0.996,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; IP_SCORE(0.81)[ip: (0.27), ipnet: 94.124.104.0/21(0.13), asn: 42000(3.58), country: CZ(0.09)]; NEURAL_SPAM_LONG(1.00)[1.000,0]; RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Apr 2020 19:42:00 -0000 Scott Long wrote on 04/17/2020 21:18: > > >> On Apr 17, 2020, at 11:46 AM, Miroslav Lachman <000.fbsd@quip.cz> wrote: >> >> Scott Long wrote on 04/17/2020 18:17: >>> You are correct about Intel vs AMD. Comparing the full output of pciconf from FreeBSD with the fragment of lspci from Linux suggests that there’s at least one set of a PCIe switch and child devices that is not being enumerated by FreeBSD. Can you send the full output of `lspci -tvv` from linux? >> >> Sorry for my late reply. Booting the Linux SystemRescueCd is too slow. >> >> lspci -tvv output is attached >> >> I tried to connect to SOL by SSH but it shows black screen only. >> >> lspci shows drives: >> >> Intel Corporation NVMe Datacenter SSD [3DNAND, Beta Rock Controller] >> >> Kind regards >> Miroslav Lachman >> > > It looks like pcib12 and pcib13 in FreeBSD should be the bridges that have the NVMe devices behind them, but those devices aren’t showing up. I don’t see any obvious code in linux to handle those bridges, so there must be something subtle that we’re missing in FreeBSD. Maybe we’re having trouble enumerating above PCI bus 128? I think that was a problem a few years ago but I also thought it was fixed. Can you do the following in FreeBSD: > > pciconf -lBc pcib12 > pciconf -lBc pcib13 Printscreen attached. Anything else I can provide? Thank you Miroslav Lachman