From owner-freebsd-current@freebsd.org Fri Apr 17 11:23:09 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9160F2B537F for ; Fri, 17 Apr 2020 11:23:09 +0000 (UTC) (envelope-from SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 493YcD4gmsz3GcJ for ; Fri, 17 Apr 2020 11:23:08 +0000 (UTC) (envelope-from SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id BA3C128432 for ; Fri, 17 Apr 2020 13:23:06 +0200 (CEST) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 2BD8C28434 for ; Fri, 17 Apr 2020 13:23:05 +0200 (CEST) From: Miroslav Lachman <000.fbsd@quip.cz> Subject: PCIe NVME drives not detected on Dell R6515 To: FreeBSD-Current Message-ID: Date: Fri, 17 Apr 2020 13:23:04 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 493YcD4gmsz3GcJ X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz has no SPF policy when checking 94.124.105.4) smtp.mailfrom=SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz X-Spamd-Result: default: False [4.00 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(0.82)[ip: (0.27), ipnet: 94.124.104.0/21(0.14), asn: 42000(3.59), country: CZ(0.09)]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[quip.cz]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_SPAM_MEDIUM(0.99)[0.987,0]; NEURAL_SPAM_LONG(1.00)[1.000,0]; RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=00MP=6B=quip.cz=000.fbsd@elsa.codelab.cz]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Apr 2020 11:23:09 -0000 I already asked on stable@ but as I tried it on 13-CURRENT with the same result I am trying to ask for help here. I have rented dedicated server Dell PowerEdge R6515 with iDRAC access only. There are 2 NVME drives which I wanted to use as ZFS root pool. They are shown in iDRAC Device Description: PCIe SSD in Slot 1 in Bay 1 Device Protocol: NVMe-MI1.0 Model: Dell Express Flash NVMe P4510 1TB SFF Bus: 130 Manufacturer: INTEL Product ID: a54 Revision: VDV1DP23 Enclosure: PCIe SSD Backplane 1 pciconf -l show many things, some of them are named "noneN@pci..." but none "nvme" The is printscreen (12.1 but 13-CURRENT is the same) https://ibb.co/tPnymL7 But I booted Linux SystemRescueCd and nvme devices are there visible in /dev/ https://ibb.co/sj22Nwg There is verbose output of Linux lspci https://ibb.co/dPZTwV1 Linux dmesg contains: nvme nvme0: pci function 0000:81:00.0 nvme nvme1: pci function 0000:82:00.0 nvme nvme0: 32/0/0 default/read/poll queues nvme nvme1: 32/0/0 default/read/poll queues The machine is Dell PowerEdge R6515 with AMD EPYC 7302P More details extracted from iDRAC web interface I found this informations PCIe SSD in Slot 1 in Bay 1 Bus: 82 BusProtocol: PCIE Device: 0 DeviceDescription: PCIe SSD in Slot 1 in Bay 1 DeviceProtocol: NVMe-MI1.0 DeviceType: PCIeSSD DriveFormFactor: 2.5 inch FailurePredicted: NO FQDD: Disk.Bay.1:Enclosure.Internal.0-1 FreeSizeInBytes: Information Not Available Function: 0 HotSpareStatus: Information Not Available InstanceID: Disk.Bay.1:Enclosure.Internal.0-1 Manufacturer: INTEL MaximumCapableSpeed: 8 GT/s MediaType: Solid State Drive Model: Dell Express Flash NVMe P4510 1TB SFF NegotiatedSpeed: 8 GT/s PCIeCapableLinkWidth: x4 PCIeNegotiatedLinkWidth: x4 PrimaryStatus: Ok ProductID: a54 RaidStatus: Information Not Available RAIDType: Unknown RemainingRatedWriteEndurance: 100 % Revision: VDV1DP23 SerialNumber: PHLJxxxxxxWF1PxxxxN SizeInBytes: 1000204886016 Slot: 1 State: Ready SystemEraseCapability: CryptographicErasePD PCIe SSD in Slot 1 in Bay 1 - PCI Device BusNumber: 130 DataBusWidth: 4x or x4 Description: Express Flash NVMe 1.0 TB 2.5" U.2 (P4510) DeviceDescription: PCIe SSD in Slot 1 in Bay 1 DeviceNumber: 0 DeviceType: PCIDevice FQDD: Disk.Bay.1:Enclosure.Internal.0-1 FunctionNumber: 0 InstanceID: Disk.Bay.1:Enclosure.Internal.0-1 LastSystemInventoryTime: 2020-04-17T03:21:39 LastUpdateTime: 2020-03-31T13:55:06 Manufacturer: Intel Corporation PCIDeviceID: 0A54 PCISubDeviceID: 2003 PCISubVendorID: 1028 PCIVendorID: 8086 SlotLength: 2.5 Inch Drive Form Factor SlotType: PCI Express Gen 3 SFF-8639 Can anybody shed some light what the real problem is? Is the hardware not properly detected or is the driver completely missing? NVME PCIe architecture is out of my knowledge. I really appreciate any help. Kind regards Miroslav Lachman