Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 14 Jan 2018 14:46:18 +0000
From:      Grzegorz Junka <list1@gjunka.com>
Cc:        FreeBSD Questions Mailing List <freebsd-questions@freebsd.org>, freebsd-drivers@freebsd.org
Subject:   Re: Server doesn't boot when 3 PCIe slots are populated
Message-ID:  <3d0ad00c-5214-71b0-017b-c2d5ba608e37@gjunka.com>
In-Reply-To: <CAOgwaMsf9zByJYhL3KqpUMW5qKAzQEHpDWcwejY-uK=9swWbUQ@mail.gmail.com>
References:  <ecce3fa6-3909-0947-685c-8a412684e99c@gjunka.com> <CAOgwaMsf9zByJYhL3KqpUMW5qKAzQEHpDWcwejY-uK=9swWbUQ@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help

On 13/01/2018 17:56, Mehmet Erol Sanliturk wrote:
>
>
> On Sat, Jan 13, 2018 at 7:21 PM, Grzegorz Junka <list1@gjunka.com 
> <mailto:list1@gjunka.com>> wrote:
>
>     Hello,
>
>     I am installing a FreeBSD server based on Supermicro H8SML-iF.
>     There are three PCIe slots to which I installed 2 NVMe drives and
>     one network card Intel I350-T4 (with 4 Ethernet slots).
>
>     I am observing a strange behavior where the system doesn't boot if
>     all three PCIe slots are populated. It shows this message:
>
>     nvme0: <Generic NVMe Device> mem 0xfd8fc000-0xfd8fffff irq 24 at
>     device 0.0 on pci1
>     nvme0: controller ready did not become 1 within 30000 ms
>     nvme0: did not complete shutdown within 5 seconds of notification
>
>     The I see a kernel panic/dump and the system reboots after 15 seconds.
>
>     If I remove one card, either one of the NVMe drives or the network
>     card, the system boots fine. Also, if in BIOS I set PnP OS to YES
>     then sometimes it boots (but not always). If I set PnP OS to NO,
>     and all three cards are installed, the system never boots.
>
>     When the system boots OK I can see that the network card is
>     reported as 4 separate devices on one of the PCIe slots. I tried
>     different NVMe drives as well as changing which device is
>     installed to which slot but the result seems to be the same in any
>     case.
>
>     What may be the issue? Amount of power drawn by the hardware? Too
>     many devices not supported by the motherboard? Too many interrupts
>     for the FreeBSD kernel to handle?
>
>     Any help would be greatly appreciated.
>
>     GregJ
>
>     _______________________________________________
>
>
>
>
>
> From my experience from other trade marked main boards , an action may 
> be to check manual of your server board to see whether there are rules 
> about use of these slots : Sometimes differently shaped slots are 
> supplied with same ports : If one slot is occupied , the other slot 
> should be left open , or rules about not to insert such a kind of 
> device into a slot , for example , graphic cards .
>
>
> Mehmet Erol Sanliturk
>

I checked the manual but couldn't find any restrictions regarding PCIe 
ports. It only says how many lanes are available in each slot. Would 
there be any obvious BIOS setting that could cause this issue? I tried 
after resetting BIOS to default settings but maybe something is set 
incorrectly by default?

GregJ



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3d0ad00c-5214-71b0-017b-c2d5ba608e37>