Date: Mon, 15 Jan 2018 22:36:36 +0000 From: Grzegorz Junka <list1@gjunka.com> Cc: freebsd-questions@freebsd.org, freebsd-drivers@freebsd.org Subject: Re: Server doesn't boot when 3 PCIe slots are populated Message-ID: <d005c90f-2276-1934-e773-bc9493192eef@gjunka.com> In-Reply-To: <57933.108.68.169.115.1516035394.squirrel@cosmo.uchicago.edu> References: <ecce3fa6-3909-0947-685c-8a412684e99c@gjunka.com> <CAOgwaMsf9zByJYhL3KqpUMW5qKAzQEHpDWcwejY-uK=9swWbUQ@mail.gmail.com> <3d0ad00c-5214-71b0-017b-c2d5ba608e37@gjunka.com> <CAOgwaMsOKrGfGNmRt-C9Skjssj8JPAtFpk8bwG9v55LmaWdoVw@mail.gmail.com> <8df1e967-01e0-d3c2-e14c-64c7fc8c66b0@gjunka.com> <CANCZdfqZ-dogHXBdoyMPLOPs_R-vD%2BwLM-r6sm6ypesd0Nvp4A@mail.gmail.com> <57933.108.68.169.115.1516035394.squirrel@cosmo.uchicago.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
[cut]
On 15/01/2018 16:56, Valeri Galtsev wrote:
> On Mon, January 15, 2018 12:18 am, Warner Losh wrote:
>>
>> Thanks Mehmet for looking into this. It's an old motherboard but my point
>> is that it boots fine when either: one NVMe and the network card, or both
>> NVMe are installed, but not when all three are installed. How would that
>> be
>> related to FreeBSD compatibility? The chipset and all devices that I am
>> trying to install are supported by FreeBSD 11.x.
>>
>> I just tried booting into a Debian live system and it also didn't
>> enumerate
>> NVMe drives properly. This means that it's not FreeBSD related and is no
>> longer relevant for this list.
> This sounds to me as having something to do with allocation of PCI address
> space to talk to devices. Many devices can alternatively use different
> ranges of addresses, so more than one such device can be attached to the
> same PCI bus. These two particular devices seem to not be successfully
> negotiated to use different (not overlapping) ranges of addresses (in
> presence of some particular third device). Maybe it only happens like that
> in this particular system board ("motherboard"). I would try the same on
> different machine. But it is likely that all these three devices do not
> have non-overlapping address ranges.
>
> I hope, someone more knowledgeable that I will chime in.
That was exactly my suspicion, that the network card somehow takes away
some resources that otherwise would be available to the NVMe cards. Not
sure if that's possible but would love to hear an informed opinion on this.
Note, that as stated somewhere else, a live Debian showed the same issue
- after booting from live USB only one NVMe and the network card were
visible. The other NVMe was ignored. This would suggest either hardware
or BIOS problems, so if I had some more information I could try to
change some of the BIOS settings to see if that would help.
GregJ
> Valeri
>
>> I will try to play with BIOS settings to
>> see
>> if I can make it work that way. Thanks for all the help.
>>
>>
>>
>> Nvme drives are weird about power. I distrust the power estimate of 5-9w
>> earlier in the thread... given the oddity with debian, it's not too crazy
>> to think that. How far does FreeBSD boot though?
>>
>> Warner
>>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d005c90f-2276-1934-e773-bc9493192eef>