From owner-freebsd-questions@freebsd.org Tue Jan 16 08:28:34 2018 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3DA61EB4807 for ; Tue, 16 Jan 2018 08:28:34 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [88.98.225.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D3DC36DB43 for ; Tue, 16 Jan 2018 08:28:33 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from ultrabook.yoonka.com (x527162a3.dyn.telefonica.de [82.113.98.163]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id w0G8SPQt066084 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 16 Jan 2018 08:28:25 GMT (envelope-from list1@gjunka.com) X-Authentication-Warning: msa1.earth.yoonka.com: Host x527162a3.dyn.telefonica.de [82.113.98.163] claimed to be ultrabook.yoonka.com Subject: Re: Server doesn't boot when 3 PCIe slots are populated Cc: FreeBSD Questions Mailing List References: <3d0ad00c-5214-71b0-017b-c2d5ba608e37@gjunka.com> <8df1e967-01e0-d3c2-e14c-64c7fc8c66b0@gjunka.com> <0e582bdb-e1f9-438c-3da2-2bcdc950aab5@gjunka.com> <57715.108.68.169.115.1516033864.squirrel@cosmo.uchicago.edu> <8fee9df3-c40b-addb-b3c9-bedd90683d62@gjunka.com> <20180116062709.a6c75fa4.freebsd@edvax.de> From: Grzegorz Junka Message-ID: Date: Tue, 16 Jan 2018 08:28:20 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20180116062709.a6c75fa4.freebsd@edvax.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB-large X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2018 08:28:34 -0000 On 16/01/2018 05:27, Polytropon wrote: > On Mon, 15 Jan 2018 22:30:18 +0000, Grzegorz Junka wrote: >> I tried a different pair of NVMe cards (different adapters with >> different SSD disks) and the result was exactly the same. Note, that the >> pair that I tried was previously working in this motherboard without >> problems for many months, so it's safe to assume that the addition of >> the network card is causing this problem. But then again, the network >> card with one of the NVMe drives works fine too. >> >> Could be that all three cause some sort of impedance mismatch but that's >> kind of hard to believe - these are simple cards, there is almost no >> circuits on the NVMe adapter and the network card is just a chipset with >> 4 slots. > No idea if it still applies, but: > > I don't know if this idea has come uo yet, but many _many_ years > ago, I was in a comparable situation. This was of course "traditional > PCI times" where a common system board had 4 - 6 PCI slots. In that > particular system, 2 NICs, an ATA "controller", a TV card, and a > sound card (5 PCI cards total + 1 AGP graphics card) were installed. > The problem was that the system wouldn't start booting after BIOS POST. > The solution was to re-arrange cards, followed by a PCI configuration > reset in the CMOS setup, followed by another boot attempt. In one > specific card configuration, the system worked as expected, and > FreeBSD (at that time, probably v4 or v5) would detect all the cards > without any problems and attach the appropriate drivers. Removing > one of the cards, or exchanging card positions (for "more convenient > cabling") would render the system non-booting again. Back to the one > configuration that worked - system booted. Let me emphasize that it > was required to reset the PCI configuration in the CMOS setup every > time such a card change was made, as returning to the "verified > locations" without doing so would _not_ let the system boot properly. > > I'm not sure if it's still that "easy" with modern hardware, though. > But maybe you can try... :-) I did try different cards in different slots kind of thing but I didn't think about resetting CMOS after changing each configuration. Thanks for the suggestion, will try that. GregJ