From owner-freebsd-questions@freebsd.org Tue Jan 16 05:27:24 2018 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5AC47E777A2 for ; Tue, 16 Jan 2018 05:27:24 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from mout.kundenserver.de (mout.kundenserver.de [217.72.192.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.kundenserver.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D230E2A21 for ; Tue, 16 Jan 2018 05:27:23 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from r56.edvax.de ([92.195.18.98]) by mrelayeu.kundenserver.de (mreue103 [212.227.15.183]) with ESMTPA (Nemesis) id 0MVasZ-1eLCcs1WKl-00Z3dZ; Tue, 16 Jan 2018 06:27:09 +0100 Date: Tue, 16 Jan 2018 06:27:09 +0100 From: Polytropon To: Grzegorz Junka Cc: FreeBSD Questions Mailing List Subject: Re: Server doesn't boot when 3 PCIe slots are populated Message-Id: <20180116062709.a6c75fa4.freebsd@edvax.de> In-Reply-To: <8fee9df3-c40b-addb-b3c9-bedd90683d62@gjunka.com> References: <3d0ad00c-5214-71b0-017b-c2d5ba608e37@gjunka.com> <8df1e967-01e0-d3c2-e14c-64c7fc8c66b0@gjunka.com> <0e582bdb-e1f9-438c-3da2-2bcdc950aab5@gjunka.com> <57715.108.68.169.115.1516033864.squirrel@cosmo.uchicago.edu> <8fee9df3-c40b-addb-b3c9-bedd90683d62@gjunka.com> Reply-To: Polytropon Organization: EDVAX X-Mailer: Sylpheed 3.1.1 (GTK+ 2.24.5; i386-portbld-freebsd8.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:23RqALmHdpFXNGHPUx14ICxjRjnAa6X9JSFjrtI9RECbawzjw9q DCnsVT7DzRge8jtswDX/loohQZ0JnosrqUKiXy/sfvsRAhSp5VrYZXcQCmv4MZ8fxRPO80N oOFFY4gMrpp5o4AfCubgoY2vouNBRq+F0a21NUd8vBzgje5WUD7AjTIaoly9xVhlMnyc3WK emLLjVWp9nIFgtUtOcjgg== X-UI-Out-Filterresults: notjunk:1;V01:K0:+zfAk+UjLRQ=:p8U6t7IIkf+7AE3Z6FN5VW tEtCz142ADWmCQRbo9S05UFeQEXnomOzpDjtFZ0I/MJWXhK1P+oHAswjlacDu6pw/NxLFZXZf vPrmCTof2U4pRI5YAeYaKFzBGl66bMxl5RwSDvhra6sdgMKx0Vxq1ZouvuPYC+EcYMCxpm5R0 IKadAuAbNedLgmnHAjB7yjeGGRJKeCHjjKIB4BilP7rnIfSqChq5cbZgOHz3QzSKLN8ypdeCp mIIhsbCPf2mlHz2nXEGdfwDCF6zhQA5tKqWEZnnqokZ+aMK4TcHrpJZzTqoL/dNO2+kVdDXTZ UYrHb+2HzGOOdc7biLAOj+ejzoNmnyOIl5tN7+XnVMMhoJnmgepppKaDabrSKrfyNnPxGjsSn ct/aFV+ZG0Z46WJDzjCdI9TOStK54hgKoNWGR9D5hwsgCL16ZdN/H5KDvWZ0Xm9nuiOgh1EK9 /qWw4fXuzSMJBjCb9Hc1LUNt6qv79WpPAeuafbbkHxcKqVhpXjvzJ2cQ4lg7UnJLMQoFi7fq2 I9NoefHEIwzVRyhRcd3Lnk7dRP6ND3Ekq5sjXi53wDfnUGAuJvjCZCXNEtE6fUCJ+kxoM5/zM 2b90TmXUksVVSktYK6UmboMIukBYN1x8XuV/fdChV1MxCP7IHNwhZeiVA/H8CPzRM1BC7tgQ2 n/HZa5jxVhYN7xTsDF4D7robBa7rki1ZSiL9EqxlVS6rPOZ6G/wXH7Hu+aUfMw5a2PVy5iR5S TRwQKL5PfODogYlNc3kEWL2gEHGUvUAZK2HOHJ+2IPPTXt2bePcaeI4U45U= X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jan 2018 05:27:24 -0000 On Mon, 15 Jan 2018 22:30:18 +0000, Grzegorz Junka wrote: > I tried a different pair of NVMe cards (different adapters with > different SSD disks) and the result was exactly the same. Note, that the > pair that I tried was previously working in this motherboard without > problems for many months, so it's safe to assume that the addition of > the network card is causing this problem. But then again, the network > card with one of the NVMe drives works fine too. > > Could be that all three cause some sort of impedance mismatch but that's > kind of hard to believe - these are simple cards, there is almost no > circuits on the NVMe adapter and the network card is just a chipset with > 4 slots. No idea if it still applies, but: I don't know if this idea has come uo yet, but many _many_ years ago, I was in a comparable situation. This was of course "traditional PCI times" where a common system board had 4 - 6 PCI slots. In that particular system, 2 NICs, an ATA "controller", a TV card, and a sound card (5 PCI cards total + 1 AGP graphics card) were installed. The problem was that the system wouldn't start booting after BIOS POST. The solution was to re-arrange cards, followed by a PCI configuration reset in the CMOS setup, followed by another boot attempt. In one specific card configuration, the system worked as expected, and FreeBSD (at that time, probably v4 or v5) would detect all the cards without any problems and attach the appropriate drivers. Removing one of the cards, or exchanging card positions (for "more convenient cabling") would render the system non-booting again. Back to the one configuration that worked - system booted. Let me emphasize that it was required to reset the PCI configuration in the CMOS setup every time such a card change was made, as returning to the "verified locations" without doing so would _not_ let the system boot properly. I'm not sure if it's still that "easy" with modern hardware, though. But maybe you can try... :-) -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ...