Skip site navigation (1)Skip section navigation (2)
Date:      Sat, 13 Jan 2018 16:21:24 +0000
From:      Grzegorz Junka <list1@gjunka.com>
To:        freebsd-questions@freebsd.org, freebsd-drivers@freebsd.org
Subject:   Server doesn't boot when 3 PCIe slots are populated
Message-ID:  <ecce3fa6-3909-0947-685c-8a412684e99c@gjunka.com>

Next in thread | Raw E-Mail | Index | Archive | Help
Hello,

I am installing a FreeBSD server based on Supermicro H8SML-iF. There are 
three PCIe slots to which I installed 2 NVMe drives and one network card 
Intel I350-T4 (with 4 Ethernet slots).

I am observing a strange behavior where the system doesn't boot if all 
three PCIe slots are populated. It shows this message:

nvme0: <Generic NVMe Device> mem 0xfd8fc000-0xfd8fffff irq 24 at device 
0.0 on pci1
nvme0: controller ready did not become 1 within 30000 ms
nvme0: did not complete shutdown within 5 seconds of notification

The I see a kernel panic/dump and the system reboots after 15 seconds.

If I remove one card, either one of the NVMe drives or the network card, 
the system boots fine. Also, if in BIOS I set PnP OS to YES then 
sometimes it boots (but not always). If I set PnP OS to NO, and all 
three cards are installed, the system never boots.

When the system boots OK I can see that the network card is reported as 
4 separate devices on one of the PCIe slots. I tried different NVMe 
drives as well as changing which device is installed to which slot but 
the result seems to be the same in any case.

What may be the issue? Amount of power drawn by the hardware? Too many 
devices not supported by the motherboard? Too many interrupts for the 
FreeBSD kernel to handle?

Any help would be greatly appreciated.

GregJ




Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?ecce3fa6-3909-0947-685c-8a412684e99c>