Date: Sat, 13 Jan 2018 12:31:37 -0600 From: Valeri Galtsev <galtsev@kicp.uchicago.edu> To: Grzegorz Junka <list1@gjunka.com>, freebsd-questions@freebsd.org, freebsd-drivers@freebsd.org Subject: Re: Server doesn't boot when 3 PCIe slots are populated Message-ID: <061ccfb3-ee6a-71a7-3926-372bb17b3171@kicp.uchicago.edu> In-Reply-To: <ecce3fa6-3909-0947-685c-8a412684e99c@gjunka.com> References: <ecce3fa6-3909-0947-685c-8a412684e99c@gjunka.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 01/13/18 10:21, Grzegorz Junka wrote: > Hello, > > I am installing a FreeBSD server based on Supermicro H8SML-iF. There are > three PCIe slots to which I installed 2 NVMe drives and one network card > Intel I350-T4 (with 4 Ethernet slots). > > I am observing a strange behavior where the system doesn't boot if all > three PCIe slots are populated. It shows this message: > > nvme0: <Generic NVMe Device> mem 0xfd8fc000-0xfd8fffff irq 24 at device > 0.0 on pci1 > nvme0: controller ready did not become 1 within 30000 ms > nvme0: did not complete shutdown within 5 seconds of notification > > The I see a kernel panic/dump and the system reboots after 15 seconds. > > If I remove one card, either one of the NVMe drives or the network card, > the system boots fine. Also, if in BIOS I set PnP OS to YES then > sometimes it boots (but not always). If I set PnP OS to NO, and all > three cards are installed, the system never boots. > > When the system boots OK I can see that the network card is reported as > 4 separate devices on one of the PCIe slots. I tried different NVMe > drives as well as changing which device is installed to which slot but > the result seems to be the same in any case. > > What may be the issue? Amount of power drawn by the hardware? Too many > devices not supported by the motherboard? Too many interrupts for the > FreeBSD kernel to handle? That would be my first suspicion. Either total power drawn off the power supply. Or total power drawn off the PCI[whichever it is] bus power leads. Check if any of the add-on cards have extra power port (many video cards do). Card likely will work without extra power connected to it, but connecting extra power on the card may solve your problem. Next: borrow more powerful power supply and see if that resolves the issue. Or temporarily disconnect everything else (like all hard drives), and boot with all three cards off live CD, and see if that doesn't crash, then it is marginally insufficient power supply. I hope this helps. Valeri > > Any help would be greatly appreciated. > > GregJ > > _______________________________________________ > freebsd-questions@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to > "freebsd-questions-unsubscribe@freebsd.org" -- ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?061ccfb3-ee6a-71a7-3926-372bb17b3171>