From owner-freebsd-questions@freebsd.org Sat Jan 13 17:56:09 2018 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09340E762C2; Sat, 13 Jan 2018 17:56:09 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-ot0-x22c.google.com (mail-ot0-x22c.google.com [IPv6:2607:f8b0:4003:c0f::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE8456F814; Sat, 13 Jan 2018 17:56:08 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: by mail-ot0-x22c.google.com with SMTP id a24so7607638otd.4; Sat, 13 Jan 2018 09:56:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=KujieS9t0pmshPCmRP9xZnOJBagSIWldVCM1+zesmDU=; b=rsfigNQnCQoqkCjITZ+CeeuNQy5u19g1kjRaChg8l+xMkgCVhg9tMPcXSXpOVqSt+N 7mGyRpqmh1EsenJB/5iFrxk5HAPW8T3Q2g2WkLiYd805+dZUKrntprDuV2uWZql4abUd yFWS1XnfLE9n2yfod497oZ3ScpDVz8FY7d1tnv4AMqwOtsqKps8H2pExnpya/1l3KmpU 7D65eacAXljvE1wSdtf4E4kV1OI1mszHEcT8WZsW/JXUt5YQJvqChPt5Qhp0Qta7nne8 /52anvR4tT9mOMsNaDO1H2DdqOLzPn4eO2Pu7h81YEh6bGZJP2t6ttN3l3lkW5684Mb3 FTAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=KujieS9t0pmshPCmRP9xZnOJBagSIWldVCM1+zesmDU=; b=UfZW/hUkd+qfaKiDjM8zBNm+0uVf2W4UadPd42SYLvBej1++Nm2EBskEEmyv+26dqU nuN5TAF0vUBxYHBXGClw1JmON3QjQocS64I41eCV6xCP8ylL02kZ/OX2vBWRH5fHC3JI n+YwDh1ZenEa/UboCZpkLbsVpuFij+um50mVCBZWqEeQAdK2rkpO+rClwvXZX28a4bMP adsRH8T5Hp7ihyXY+Q1jlSD11GsXmo9A23MnMSs2dylHoxyY48PKd3lUgXBL7VpwY+Zc bcx2GcKUuOgp+ljAFQBCjNMJWF/5DaTLZbWsU5XnOlaPA1O0+2xniGJVG0w83zkR4CAs t8CA== X-Gm-Message-State: AKwxytexeSBM/gIebGCjG0fB4L5PuNaw5T/xvoEE33azISwABNaSR2RF /BZZHii1wgPDKN9dIkQ8Lm8BbiZ4cdOwLAMHUAw= X-Google-Smtp-Source: ACJfBoue3RcqP8BGWIQRIyPBVyHBd8lhJHC97rE7YYJR3KQOddZNyX6uAyaMNImdE/ODgdqw+55wApQk88k+M4EVh+A= X-Received: by 10.157.55.246 with SMTP id x109mr3424593otb.266.1515866168011; Sat, 13 Jan 2018 09:56:08 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.0.2 with HTTP; Sat, 13 Jan 2018 09:56:07 -0800 (PST) In-Reply-To: References: From: Mehmet Erol Sanliturk Date: Sat, 13 Jan 2018 20:56:07 +0300 Message-ID: Subject: Re: Server doesn't boot when 3 PCIe slots are populated To: Grzegorz Junka Cc: FreeBSD Questions Mailing List , freebsd-drivers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Jan 2018 17:56:09 -0000 On Sat, Jan 13, 2018 at 7:21 PM, Grzegorz Junka wrote: > Hello, > > I am installing a FreeBSD server based on Supermicro H8SML-iF. There are > three PCIe slots to which I installed 2 NVMe drives and one network card > Intel I350-T4 (with 4 Ethernet slots). > > I am observing a strange behavior where the system doesn't boot if all > three PCIe slots are populated. It shows this message: > > nvme0: mem 0xfd8fc000-0xfd8fffff irq 24 at device > 0.0 on pci1 > nvme0: controller ready did not become 1 within 30000 ms > nvme0: did not complete shutdown within 5 seconds of notification > > The I see a kernel panic/dump and the system reboots after 15 seconds. > > If I remove one card, either one of the NVMe drives or the network card, > the system boots fine. Also, if in BIOS I set PnP OS to YES then sometimes > it boots (but not always). If I set PnP OS to NO, and all three cards are > installed, the system never boots. > > When the system boots OK I can see that the network card is reported as 4 > separate devices on one of the PCIe slots. I tried different NVMe drives as > well as changing which device is installed to which slot but the result > seems to be the same in any case. > > What may be the issue? Amount of power drawn by the hardware? Too many > devices not supported by the motherboard? Too many interrupts for the > FreeBSD kernel to handle? > > Any help would be greatly appreciated. > > GregJ > > _______________________________________________ > > >From my experience from other trade marked main boards , an action may be to check manual of your server board to see whether there are rules about use of these slots : Sometimes differently shaped slots are supplied with same ports : If one slot is occupied , the other slot should be left open , or rules about not to insert such a kind of device into a slot , for example , graphic cards . Mehmet Erol Sanliturk