From owner-freebsd-hackers@freebsd.org  Wed Jul 22 17:14:22 2015
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id A48829A7E96;
 Wed, 22 Jul 2015 17:14:22 +0000 (UTC)
 (envelope-from gpalmer@freebsd.org)
Received: from mail.in-addr.com (mail.in-addr.com
 [IPv6:2a01:4f8:191:61e8::2525:2525])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 6BC9C1DA3;
 Wed, 22 Jul 2015 17:14:22 +0000 (UTC)
 (envelope-from gpalmer@freebsd.org)
Received: from gjp by mail.in-addr.com with local (Exim 4.85 (FreeBSD))
 (envelope-from <gpalmer@freebsd.org>)
 id 1ZHxb1-000OnG-Bw; Wed, 22 Jul 2015 18:14:19 +0100
Date: Wed, 22 Jul 2015 18:14:19 +0100
From: Gary Palmer <gpalmer@freebsd.org>
To: Mark Saad <nonesuch@longcount.org>
Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>,
 FreeBSD-Stable ML <freebsd-stable@freebsd.org>
Subject: Re: Interesting Boot failure on HEAD with a large number of IGB nics
Message-ID: <20150722171419.GE41419@in-addr.com>
References: <CAMXt9NbJ5quUXwLhLzU9KzpJ0cXGCbHsnmc1D-L3ycKktn6j7A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAMXt9NbJ5quUXwLhLzU9KzpJ0cXGCbHsnmc1D-L3ycKktn6j7A@mail.gmail.com>
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: gpalmer@freebsd.org
X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
 <freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-hackers>, 
 <mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
 <mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 22 Jul 2015 17:14:22 -0000

On Wed, Jul 22, 2015 at 12:11:09PM -0400, Mark Saad wrote:
> All
>  I am wondering if anyone has run into this issue before , and if there is
> a fix.
> I have a Scalable Informatics siRouter with 24 Intel I350 igb nics and 8
> intel 82599ES ixgbe/ix nic . The SiRouter is a Supermicro X9DRX+-F with a
> bunch of intel nics.
> 
> On Head I run into a boot panic out of the box while plumbing the usb
> controller . The box crashes with
> 
> ehci0: <Intel Patsburg USB 2.0 controller> mem 0xdd923000-0xdd9233ff irq 16
> at device 26.0 on pci0
> panic: Couldn't find an APIC vector for IRQ 16
> 
> However it has only probed 8 of the 24 igb when is crashes. Here is the
> complete boot up with some additional info.
> 
> http://pastebin.com/MfbEwwg4
> 
> Now the interesting thing.
> 
>  DragonFly 4.2x boots fine  see the dmesg here
> http://dmesgd.nycbug.org/index.cgi?action=dmesgd&do=view&id=2778
> 
> OpenBSD boots and finds the nics as well see dmesg here
> http://dmesgd.nycbug.org/index.cgi?action=dmesgd&do=view&id=2777
> 
> NetBSD Pukes while probing something out of the box and I cant get the
> serial console to work.
> 
> Linux 2.6 CentOS 6.6 works
> 
> OmniOS / Illumos-gate and Illumos-joyent work out of the box
> 
> 
> 
> So my goal here is to make the box boot FreeBSD . It had been used prior
> running OmniOS and CentOS and well now I want to make it work. :)
> 
> Any ideas here ?


I'm no expert, but you may want to try setting

hw.igb.num_queues=1

and maybe

hw.ixgbe.num_queues=1

in the boot loader and trying that.

Regards,

Gary

P.S. freebsd-net@ may be a better list than either -stable or -hackers.