Date: Mon, 5 Jan 2015 15:43:16 +0200 From: Konstantin Belousov <kostikbel@gmail.com> To: Hans Petter Selasky <hps@selasky.org> Cc: markb@mellanox.com, FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: [RFC] Start SMP subsystem earlier Message-ID: <20150105134316.GE42409@kib.kiev.ua> In-Reply-To: <54AA8F19.9030300@selasky.org> References: <54AA8F19.9030300@selasky.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jan 05, 2015 at 02:18:17PM +0100, Hans Petter Selasky wrote: > Hi, > > There is a limitiation on the number of interrupt vectors available when > only a single processor is running. To have more interrupts available we > need to start SMP earlier when building a monotolith kernel and not > loading drivers as modules. The driver in question is a network driver > and because it cannot be started after SI_SUB_ROOT_CONF due to PXE > support I see no other option than to move SI_SUB_SMP earlier. > > Suggested patch: > > > Index: sys/kernel.h > > =================================================================== > > --- sys/kernel.h (revision 276691) > > +++ sys/kernel.h (working copy) > > @@ -152,6 +152,7 @@ > > SI_SUB_KPROF = 0x9000000, /* kernel profiling*/ > > SI_SUB_KICK_SCHEDULER = 0xa000000, /* start the timeout events*/ > > SI_SUB_INT_CONFIG_HOOKS = 0xa800000, /* Interrupts enabled config */ > > + SI_SUB_SMP = 0xa850000, /* start the APs*/ > > SI_SUB_ROOT_CONF = 0xb000000, /* Find root devices */ > > SI_SUB_DUMP_CONF = 0xb200000, /* Find dump devices */ > > SI_SUB_RAID = 0xb380000, /* Configure GEOM classes */ > > @@ -165,7 +166,6 @@ > > SI_SUB_KTHREAD_BUF = 0xea00000, /* buffer daemon*/ > > SI_SUB_KTHREAD_UPDATE = 0xec00000, /* update daemon*/ > > SI_SUB_KTHREAD_IDLE = 0xee00000, /* idle procs*/ > > - SI_SUB_SMP = 0xf000000, /* start the APs*/ > > SI_SUB_RACCTD = 0xf100000, /* start racctd*/ > > SI_SUB_LAST = 0xfffffff /* final initialization */ > > }; Did you inspected all reordered sysinit routines and ensured that the reordering is safe ? At very least, SUB_SMP starts event timers, while KTHREAD_IDLE is about configuring some hardware which might be required/not ready for that. > > This fixes a problem for Mellanox drivers in the OFED layer. Possibly we > need to move the SMP even earlier to not miss the generic FreeBSD PCI > device enumeration or maybe this is not possible. Does anyone know how > early we can start SMP?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150105134316.GE42409>