From owner-freebsd-current@FreeBSD.ORG Wed Apr 6 17:21:05 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 360D81065674 for ; Wed, 6 Apr 2011 17:21:05 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id E25658FC1F for ; Wed, 6 Apr 2011 17:21:04 +0000 (UTC) Received: by ywf9 with SMTP id 9so752180ywf.13 for ; Wed, 06 Apr 2011 10:21:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=2mefYI23DOumE277csOXse/4eVXdZUAhD8mTuqOJf7Y=; b=ihKjXch+3lv3yH0gTgZ0mIiLIU333SzTE/q1DtEcEF1Rmzd/2icQwP37HjH8gIXRnI JepIbN79R2uCzj3uOESw4ome9MMcpZH40lkqOxFHjpm44K3SZaBRUOC3KdUdf02vyKOP P5UllGEaufmc45J0mAmXXZ+kQpwz9Hn5aakvA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=k3IkDLnsxdqmhXePLKsRZEEeqtq/ZtI2xmWnsUweajdtwG7VNnkrEk28BnMsMqi8KZ olzo2jnj/nm44TaSfr4dIBZoJHnlddqHzirRJQXxIK4vTfoz30AfZPztv9Vu1jSLQz7+ VTBqBS9QZ8WGZbZLQbeetnL03cl0IBmKlMfAQ= MIME-Version: 1.0 Received: by 10.236.136.130 with SMTP id w2mr1372534yhi.141.1302110464163; Wed, 06 Apr 2011 10:21:04 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.236.103.34 with HTTP; Wed, 6 Apr 2011 10:21:04 -0700 (PDT) In-Reply-To: References: <201104060836.56542.jhb@freebsd.org> Date: Wed, 6 Apr 2011 13:21:04 -0400 X-Google-Sender-Auth: XFRki6VuQdvCaAXdHDYPHZwT1Zk Message-ID: From: Attilio Rao To: Ryan Stone Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current@freebsd.org Subject: Re: sched_4bsd startup crash trying to run a bound thread on an AP that hasn't started X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2011 17:21:05 -0000 2011/4/6 Ryan Stone : > On Wed, Apr 6, 2011 at 8:36 AM, John Baldwin wrote: >> Hummm. =C2=A0Patching 4BSD to use the same route as ULE may be the best = solution >> for now if that is easiest. =C2=A0Alternatively, you could change 4BSD's >> sched_add() to not try to kick other CPUs until smp_started is true. > > At first I thought that it was a consequence of the way it does CPU > affinity, but now I see that it shortcuts if smp_started is not true. > How about something like the following for 4BSD? > > --- sched_4bsd.c =C2=A0 =C2=A0 =C2=A0 =C2=A0(revision 220222) > +++ sched_4bsd.c =C2=A0 =C2=A0 =C2=A0 =C2=A0(working copy) > @@ -1242,14 +1242,14 @@ > =C2=A0 =C2=A0 =C2=A0 =C2=A0} > =C2=A0 =C2=A0 =C2=A0 =C2=A0TD_SET_RUNQ(td); > > - =C2=A0 =C2=A0 =C2=A0 if (td->td_pinned !=3D 0) { > + =C2=A0 =C2=A0 =C2=A0 if (smp_started && td->td_pinned !=3D 0) { > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cpu =3D td->td_las= tcpu; > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ts->ts_runq =3D &r= unq_pcpu[cpu]; > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0single_cpu =3D 1; > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0CTR3(KTR_RUNQ, > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"sch= ed_add: Put td_sched:%p(td:%p) on cpu%d runq", ts, td, > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cpu)= ; > - =C2=A0 =C2=A0 =C2=A0 } else if (td->td_flags & TDF_BOUND) { > + =C2=A0 =C2=A0 =C2=A0 } else if (smp_started && (td->td_flags & TDF_BOUN= D)) { > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* Find CPU from b= ound runq. */ > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0KASSERT(SKE_RUNQ_P= CPU(ts), > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0("sc= hed_add: bound td_sched not on cpu runq")); > @@ -1258,7 +1258,7 @@ > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0CTR3(KTR_RUNQ, > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0"sch= ed_add: Put td_sched:%p(td:%p) on cpu%d runq", ts, td, > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cpu)= ; > - =C2=A0 =C2=A0 =C2=A0 } else if (ts->ts_flags & TSF_AFFINITY) { > + =C2=A0 =C2=A0 =C2=A0 } else if (smp_started && (ts->ts_flags & TSF_AFFI= NITY)) { > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* Find a valid CP= U for our cpuset */ > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cpu =3D sched_pick= cpu(td); > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ts->ts_runq =3D &r= unq_pcpu[cpu]; > > The flow control is a bit awkward because of the multiple > affinity/bound cpu cases. =C2=A0If somebody prefers the code to be > structured differently I'd be open to suggestions. That is more or less what ULE does -- in ULE it is simpler because it goes via sched_pickcpu(), which still returns always CPU0 if APs still didn't kick off. I would also add a comment on top explaining the check, eventually, but otherwise looks fine. Attilio --=20 Peace can only be achieved by understanding - A. Einstein