Date: Mon, 13 Sep 2004 11:08:58 -0700 From: Julian Elischer <julian@elischer.org> To: John Baldwin <jhb@FreeBSD.org> Cc: Peter Wemm <peter@FreeBSD.org> Subject: Re: [Patch] panics/hangs with preemption and threads. Message-ID: <4145E23A.9040108@elischer.org> In-Reply-To: <200409131331.03881.jhb@FreeBSD.org> References: <4143EF29.2080404@elischer.org> <200409131331.03881.jhb@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
John Baldwin wrote:
>On Sunday 12 September 2004 02:39 am, Julian Elischer wrote:
>
>
>>Guys I think I found a (the?) major cause for the corruptions of the
>>ksegrp/thread runqueue for threaded processes when Premption is turned on..
>>
>>When a thread is scheduled in setrunqueue() the firt thing that is done
>>is that it is put in the correct place in the ksegrp's run queue,.
>>then if it is in the top N spots (where N is the defined concurrency
>>and is usually <= NCPU) it is passed down to the system scheduler
>>using sched_add().
>>Sched_add can call maybe_preempt() which can decide to switch out the
>>current thread and switch to the new one immediatly.
>>The trouble with that is that we have already put the new one on the
>>ksegrp's run queue! When that thread is next put on the run queue using
>>setrunqueue() it is already there, and we end up with an infinitly looping
>>run queue. Any code that follows that list will never end. and the system
>>will freeze.
>>
>>Here is a patch that solves it but I'm not happy about it..
>>John, you wrote the preemption code..
>>do you have any ideas about how to do this cleaner?
>>
>>One possibility is to make sched_add return a value that indicates if the
>>thread was handled immediatly. that would allow setrunqueue to only set it
>>into the ksegrp's run queue if it was not already handled.
>>
>>Other suggestions welcome.
>>
>>
>
>I think it's probably a good idea to do the preemption check before putting
>the thread on the kse group. However, that might break ULE and some things
>it does (ULE pins interrupt threads but does it in sched_add, perhaps that is
>a hack and the pinning should be done in ithread_schedule instead). Changing
>sched_add() to return a boolean similar to maybe_preempt() is probably ok as
>an alternative then. Also, there's really no need for an additional
>SRQ_NOPREEMPT flag, that just duplicates critical_enter()/critical_exit().
>The same is probably true of SRQ_YIELDING and SRQ_MYSELF (preemption already
>doesn't preempt to curthread since the priorities are equal). The place that
>uses SRQ_YIELDING can just add a critical section around the call to
>setrunqueue(). Note that when a preemption is deferred due to a nested
>critical section, the preemption doesn't actually occur until the outermost
>critical section is exited, so if you do this:
>
> mtx_lock_spin(&sched_lock);
> blah blah;
> if (foo) {
> critical_enter();
> setrunqueue(td2);
> critical_exit();
> mi_switch(NULL, SW_VOL);
> }
> mtx_unlock_spin(&sched_lock);
>
>That won't actually preempt.
>
The flags were not only for this problem but also for another scheduer
I was playign with and
I thought they might be useful in trying to find/fix this problem.
The critical nest thing is ok, but I had a lot of debug code in at one
stage and I wanted to know
more about where I had come from. so the flags I had already gave me that..
I don't see the harm in having more information but I realised
afterwards that there was already some of this info available..
for MYSELF, curthead == newtd and for INTR teh process/thread is marked
as an interrupt thread, which leaves only "Yielding"
which I do think is useful info to know. but the critnest solves teh
same problem in a more specific manner.
>
>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4145E23A.9040108>