Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 5 Jan 1999 16:22:34 -0800 (PST)
From:      Matthew Dillon <dillon@apollo.backplane.com>
To:        Alfred Perlstein <bright@hotjobs.com>
Cc:        Terry Lambert <tlambert@primenet.com>, wes@softweyr.com, hackers@FreeBSD.ORG
Subject:   Re: question about re-entrancy.
Message-ID:  <199901060022.QAA98857@apollo.backplane.com>
References:   <Pine.BSF.4.05.9901051838400.37756-100000@bright.fx.genx.net>

next in thread | previous in thread | raw e-mail | index | archive | help
:> :
:> :-Alfred
:> 
:>     Generally speaking you do not want to use multi-valued locks for SMP
:>     operations.  i.e. the concept of a 'shared' lock verses an 'exclusive'
:>     lock does not work well for locks used to manage SMP functions.  The
:>     reason it doesn't work well is because of truely serious starvation
:>     issues -- it is virtually always more efficient for the fine-grained 
:>     SMP locks as *only* exclusive locks and then work on optimizing the 
:>     pipelines.
:
:You may have misunderstood me, several threads could enter a 'read' state,
:when a 'write' must occur no more threads are allowed into the 'read'
:state until the 'write' state is complete.  All the readers have to drain
:out of their code before a write can take place.
:
:Or is this exactly what you are objecting to?

    This is the typical hack that people make, but it isn't a good idea,
    especially for a fine-grained SMP lock.  The reason it isn't a good idea
    is multi-fold fold:  First, it adds complexity and cpu cycles to the
    locking mechanism that can often equal the complexity of the code being
    locked, making the result less efficient. 

    Second, because we are talking about fine-grained SMP locks here, not 
    course-grained locks, and it is more important to maintain the time 
    locality of the pipeline then to try to optimize parallel readers,
    because even if you can use read-locks in one section of code you won't
    be able to use them everywhere, so parallelizing one section of code 
    doesn't gain you anything when you have to serialize another other then
    destroying the time locality of the pipeline. 

    The 'bunching' that occurs with multiple cpu's all trying for the same
    lock destroys the efficiency of the *hardware* cache coherency mechanisms,
    further slowing things down.  But if the processes are already pipelined,
    the memory location containing the lock tends to be 'free', allowing a
    cpu to grab it more quickly.

    Third, because you have a lock starvation issue even when you block 
    further readers unless you also block further writers in the face of a 
    held write lock and pending reader requests.  This can lead into flip-flop
    situations which devolve down into just doing an exclusive lock in the
    first place... but with a less efficient locking mechanism.  You have
    to protect against the inefficient flip-flop situation.

    Fourth, when you have multiple cpu's vying for the *same* lock (the
    bunching, again), not only is hardware cache coherency strained but 
    you get into the situation where cpu's can jump ahead of each other going
    through the same pipeline.  This disrupts the pipeline further, creating
    temporary interlocks and more bunching down the line, destroying the
    time locality of the pipeline.

    Usually for a fine-grained SMP lock, the key issue is to be able to get
    and release the lock as quickly as possible.  If it takes 4 cpu cycles
    to get a simple lock and 4 to release it in a pipeline, it might take 8 cpu
    cycles to get a read/write lock (and 4 to release it).  When vying for the
    *same* lock, it might take 10 cycles to get it and 8 cycles to release
    it due to hardware cache coherency collisions (or even memory commit
    collisions!).  If this is a fine-grained SMP lock, there is probably
    only one or two cpu's going through it even in a 128-cpu SMP machine.
    You want to optimize for the case that is most likely to occur and make
    it go as fast as possible.

						-Matt

:>     course-grained locks should typically *not* be SMP locks but instead be
:>     structural locks with real blocking (the 'go to sleep' type of blocking).
:>     The exception, of course, is the One-Big-Lock model:  the model is really
:>     just a big hack so the normal rules do not apply to it.
:
:Well yes, ala SunOS 4.x.x :)
:
:-Alfred 
:-who's finding -current a better source of knowledge than his short
:college stint :)
:
:thanks for the patience and explanations.

    Matthew Dillon  Engineering, HiWay Technologies, Inc. & BEST Internet 
                    Communications & God knows what else.
    <dillon@backplane.com> (Please include original email in any response)    

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199901060022.QAA98857>