Date: Thu, 2 May 2013 18:55:00 GMT From: John Baldwin <jhb@FreeBSD.org> To: Perforce Change Reviews <perforce@FreeBSD.org> Subject: PERFORCE change 228335 for review Message-ID: <201305021855.r42It0Mo097216@skunkworks.freebsd.org>
next in thread | raw e-mail | index | archive | help
http://p4web.freebsd.org/@@228335?ac=10 Change 228335 by jhb@jhb_fiver on 2013/05/02 18:54:47 WIP on updating bits of this. Affected files ... .. //depot/projects/smpng/share/man/man9/locking.9#12 edit Differences ... ==== //depot/projects/smpng/share/man/man9/locking.9#12 (text+ko) ==== @@ -33,11 +33,11 @@ .Sh DESCRIPTION The .Em FreeBSD -kernel is written to run across multiple CPUs and as such requires -several different synchronization primitives to allow the developers -to safely access and manipulate the many data types required. +kernel is written to run across multiple CPUs and as such provides +several different synchronization primitives to allow developers +to safely access and manipulate many data types. .Ss Mutexes -Mutexes (also erroneously called "sleep mutexes") are the most commonly used +Mutexes (also called "blocking mutexes") are the most commonly used synchronization primitive in the kernel. A thread acquires (locks) a mutex before accessing data shared with other threads (including interrupt threads), and releases (unlocks) it afterwards. @@ -55,31 +55,32 @@ See .Xr mutex 9 for details. -.Ss Spin mutexes -Spin mutexes are variation of basic mutexes; the main difference between -the two is that spin mutexes never yield the processor - instead, they spin, -waiting for the thread holding the lock, -(which must be running on another CPU), to release it. -Spin mutexes disable interrupts while the held so as to not get pre-empted. -Since disabling interrupts is expensive, they are also generally slower. +.Ss Spin Mutexes +Spin mutexes are a variation of basic mutexes; the main difference between +the two is that spin mutexes never block. +Instead, they spin while waiting for the lock to be released. +Note that a thread that holds a spin mutex must never yield its CPU to +avoid deadlock. +Unlike ordinary mutexes, spin mutexes disable interrupts when acquired. +Since disabling interrupts can be expensive, they are generally slower. Spin mutexes should be used only when necessary, e.g. to protect data shared with interrupt filter code (see .Xr bus_setup_intr 9 -for details). -.Ss Pool mutexes -With most synchronization primitives, such as mutexes, programmer must -provide a piece of allocated memory to hold the primitive. +for details), +or for scheduler internals. +.Ss Mutex Pools +With most synchronization primitives, such as mutexes, the programmer must +provide memory to hold the primitive. For example, a mutex may be embedded inside the structure it protects. -Pool mutex is a variant of mutex without this requirement - to lock or unlock -a pool mutex, one uses address of the structure being protected with it, -not the mutex itself. -Pool mutexes are seldom used. +Mutex pools provide a preallocated set of mutexes to avoid this +requirement. +Note that mutexes from a pool may only be used as leaf locks. .Pp See .Xr mtx_pool 9 for details. -.Ss Reader/writer locks -Reader/writer locks allow shared access to protected data by multiple threads, +.Ss Reader/Writer Locks +Reader/writer locks allow shared access to protected data by multiple threads or exclusive access by a single thread. The threads with shared access are known as .Em readers @@ -91,26 +92,16 @@ Reader/writer locks can be treated as mutexes (see above and .Xr mutex 9 ) with shared/exclusive semantics. -More specifically, regular mutexes can be -considered to be equivalent to a write-lock on an -.Em rw_lock. -The -.Em rw_lock -locks have priority propagation like mutexes, but priority -can be propagated only to an exclusive holder. +Reader/writer locks support priority propagation like mutexes, +but priority is propagated only to an exclusive holder. This limitation comes from the fact that shared owners are anonymous. -Another important property is that shared holders of -.Em rw_lock -can recurse, but exclusive locks are not allowed to recurse. -This ability should not be used lightly and -.Em may go away. .Pp See .Xr rwlock 9 for details. -.Ss Read-mostly locks -Mostly reader locks are similar to +.Ss Read-Mostly Locks +Read-mostly locks are similar to .Em reader/writer locks but optimized for very infrequent write locking. .Em Read-mostly @@ -122,21 +113,41 @@ See .Xr rmlock 9 for details. +.Ss Sleepable Read-Mostly Locks +Sleepable read-mostly locks are a variation on read-mostly locks. +Threads holding an exclusive lock may sleep, +but threads holding a shared lock may not. +Priority is propagated to shared owners but not to exclusive owners. .Ss Shared/exclusive locks Shared/exclusive locks are similar to reader/writer locks; the main difference -between them is that shared/exclusive locks may be held during unbounded sleep -(and may thus perform an unbounded sleep). -They are inherently less efficient than mutexes, reader/writer locks -and read-mostly locks. -They do not support priority propagation. -They should be considered to be closely related to -.Xr sleep 9 . -They could in some cases be -considered a conditional sleep. +between them is that shared/exclusive locks may be held during unbounded sleep. +Acquiring a contested shared/exclusive lock can perform an unbounded sleep. +These locks do not support priority propagation. .Pp See .Xr sx 9 for details. +.Ss Lockmanager locks +Lockmanager locks are sleepable shared/exclusive locks used mostly in +.Xr VFS 9 +.Po +as a +.Xr vnode 9 +lock +.Pc +and in the buffer cache +.Po +.Xr BUF_LOCK 9 +.Pc . +They have features other lock types do not have such as sleep +timeouts, blocking upgrades, +writer starvation avoidance, draining, and an interlock mutex, +but this makes them complicated to both use and implement; +for this reason, they should be avoided. +.Pp +See +.Xr lock 9 +for details. .Ss Counting semaphores Counting semaphores provide a mechanism for synchronizing access to a pool of resources. @@ -149,43 +160,21 @@ .Xr sema 9 for details. .Ss Condition variables -Condition variables are used in conjunction with mutexes to wait for +Condition variables are used in conjunction with locks to wait for conditions to occur. -A thread must hold the mutex before calling the +A thread must hold the associated lock before calling the .Fn cv_wait* , functions. -When a thread waits on a condition, the mutex +When a thread waits on a condition, the lock is atomically released before the thread yields the processor, then reacquired before the function call returns. +Condition variables can be used with blocking mutexes, +reader/writer locks, and shared/exclusive locks. .Pp See .Xr condvar 9 for details. -.Ss Giant -Giant is an instance of a mutex, with some special characteristics: -.Bl -enum -.It -It is recursive. -.It -Drivers and filesystems can request that Giant be locked around them -by not marking themselves MPSAFE. -Note that infrastructure to do this is slowly going away as non-MPSAFE -drivers either became properly locked or disappear. -.It -Giant must be locked first before other locks. -.It -It is OK to hold Giant while performing unbounded sleep; in such case, -Giant will be dropped before sleeping and picked up after wakeup. -.It -There are places in the kernel that drop Giant and pick it back up -again. -Sleep locks will do this before sleeping. -Parts of the network or VM code may do this as well, depending on the -setting of a sysctl. -This means that you cannot count on Giant keeping other code from -running if your code sleeps, even if you want it to. -.El -.Ss Sleep/wakeup +.Ss Sleep/Wakeup The functions .Fn tsleep , .Fn msleep , @@ -236,36 +225,54 @@ In addition, all of the sleep routines will fully drop the .Va Giant mutex -(even if recursed) +.Pq even if recursed while the thread is suspended and will reacquire the .Va Giant -mutex before the function returns. +mutex +.Pq restoring any recursion +before the function returns. .Pp See .Xr sleep 9 for details. -.Ss Lockmanager locks -Shared/exclusive locks, used mostly in -.Xr VFS 9 , -in particular as a -.Xr vnode 9 -lock. -They have features other lock types do not have, such as sleep timeout, -writer starvation avoidance, draining, and interlock mutex, but this makes them -complicated to implement; for this reason, they are deprecated. -.Pp -See -.Xr lock 9 -for details. +.Ss Giant +Giant is a special mutex used to protect data structures that do not +yet have their own locks. +Since it provides semantics akin to the old +.Xr spl 9 +interface, +Giant has special characteristics: +.Bl -enum +.It +It is recursive. +.It +Drivers and filesystems can request that Giant be locked around them +by not marking themselves MPSAFE. +Note that infrastructure to do this is slowly going away as non-MPSAFE +drivers either became properly locked or disappear. +.It +Giant must be locked before other non-sleepable locks. +.It +Giant is dropped during unbounded sleeps and reacquired after wakeup. +.It +There are places in the kernel that drop Giant and pick it back up +again. +Sleep locks will do this before sleeping. +Parts of the network or VM code may do this as well, depending on the +setting of a sysctl. +This means that you cannot count on Giant keeping other code from +running if your code sleeps, even if you want it to. +.El .Sh INTERACTIONS The primitives interact and have a number of rules regarding how they can and can not be combined. -Many of these rules are checked using the -.Xr witness 4 -code. +Many of these rules are checked by +.Xr witness 4 . .Ss Bounded vs. unbounded sleep +A bounded sleep i + The following primitives perform bounded sleep: - mutexes, pool mutexes, reader/writer locks and read-mostly locks. +mutexes, pool mutexes, reader/writer locks and read-mostly locks. .Pp The following primitives may perform an unbounded sleep: shared/exclusive locks, counting semaphores, condition variables, sleep/wakeup and lockmanager locks.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201305021855.r42It0Mo097216>