Date: Mon, 12 Nov 2001 18:31:08 -0800
From: Terry Lambert <tlambert2@mindspring.com>
To: John Baldwin <jhb@FreeBSD.org>
Cc: Julian Elischer <julian@elischer.org>, freebsd-arch@FreeBSD.org, Robert Watson <rwatson@FreeBSD.org>, Matthew Dillon <dillon@apollo.backplane.com>
Subject: Re: cur{thread/proc}, or not.
Message-ID: <3BF085EC.AEE7DE9C@mindspring.com>
References: <XFMail.011112162432.jhb@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
John Baldwin wrote: > > ANyhow can you explain the idea of a pool mutex more clearly? > > Heh, think of it as a pool of mutexes, not a different type of > mutex. Instead of having 1 mutex for each object, you use a hash > table of mutexes for a set of objects. Thus, if you have 50 objects > vs. 500 objects, if you embed 1 mutex for each object, you bloat > each object and have 500 locks instead of 50 locks. Using pool > mutexes, you only have N number of mutexes regardless of the number > of mutexes. Note that if pool mutexes are non-recursive, they can't > be safely used when you might have more than one object of a given > set locked at a time. For example, process locks are the only object > we do this with currently. Pool mutexes are evil, if not implemented exactly right, and "exactly right" will vary over time. We need only look at the allocation unit optimization for things like struct socket allocations, which weren't updated when kevent came in and changed the size of the structure and therefore made the previous optimal cluster allocation block pessimal instead. Pool mutexes have the same problem that the fixed hash size for TCP connections has, in that you end up with relatively large collision domains when you get to a relatively large number of objects being hashed. Increasing the hash is not an answer, since it means that the default tuned case tries to handle the max for everything and ends up taking up so much memory you get the max for nothing. You might be able to keep a "pool ratio"; e.g. "for every N objects, there will be 1 mutex bucket", but then you get into the problem of refactoring the existing buckets. There is also the issue of collision domain; we tend to see this with an incredible number of client connections to HTTP servers with the in_pcbhash code (to keep the same example), because the hash values for port 80 on a particular IP tend to be pretty limited. In other words, I think that you will run into locality issues which will give you a hash that results in a particular bucket being inordinately busy, while another one is idle. Unless you address the locality balancing issue up front, it is a bad idea to use this for mutexes for objects, even if each object type gets its own mutex pool to avoid collision multiplication when multiple object types are referenced from the same pool. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3BF085EC.AEE7DE9C>