Date: Sun, 23 Nov 2014 18:15:58 -0800 From: Rui Paulo <rpaulo@me.com> To: Mateusz Guzik <mjguzik@gmail.com> Cc: freebsd-arch@freebsd.org Subject: Re: rarely changing process-wide data vs threads Message-ID: <7683D4D1-9458-48D1-A4DF-602E2C4D13C2@me.com> In-Reply-To: <20141123231435.GA32084@dft-labs.eu> References: <20141123231435.GA32084@dft-labs.eu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Nov 23, 2014, at 15:14, Mateusz Guzik <mjguzik@gmail.com> wrote: >=20 > Currently we have some things frequently accessed which require > locking, even though they very rarely change. >=20 > This includes: > - cwd, root, jdir vnodes > - resource limits >=20 > File lookup typically requires us to vref and unref cwd and root dir = and > locking filedesc lock shared which competes with fd open/close in = other > threads. >=20 > Any resource limit checks requires taking PROC_LOCK, which is an > exclusive lock. >=20 > Turns out we already have a nice solution which only needs some minor > refining and it was used to manage credentials: >=20 > Each thread has a reference on active credentials and has its own > pointer. When credentials are updated, a new structure is allocated = and > threads check that they got the right pointer on syscall boundary. If > they got the wrong one, they lock PROC_LOCK and update. >=20 > We can make this more general to suit other needs with an introduction > of 'generation' counter and optionally an rwlock instead of using > PROC_LOCK. If 'generation' is unequal to what is set in the process, > at least one of creds/dirs/rlimits/$something needs updating and we = can > take the lock and iterate over structs. Right, this is the same model used by the routing table. > This may pose some concern since it may seem this introduces a window > where given thread uses stale data while a concurrently executing = thread > uses new one. Likewise there's a small race for the networking stack. > This window is already present for all users that I can see. >=20 > During file lookups filedesc lock is only temporarily held (and = current > code even has a possible use after free since it does not start with > refing root vnode while fdp is locked so it can be freed/recycled). >=20 > resource limits are inherently racy anyway. proc lock is held only for = a > short them to read them, that's it. >=20 > As such, I don't believe this approach introduces any new windows > (although it extends already existing ones). I agree. > When it comes to implementation of this concept for dir vnodes, one > would need to split current struct filedesc. chdir in threaded = processes > would be more expensive since new struct would have to be allocated = and > vnodes vrefed, but chdirs are way less frequent than lookups so it > should be worth it anyway. I agree. A lookup is a different operation but most of the time a chdir = is followed by a lookup, so if we optimise the lookup case the end = result might still be better. > There is also a note on filedescs shared between processes. In such > cases we would abandon this optimisation (dir struct can have a flag = to > note cow is not suitable and lookups need to vref like they do now). Are you talking about your optimisation or something that's already = there? -- Rui Paulo
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7683D4D1-9458-48D1-A4DF-602E2C4D13C2>