Date: Mon, 24 Nov 2014 00:14:35 +0100 From: Mateusz Guzik <mjguzik@gmail.com> To: freebsd-arch@freebsd.org Subject: rarely changing process-wide data vs threads Message-ID: <20141123231435.GA32084@dft-labs.eu>
next in thread | raw e-mail | index | archive | help
Currently we have some things frequently accessed which require locking, even though they very rarely change. This includes: - cwd, root, jdir vnodes - resource limits File lookup typically requires us to vref and unref cwd and root dir and locking filedesc lock shared which competes with fd open/close in other threads. Any resource limit checks requires taking PROC_LOCK, which is an exclusive lock. Turns out we already have a nice solution which only needs some minor refining and it was used to manage credentials: Each thread has a reference on active credentials and has its own pointer. When credentials are updated, a new structure is allocated and threads check that they got the right pointer on syscall boundary. If they got the wrong one, they lock PROC_LOCK and update. We can make this more general to suit other needs with an introduction of 'generation' counter and optionally an rwlock instead of using PROC_LOCK. If 'generation' is unequal to what is set in the process, at least one of creds/dirs/rlimits/$something needs updating and we can take the lock and iterate over structs. This may pose some concern since it may seem this introduces a window where given thread uses stale data while a concurrently executing thread uses new one. This window is already present for all users that I can see. During file lookups filedesc lock is only temporarily held (and current code even has a possible use after free since it does not start with refing root vnode while fdp is locked so it can be freed/recycled). resource limits are inherently racy anyway. proc lock is held only for a short them to read them, that's it. As such, I don't believe this approach introduces any new windows (although it extends already existing ones). When it comes to implementation of this concept for dir vnodes, one would need to split current struct filedesc. chdir in threaded processes would be more expensive since new struct would have to be allocated and vnodes vrefed, but chdirs are way less frequent than lookups so it should be worth it anyway. There is also a note on filedescs shared between processes. In such cases we would abandon this optimisation (dir struct can have a flag to note cow is not suitable and lookups need to vref like they do now). Comments? -- Mateusz Guzik <mjguzik gmail.com>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141123231435.GA32084>