Date: Sat, 13 Mar 2021 17:29:39 +0100 From: Alexander Lochmann <alexander.lochmann@tu-dortmund.de> To: Konstantin Belousov <kostikbel@gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: [RFC] Understanding the locking of struct buf Message-ID: <48913698-d2e8-9721-ee1a-4828a9265e55@tu-dortmund.de> In-Reply-To: <YEy3JET6Lx7BkJP3@kib.kiev.ua> References: <49130618-349a-bfc7-6d26-0c3763904dc5@tu-dortmund.de> <YEwx0WdSuJpzhONL@kib.kiev.ua> <c49dc72a-da0f-a089-7e93-e4e54d0c03eb@tu-dortmund.de> <YEy3JET6Lx7BkJP3@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On 13.03.21 13:59, Konstantin Belousov wrote: > What consistency? If you are talking about multithreading memory model > as expected by the FreeBSD kernel, look at atomic(9). It has assumptions > like atomicity of naturally-aligned word-sized integer accesses written > out explicitly. > Yeah, I think that's what I meant. Thx! In sys/sys/buf.h, it says "All fields are protected by the buffer lock except those marked [...]". How does this fit together with atomic(9)? If I had to implement some extension to vfs_bio.c, for example, I would follow those locking rules without knowing that I may access some fields without any lock. Shouldn't that particular piece of documentation be updated? For example: b_bcount /* w: b_lock, r: no lock needed */ >> In FreeBSD, do I have to use lock X in any case except Y and Z? >> Or is it the other way round: Do I need no lock at all except for case X >> and Y? > I do not understand this question(?). > I'm sry. Maybe my question is just nuts.>> >>> Are you reporting a bug or just asking about LK_KERNPROC. Lockmgr >>> (kern_lock.c)is careful enough to handle LK_KERNPROC owner specially. In >>> particular, it does not report unlock of such lock to witness. >> First of all, I want to understand how things in FreeBSD work. >> >From what I understand now: When setting up an asynchronous IO request, >> buf.b_lock is taken by thread X. Later on LK_KERNPROC is used to hand >> over the ownership to the kernel. The lock itself is still acquired. > The lock is acquired in the sense that nobody else can take the buffer' > lock until the call to lockmgr(LK_RELEASE). So can we safely assume, for our analysis, that the b_lock is still acquired during bufdone and the other related fns? But from this point, there > is no thread owning the lock. Consider that the lock was converted to > the 1-counting semaphore.I'm not sure. What do you mean bei 1-counting semaphore? > >> The completion of the IO job is performed in the kernel's context, which >> releases buf.b_lock in brelse(). > The completion for async IO is performed in the context of some other > thread, typically either geom io up thread, or direct completion thread > of some disk driver. This is why this somewhat strange LK_KERNPROC > business is needed at all. > > For sync IO, that thread only signals original thread that io completed. > In this case, no LK_KERNPROC trick is performed. > That is clear to me. >> So there is no explicit lock call in the latter context, is it? > No lock call, but there is an unlock. Are you talking about BUF_UNLOCK()? > Assuming you wrote the stack bottom-up, this is exactly what I wrote above: > xpt_done_td is CAM IO completion thread, and it performs actions after hw > informed that io request (bio) was completed. > Thx. >> allocbuf > When brelse() notes that buffer was marked as 'no cache', it demolishes > the buffer right after async io finishes. Perhaps this is the case that > you observed. Yeah, but maybe our approach is just inaccurate. Due to the WITNESS_UNLOCK() call in __lockmgr_disown(), we assume the b_lock has already been released. - Alex -- Technische Universität Dortmund Alexander Lochmann PGP key: 0xBC3EF6FD Otto-Hahn-Str. 16 phone: +49.231.7556141 D-44227 Dortmund fax: +49.231.7556116 http://ess.cs.tu-dortmund.de/Staff/al
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48913698-d2e8-9721-ee1a-4828a9265e55>