Date: Wed, 1 Jul 1998 18:22:15 +0000 (GMT) From: Terry Lambert <tlambert@primenet.com> To: dbj@iglou.com (David E. Brooks Jr) Cc: tlambert@primenet.com, hackers@FreeBSD.ORG Subject: Re: mlock()/mclear (was Re: Unsupport calls) Message-ID: <199807011822.LAA29206@usr01.primenet.com> In-Reply-To: <Pine.BSF.3.96.980630190503.929A-100000@localhost> from "David E. Brooks Jr" at Jun 30, 98 07:32:00 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> > It is trivial to implement p/v semaphores using __asm__ to generate > > single instruction spinlocks; since the VM/buffer cache is unified, > > this will work on shared memory/mmap'ed files without needing a > > special system call to guaranteee lock coherency. > > Funny you should mention this. I was poking around the man page to > mmap() the other day looking for any hints on a "standard" way to > perform inter-process synchronization when sharing memory. This led > me to read the 'newvm' paper, specifically about > mlock()/mclear(). Well, to make a short story even shorter, I ended up > having a brief e-mail exchange with David Greenman where he said in my > mail batch this morning "...a bright beginner could tackle it." > > So, I figured I'd try and find out how bright I really am <grin>. Look at the SMP locking code; it handles all of this. Specifically, look at the big giant kernel reentrancy lock. > I haven't gotten very far (I'm re-reading relevant portions of _The > Design and Implementation of 4.4 BSD Operating System_ and lots of > stuff in section 9 right now), but I do have one question right off > the bat. Both mmap(2) and the newvm paper make mention of the > MAP_HASSEMAPHORE flag; Why is (or was) this necessary? All that comes > to mind is multi-processor systems and keeping any copies of that > page of memory synchronized. SMP synchornization is automatic, via IPI (Inter Processor Interrupt) as part of the APIC (Advanced Programmable Interrupt Controller) that is built into the CPU. Intel supports full MESI cache coherence. The place you need MAP_HASSEMAPHORE is when you need to take action to synchronize the memory. For example, the multiprocessor BeBox (and most SMP capable Apple PPC hadrware not running the PPC620) designs remove the L2 cache, and use the cache notification lines for coherency signalling. This is an MEI, not a MESI implementation. Because of this, memory that is shared between processors has to be updated like so in order to ensure cache invalidation: 1) Get a mapping for the memory 2) Mark it non-writeable 3) Take the write fault 4) Do the operation anyway (in the trap handler, using a lookaside) 5) Assert the "L2 cache changed" to invalidate the L1 cache on the other processor(s). For a non-unified VM and buffer cache, it means that you must echo the changes between the buffer cache and the VM immediately. In general, you use the same trick. For many System V shared memory (and for BSD's without a unified cache), you would use the MAP_HASSEMAPHORE flag to notify the kernel. This lets the kernel be lazy (ie: it late binds the updates from the VM to the buffer cache, and vice versa), unless it has no choice in the matter, which is much less expensive. You can actually look at the 386 support code for kernel write faults; the "copyout" function (called from uiomove) has to do similar juggling in order to cause a SEGV if the user space page it is copying out to is not present. This is because you can't get a write fault on a 386 in kernel (supervisor) mode for an unwriteable page. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199807011822.LAA29206>