Date: Thu, 18 Dec 2014 11:38:25 +1100 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Jilles Tjoelker <jilles@stack.nl> Cc: arch@freebsd.org Subject: Re: Change default VFS timestamp precision? Message-ID: <20141218104134.W15347@besplex.bde.org> In-Reply-To: <20141217205520.GB10644@stack.nl> References: <201412161348.41219.jhb@freebsd.org> <20141217085846.G1087@besplex.bde.org> <20141217205520.GB10644@stack.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 17 Dec 2014, Jilles Tjoelker wrote: > On Wed, Dec 17, 2014 at 10:43:55AM +1100, Bruce Evans wrote: >> On Tue, 16 Dec 2014, John Baldwin wrote: >>> I'm less sure of what to do for other cases such as i386/amd64 when >>> not using TSC, or on other platforms. OTOH, perhaps you aren't >>> doing lots of heavy I/O access on a system with a slow timecounter >>> (or if you are doing heavy I/O, slow timecounter access won't be >>> your bottleneck)? > >> File times are rarely updated since the updates are normally delayed (except >> in devfs, where accurate times are least needed so caching would be most >> beneficial). But the caching means that increasing the timestamp precision >> won't help much to fix the problem with make. Consider build activity >> where there are a lot of writes which produce cached mtimes. Some time >> later, make runs and causes these times to be updated by stat()ing the >> files. It then sees times that are about the same, and likely to be out >> of order with respect to the actual writes. Extra precision actually >> makes the problem worse: with seconds precision, make usually sees the >> same time for all files, but with nanoseconds precision it sees its own >> stat() order. The extra precision is only helpful when there is a >> certain serialization of the stat()s relative to the build operations. > > This would be strange and indeed it does not work that way. Per POSIX, > timestamps marked for update must also be updated when the file ceases > to be open by any process; FreeBSD updates them on any file close > (VOP_CLOSE). Therefore, the timestamps reflect at latest the time the > files were closed, when looked at after the writing program is done. Yes, there is also an update on [last] close. (It is usually only on last close -- e.g., ffs's VOP_CLOSE only does it if v_usecount > 1.). > The problems you describe only occur when the affected files remain > open, which is not the case in usual build processes. Only the problem with seeing the stat() order, and that problem is very small (all the problems are small, but I'm trying to think of them all). Consider some cc -c operations in parallel. Make has to wait for these cc's to exit. If close() didn't update, then make could stat the produced .o files to force the update. The resulting times are unordered relative to each other, so must not be used for for determining out-of-dateness of the generated files relative to each other, but it is hard to think of a valid makefile where these files depend on each other. (They shouldn't be built in parallel if they do depend on each other.) For directory and metadata operations, there is often no last close to force the update. The nfs problems mentioned in this thread are mostly for directories. nfs asks for directory times using GETATTR or similar. Thus is like stat(). It will update the times, but often just to the current time. This is enough for determining if the directory changed, provided the clock wasn't stepped backwards and has a fine enough granularity. It is useless for inter-file comparisons, since unlike for make, nfs cannot control the production or timestamp update of other files. Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141218104134.W15347>