Date: Sat, 7 Feb 2015 00:18:12 +0100 From: Jilles Tjoelker <jilles@stack.nl> To: Bruce Evans <brde@optusnet.com.au> Cc: arch@freebsd.org Subject: Re: Change default VFS timestamp precision? Message-ID: <20150206231812.GA13408@stack.nl> In-Reply-To: <20141217105256.J1490@besplex.bde.org> References: <201412161348.41219.jhb@freebsd.org> <20141216233844.GA1490@stack.nl> <20141217105256.J1490@besplex.bde.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Dec 17, 2014 at 11:39:37AM +1100, Bruce Evans wrote:
> On Wed, 17 Dec 2014, Jilles Tjoelker wrote:
> > Although some breakage may be caused, increasing precision sounds fine
> > to me, but only to the level of microseconds, since there is no way to
> > set a timestamp to the nanosecond (this would be futimens/utimensat). It
> > is easy to be surprised when cp -p creates an file that appears older
> > than the original.
> As I said in too many more words in another reply.
> > To avoid cross-arch surprises with applications that use
> > second-resolution APIs, either all or no architectures should generate
> > timestamps more accurate than seconds.
> That might be a feature for finding bugs. And it doesn't work for
> avoiding them, because foreign file systems may have uncontrollable
> timestamp precision. E.g., a foreign nfs server.
Architecture-dependent bugs are not nice. Bugs specific to foreign file
systems are unavoidable.
> > There is no benefit for the particular case of make(1), since it only
> > uses timestamps in seconds.
> Urk, I now rememeber seeing that 15-20 years ago and deciding that the
> problem with timestamps for make couldn't be very large.
:)
> > A quick grep also shows various calls to utime(3) with a non-NULL timep
> > argument. The ones in contrib/bmake and usr.bin/make definitely look
> > incorrect.
> Do you mean that they are incorrect because they use utime(3) and not
> utimes(3), or just because they use the non-NULL timep? The latter
> is just a style bug in at least usr.bin/make. Since it hasn't
> caught up with the existence of utimes(3), it is reasonable for it
> to also have not caught up with the existence of the non-NULL timep.
> It just wants to set the time to the current time. It uses its own
> copy of the current time for this.
Calling any utime/utimes/utimens function with the current time subverts
vfs.timestamp_precision. This may cause timestamps set by different
applications to compare differently from provable ordering. For example,
if "make -t" finishes within one second, a file created before it was
started might have a timestamp 1423261861.1 while files touched by "make
-t" will have timestamp 1423261861.0.
Also, with NFS it is better to use the server's clock for timestamps, so
timestamps in the same filesystem are consistent.
Calling a utime/utimes function with a timestamp obtained from stat
loses precision.
If the timestamp comes from somewhere else, that other place (e.g.
network protocol, archive file) may cause unavoidable precision loss.
Calling utime() with a NULL timep is not inherently wrong, although
futimes() or futimens() is preferable if the file is open.
> Perhaps this is technically better too -- it makes "make -t" touch
> everything with the same time.
There is no need for "make -t" to touch everything with the same time if
it processes files in dependency order.
> This reminds me of related bugs in touch(1):
> - touch also tries first with a non-NULL timep. So it uses its own idea
> of the current time. This is normally the one returned a little in
> the past by gettimeofday(). It has microseconds resolution. This
> normally succeeds, giving a result that is inconsistent with the
> one that would result from using a NULL timep, since the kernel now
> honors vfs.timestamp_precision and that is not usually 2.
Yes, this is wrong the same way as in "make -t".
> - sometimes the first try fails. Then touch retries with a NULL timep,
> if the semantics allow this. Sometimes this succeeds.
Another bug: it retries with a NULL timep even if -a or -m is given,
requesting only one of the timestamps be updated.
The new functionality in futimens/utimensat allows doing "touch -a" and
"touch -m" with the current time properly (setting one timestamp to
vfs_timestamp and leaving the other timestamp unchanged). As with the
old methods, only the owner of the file and root can do this.
The below patch fixes these problems.
> - sometimes both tries fail due to permissions problems. Then touch
> used to retry using read() and write(). Someone removed this, with
> the justification that all file systems support utimes(). But this
> is broken, since touching for read requires only read permission,
> while utimes() requires some sort of write permission. For touching
> for write, I don't see any problem now except for the above ordering
> problems: a NULL timep should be tried first since it requires fewer
> permissions and gives the best semantics; if it fails then there is
> no point in retrying with a non-NULL timep.
"touch -a" with the current time could be done using read() on regular
files (and readdir() on directories). This only accidentally worked
using the old readwrite code. If the filesystem is mounted read-only or
noatime, read() does not update the timestamp but this situation cannot
be reliably distinguished from a coarse vfs.timestamp_precision on a
file that had already been read recently. An fstat() call should follow
anyway to force the timestamp to be updated like utime calls do.
> The non-NULL timep in make gives the following bug since make doesn't
> retry with a NULL timep: make -t fails if the files ar writeable but
> not owned.
This is best fixed by always using a NULL timep.
Index: usr.bin/touch/touch.c
===================================================================
--- usr.bin/touch/touch.c (revision 277645)
+++ usr.bin/touch/touch.c (working copy)
@@ -76,8 +76,8 @@
myname = basename(argv[0]);
Aflag = aflag = cflag = mflag = timeset = 0;
atflag = 0;
- if (clock_gettime(CLOCK_REALTIME, &ts[0]) == -1)
- err(1, "clock_gettime(CLOCK_REALTIME)");
+ ts[0].tv_sec = ts[1].tv_sec = 0;
+ ts[0].tv_nsec = ts[1].tv_nsec = UTIME_NOW;
while ((ch = getopt(argc, argv, "A:acd:fhmr:t:")) != -1)
switch(ch) {
@@ -152,6 +152,11 @@
ts[1] = ts[0];
}
+ if (!aflag)
+ ts[0].tv_nsec = UTIME_OMIT;
+ if (!mflag)
+ ts[1].tv_nsec = UTIME_OMIT;
+
if (*argv == NULL)
usage(myname);
@@ -183,11 +188,6 @@
continue;
}
- if (!aflag)
- ts[0] = sb.st_atim;
- if (!mflag)
- ts[1] = sb.st_mtim;
-
/*
* We're adjusting the times based on the file times, not a
* specified time (that gets handled above).
@@ -203,26 +203,9 @@
}
}
- /* Try utimensat(2). */
if (!utimensat(AT_FDCWD, *argv, ts, atflag))
continue;
- /* If the user specified a time, nothing else we can do. */
- if (timeset || Aflag) {
- rval = 1;
- warn("%s", *argv);
- continue;
- }
-
- /*
- * System V and POSIX 1003.1 require that a NULL argument
- * set the access/modification times to the current time.
- * The permission checks are different, too, in that the
- * ability to write the file is sufficient. Take a shot.
- */
- if (!utimensat(AT_FDCWD, *argv, NULL, atflag))
- continue;
-
rval = 1;
warn("%s", *argv);
}
@@ -239,7 +222,7 @@
int yearset;
char *p;
/* Start with the current time. */
- now = tvp[0].tv_sec;
+ now = time(NULL);
if ((t = localtime(&now)) == NULL)
err(1, "localtime");
/* [[CC]YY]MMDDhhmm[.SS] */
@@ -301,7 +284,7 @@
time_t now;
struct tm *t;
/* Start with the current time. */
- now = tvp[0].tv_sec;
+ now = time(NULL);
if ((t = localtime(&now)) == NULL)
err(1, "localtime");
--
Jilles Tjoelker
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150206231812.GA13408>