Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 3 Mar 2017 20:21:43 +1100
From:      Peter Jeremy <peter@rulingia.com>
To:        "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Cc:        Alan Somers <asomers@freebsd.org>, Ngie Cooper <yaneurabeya@gmail.com>, Subbsd <subbsd@gmail.com>, freebsd-hackers <freebsd-hackers@freebsd.org>, freebsd-current Current <freebsd-current@freebsd.org>
Subject:   Re: effect of strip(1) on du(1)
Message-ID:  <20170303092143.GM4503@server.rulingia.com>
In-Reply-To: <201703030619.v236JAph067805@pdx.rh.CN85.dnsmgr.net>
References:  <CAOtMX2hbvnDqBWry4GNX=9-1nqEgcF8PwFZLQS_EaX2nkZsPRQ@mail.gmail.com> <201703030619.v236JAph067805@pdx.rh.CN85.dnsmgr.net>

next in thread | previous in thread | raw e-mail | index | archive | help

--LpQ9ahxlCli8rRTG
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2017-Mar-02 22:19:10 -0800, "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.=
dnsmgr.net> wrote:
>> du(1) is using fts_read(3), which is based on the stat(2) information.
>> The OpenGroup defines st_blocksize as "Number of blocks allocated for
>> this object."  In the case of ZFS, a write(2) may return before any
>> blocks are actually allocated.  And thanks to compression, gang
=2E..
>My gut tells me that this is gona cause problems, is it ONLY
>the st_blocksize data that is incorrect then not such a big
>problem, or are we returning other meta data that is wrong?

Note that it's st_blocks, not st_blocksize.

I did an experiment, writing a (roughly) 113MB file (some data I had
lying around), close()ing it and then stat()ing it in a loop.  This is
FreeBSD 10.3 with ZFS and lz4 compression.  Over the 26ms following the
close(), st_blocks gradually rose from 24169 to 51231.  It then stayed
stable until 4.968s after the close, when st_blocks again started
increasing until it stabilized after a total of 5.031s at 87483.  Based
on this, st_blocks reflects the actual number of blocks physically
written to disk.  None of the other fields in the struct stat vary.

The 5s delay is presumably the TXG delay (since this system is basically
unloaded).  I'm not sure why it writes roughly =BD the data immediately
and the rest as part of the next TXG write.

>My expectactions of executing a stat(2) call on a file would
>be that the data returned is valid and stable.  I think almost
>any program would expect that.

I think a case could be made that st_blocks is a valid representation
of "the number of blocks allocated for this object" - with the number
increasing as the data is physically written to disk.  As for it being
stable, consider a (hypothetical) filesystem that can transparently
migrate data between different storage media, with different compression
algorithms etc (ZFS will be able to do this once the mythical block
rewrite code is written).

--=20
Peter Jeremy

--LpQ9ahxlCli8rRTG
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQJ8BAEBCgBmBQJYuTWnXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFRUIyOTg2QzMwNjcxRTc0RTY1QzIyN0Ux
NkE1OTdBMEU0QTIwQjM0AAoJEBall6Dkogs0iJkP/R2TZlwppRH4SdrvGi9QddPZ
ZRZWbK9Cx0PDn26s7IG8u1kuCuXRPcMuz+GH7BZ/zf+GhGzpbVfTzUV590S/0QMP
/9VLpmqX7/6oR0RVH5iPZZ6fI1WNuycbeipr8wKXKfkc3B8zeJfVeFLyWVCOkbVC
MMqcn/1fwRIyYLbjeJfJvgKrKNiB4ubhVOz+TnASXm1Z/SWavyMmX2nOOA8OTxUe
CUGmbu5533NYlkLiBT7Jm+hcgiUwjLyzj0cg5yEgeR8CE9uCvqPma4sHpseVrBWl
95nyuR6mxfwRaUyJtcwbZNgE6f+QR3yOlQudK4zo50BTfHxnk4swwz4MS+VXoz6o
OprH9VahjO5RV5Gbo6IMSOYZBqkRxgov8xhFFG+IeZEeWMNS9+EEMYGOEW+cbPdT
pNOtatqbGDG7Q0q9rEIAP9t6OpLjTsExxomxBmWaLzuDz5Gn/qeAIp3ZtqLLJP37
MP8OU8QO5g82bdETk+o8gZ3zb+2IwWJLCU4iRoWZuiqbDRYbuX9pgfIAb4E91aOU
hEl5pQBOq0BZC9Wx/vFdFmJYZsTdxg7JDl4QtV50sno9OHCJ5LtLrU5Ir5gS+KeK
e6M7x6kyCTEJkfuMvBHBYrgeEZiN+G4TVuLBxDV/5YdNcCun1VpqvCFRe86WB8RX
f366wz6bQC3BuFA41ZV+
=0Gmi
-----END PGP SIGNATURE-----

--LpQ9ahxlCli8rRTG--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170303092143.GM4503>